none
Converting synchronous VirusTotal method to asynchronous using Rx RRS feed

  • Question

  • I'm attempting to redo a small helper tool I have that checks URL's and Files against VirusTotal to get some basic information. The code below works quite well but locks up the UI. I was told that I should look into Rx and am enjoying reading up on it but cannot seem to get my head wrapped around it. So now here is where the question comes in, what is the best way to design the following code to make it utilize Rx so that it is asynchronous and leaves my UI alone while it does it's thing. VirusTotal also utilizes multilevel JSON for responses so if anyone has a nice way of integrating that into this that would even be better.

    I've been researching this problem for awhile now and have not come up with any decent examples. I went out and bought the only Rx book and I'm starting to begin to think I have a incorrect interpretation of what Rx is or is not. Any help would be fantastic.

    class Virustotal
    {
        private string APIKey = "REMOVED";
        private string FileReportURL = "https://www.virustotal.com/vtapi/v2/file/report";
        private string URLReportURL = "http://www.virustotal.com/vtapi/v2/url/report";
        private string URLSubmitURL = "https://www.virustotal.com/vtapi/v2/url/scan";
    
        WebRequest theRequest;
        HttpWebResponse theResponse;
        ArrayList  theQueryData;
    
        public string GetFileReport(string checksum) // Gets latest report of file from VT using a hash (MD5 / SHA1 / SHA256)
        {
            this.WebPostRequest(this.FileReportURL);
            this.Add("resource", checksum);
            return this.GetResponse();
        }
    
        public string GetURLReport(string url) // Gets latest report of URL from VT
        {
            this.WebPostRequest(this.URLReportURL);
            this.Add("resource", url);
            this.Add("scan", "1"); //Automatically submits to VT if no result found
            return this.GetResponse();
        }
    
        public string SubmitURL(string url) // Submits URL to VT for insertion to scanning queue
        {
            this.WebPostRequest(this.URLSubmitURL);
            this.Add("url", url);
            return this.GetResponse();
        }
    
        public string SubmitFile() // Submits File to VT for insertion to scanning queue
        {
            // File Upload code needed
            return this.GetResponse();
        }
    
        private void WebPostRequest(string url)
        {
            theRequest = WebRequest.Create(url);
            theRequest.Method = "POST";
            theQueryData = new ArrayList();
            this.Add("apikey", APIKey);
        }
    
        private void Add(string key, string value)
        {
            theQueryData.Add(String.Format("{0}={1}", key, Uri.EscapeDataString(value)));
        }
    
        private string GetResponse()
        {
            // Set the encoding type
            theRequest.ContentType="application/x-www-form-urlencoded";
    
            // Build a string containing all the parameters
            string Parameters = String.Join("&",(String[]) theQueryData.ToArray(typeof(string)));
            theRequest.ContentLength = Parameters.Length;
    
            // We write the parameters into the request
            StreamWriter sw = new StreamWriter(theRequest.GetRequestStream());
            sw.Write(Parameters);
            sw.Close();
    
            // Execute the query
            theResponse =  (HttpWebResponse)theRequest.GetResponse();
            StreamReader sr = new StreamReader(theResponse.GetResponseStream());
            return sr.ReadToEnd();
        }
    }

    Saturday, June 9, 2012 8:38 PM

All replies

  • Rx (like standard .NET events) is single threaded by default. You will need to give Rx "guidance" on how you want it to manage concurrency.

    As you have no Rx code in the sample you have posted it is a little hard to know where you are tripping up.

    I think this is what you probably want to do in your consuming code base:

    Observable.Create<string>(
        o =>
            {
                try
                {
                    var vt = new Virustotal();
                    var report = vt.GetFileReport("checksum");
                    o.OnNext(report);
                    o.OnCompleted();
                }
                catch (Exception ex)
                {
                    o.OnError(ex);
                }
                return Disposable.Empty;
            })
        .SubscribeOn(Scheduler.NewThread)
        .ObserveOn(Scheduler.Dispatcher)
        .Subscribe(report => ViewModel.ReportResult = report);


    I am hoping to fix the "only Rx book" problem now. Introduction to Rx will be a free book and I really hope it fills the gap in the market. It is almost complete and is already available for free online. If you are new to Rx, I would suggest reading it from the start, but the main chapter you are looking for is Scheduling and Threading.

    http://www.introtorx.com/Content/v1.0.10621.0/15_SchedulingAndThreading.html#SchedulingAndThreading

    Hope that helps.

    Lee


    Lee Campbell http://LeeCampbell.blogspot.com

    Monday, June 11, 2012 10:45 AM