none
Loading data into Data Lake Asynchronously

    Question

  • Can you load data asynchronously through the DataLakeStoreUploader? I am assuming that you can, but you know what they say about assuming...

    N

    Thursday, June 30, 2016 3:46 PM

All replies

  • You can upload asynchronously, I'm not aware of any issues.  Here's some sample code using BeginInvoke and EndInvoke, just in case.

    Here's the callback delegate

    public delegate void AsyncUploadDelegate(string srcFilePath, string destFilePath, bool force = true);

    And I took the sample code from the Getting started Tutorial and added the async parts to it.  You'd replace the UploadFile call at approximately line 56.

    WaitForNewline("Uploading");
    AsyncUploadDelegate uploader = new AsyncUploadDelegate(UploadFile);
    IAsyncResult result = uploader.BeginInvoke(localFolderPath + "warandpeace.txt", "/warandpeace.txt", true, null, null);
    
    while(!result.IsCompleted)
    {
      Thread.Sleep(100);
      Console.Write("Blah ");
    }
    
    uploader.EndInvoke(result);
    
    WaitForNewline("and we're done");



    Friday, July 1, 2016 11:32 PM