locked
UploadOperation executed twice RRS feed

  • Question

  • Hi,

    I'm uploading an image to the server using the following code:

    string filename = "TestImage.png";
    
    List<storagefile> files = new List<storagefile>();
    StorageFolder localFolder = ApplicationData.Current.LocalFolder;
    StorageFile file = await localFolder.GetFileAsync(filename);
    files.Add(file);
    
    // Display upload progress.
    Action<UploadOperation> uploadProgress = async (upload) =>
    {
        BackgroundUploadProgress progress = upload.Progress;
    
        // Calculate the progress as %.
        double percentSent = 100;
        if (progress.TotalBytesToSend > 0)
            percentSent = progress.BytesSent * 100 / progress.TotalBytesToSend;
    
        // Display the progress.
        await this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
        {
            // Update the status.
            string str = String.Format("{0} {3}%    ({1:N0} of {2:N0})",
                upload.Progress.Status.ToString(), progress.BytesSent, progress.TotalBytesToSend, percentSent);
            Debug.WriteLine(str);
            StatusTextBlock.Text = str;
    
            // Update the progress bar.
            UploadingProgressBar.Value = percentSent;
        });
    };
    
    CancellationTokenSource ctsTest = new CancellationTokenSource();
    
    try
    {
        // Read the URL and creds from the config.
        Uri uploadUri = new Uri("...url...");
        string userName = "...username...";
        string password = "...password...";
    
        // Create the BackgroundUploader object.
        BackgroundUploader uploader = new BackgroundUploader();
    
        // Set the credentials on the BackgroundUploader.
        PasswordCredential cred = new PasswordCredential();
        cred.UserName = userName;
        cred.Password = password;
        uploader.ServerCredential = cred;
    
        // Set the files.
        List<BackgroundTransferContentPart> parts = new List<BackgroundTransferContentPart>();
        for (int i = 0; i < files.Count; i++)
        {
            BackgroundTransferContentPart part = new BackgroundTransferContentPart("File" + i, files[i].Name);
            part.SetFile(files[i]);
            parts.Add(part);
        }
    
        // Create the UploadOperation.
        UploadOperation upload = await uploader.CreateUploadAsync(uploadUri, parts);
    
        try
        {
            // Start the upload and attach a progress handler.
            await upload.StartAsync().AsTask(ctsTest.Token, new Progress<UploadOperation>(uploadProgress));
    
            ResponseInformation response = upload.GetResponseInformation();
            Debug.WriteLine(String.Format("Completed: {0}, Status Code: {1}", upload.Guid, response.StatusCode));
        }
        catch (TaskCanceledException)
        {
            Debug.WriteLine("Upload cancelled.");
        }
    }
    catch (Exception exc)
    {
        // ... log the exception
    }
    finally
    {
        StatusTextBlock.Text = "";
        UploadingProgressBar.Value = 0.0;
    }</storagefile></storagefile>

    StatusTextBlock is a TextBlock and UploadingProgressBar is a ProgressBar control.

    The output in the Debug Window is as follows:

    Running 0%    (40 of 230,750)

    Running 99%    (230,706 of 230,750)

    Running 100% (230,750 of 230,750)

    Running 0%    (40 of 230,750)

    Running 99%    (230,706 of 230,750)

    Running 100%    (230,750 of 230,750)

    Completed 100%    (230,750 of 230,750)

    Completed: e7ebbd9d-bfaa-444f-b5a8-84d701bc7f5e, Status Code: 200

    Does anybody know why the file is uploaded twice?

    Thanks,

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com


    • Edited by ata6502 Monday, November 25, 2013 8:27 PM
    Monday, November 25, 2013 8:26 PM

Answers

  • This is interesting.

    First, I can only see the issue when I work against your server.  I think this is because my own server is local and can receive/respond much faster than a non-local server. 

    What's happening is that the initial post to the server does not contain credentials, which is normal.  The post also contains the entire image encoding embedded into the request.  The first part of the message reaches the server, but not the entire message containing the uploaded image.  The server receives the request, and instantly responds, without waiting for the entire message to come through.  The response is a 401 - not authorized - which is normal when the upload site needs authentication.

    At this point, although the response has been received, the upload is still taking place on the client.  This is why you see the first upload status.  When the first upload has completed, the post must take place again, but this time it must include authentication, and it must include the image again.  This is when you see the second upload to the server.

    The size of the image helps confuse the problem as it can send the smaller image faster and in less frames than the bigger image. This is why you might see the smaller image appear to be uploaded only once. Also, I would sometimes see my own tests seem to upload twice as well but I could not reliably repeat the procedure.

    I believe that that the only real resolution to this is to pass the basic authentication to the server in the first request.  You can do this by properly encoding the username/password and passing it in as a header into the initial request.  I don't know the exact mechanism to do this for the backgroundtransfer, though.

    EDIT: BackgroundUploader has a method called "SetRequestHeader" where you can set the basic authentication header.  This won't work with NTML/Kerberos.

    Please let me know if you have any questions about this.


    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.



    Wednesday, December 4, 2013 5:10 PM
    Moderator

All replies

  • Are you stepping through the code to see if it's actually uploading the file twice?  Can you post a reproduction project, along with reproduction steps?


    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Tuesday, November 26, 2013 1:58 PM
    Moderator
  • Thanks Matt for your answer.

    Yes, I'm stepping through the code and when it gets to the line

    await upload.StartAsync().AsTask(ctsTest.Token, new Progress<UploadOperation>(uploadProgress));

    the images are uploaded twice although the StartAsync() method is called only once. I can see that because the progress bar goes from 0% to 100% and then again from 0% to 100%. Once images are uploaded, the execution continues on the next line.


    These are reproduction steps:

    1. Create a new Windows Store App project.

    2. Add a button.

    3. Add the event handler for the button:

    using System;
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.Text;
    using System.Threading;
    using System.Threading.Tasks;
    using Windows.Networking.BackgroundTransfer;
    using Windows.Security.Credentials;
    using Windows.Storage;
    using Windows.UI.Xaml;
    using Windows.UI.Xaml.Controls;
    namespace Win8Tests
    {
        public sealed partial class MainPage : Page
        {
            public MainPage()
            {
                this.InitializeComponent();
            }
            private const string TestFilename = "TestImage.png";
            private const string UploadUrlDemo = "http://yourserver/Upload.aspx";
            private const string UploadUserNameDemo = "yourdomain/yourusername";
            private const string UserPasswordDemo = "yourpassword";
            private async void TestButton_Click(object sender, RoutedEventArgs e)
            {
                List<StorageFile> files = new List<StorageFile>();
                StorageFolder localFolder = ApplicationData.Current.LocalFolder;
                StorageFile file = await localFolder.GetFileAsync(TestFilename);
                files.Add(file);
                // Display upload progress.
                Action<UploadOperation> uploadProgress = async (upload) =>
                {
                    BackgroundUploadProgress progress = upload.Progress;
                    // Calculate the progress as %.
                    double percentSent = 100;
                    if (progress.TotalBytesToSend > 0)
                        percentSent = progress.BytesSent * 100 / progress.TotalBytesToSend;
                    // Display the progress.
                    await this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
                    {
                        // Update the status.
                        string str = String.Format("{0} {3}%    ({1:N0} of {2:N0})",
                            upload.Progress.Status.ToString(), progress.BytesSent, progress.TotalBytesToSend, percentSent);
                        WriteText(str);
                    });
                };
                CancellationTokenSource ctsTest = new CancellationTokenSource();
                try
                {
                    // Create the BackgroundUploader object.
                    BackgroundUploader uploader = new BackgroundUploader();
                    // Set the credentials on the BackgroundUploader.
                    PasswordCredential cred = new PasswordCredential();
                    cred.UserName = UploadUserNameDemo;
                    cred.Password = UserPasswordDemo;
                    uploader.ServerCredential = cred;
                    // Set the files.
                    List<BackgroundTransferContentPart> parts = new List<BackgroundTransferContentPart>();
                    for (int i = 0; i < files.Count; i++)
                    {
                        BackgroundTransferContentPart part = new BackgroundTransferContentPart("File" + i, files[i].Name);
                        part.SetFile(files[i]);
                        parts.Add(part);
                    }
                    // Create the UploadOperation.
                    Uri uploadUri = new Uri(UploadUrlDemo);
                    UploadOperation upload = await uploader.CreateUploadAsync(uploadUri, parts);
                    try
                    {
                        // Start the upload and attach a progress handler.
                        await upload.StartAsync().AsTask(ctsTest.Token, new Progress<UploadOperation>(uploadProgress));
                        ResponseInformation response = upload.GetResponseInformation();
                        WriteText(String.Format("Completed: {0}, Status Code: {1}", upload.Guid, response.StatusCode));
                    }
                    catch (TaskCanceledException)
                    {
                        WriteText("Upload cancelled.");
                    }
                }
                catch (Exception exc)
                {
                    WriteText(String.Format("ERROR: {0}", exc.Message));
                }
            }
            private void WriteText(string str)
            {
                Debug.WriteLine(str);
            }
        }
    }

    4. Modify the URL (UploadUrlDemo) to point to your server.

    5. Modify credentials if you use any (UploadUserNameDemo and UserPasswordDemo).

    6. Build the App.

    7. Copy a test image to the folder where the App is located (in this demo code the image is called TestImage.png).

    8. Create the Upload.aspx file on your server. This is the same uploader as in the MSDN samples:

    <%@ Page Language="C#" AutoEventWireup="true" %>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <script runat="server">
    protected void Page_Load(object sender, EventArgs e)
    {
        try
        {
            if (Request.Headers["Filename"] != null)
            {
                // Simple upload scenario.
                string fileName = Request.Headers["Filename"];
                Response.Write("Filename is " + fileName);
                string saveLocation = Server.MapPath("Data") + "\\" + fileName;
                using (System.IO.FileStream fs = new System.IO.FileStream(saveLocation, System.IO.FileMode.Create))
                {
                    Request.InputStream.CopyTo(fs);
                }
            }
            else
            {
                string formData = Request.Form["FormData"];
                for (int i = 0; i < Request.Files.Count; i++)
                {
                    var file = Request.Files[i];
                    if (file.ContentLength > 0)
                    {
                        string fileName = System.IO.Path.GetFileName(file.FileName);
                        string saveLocation = Server.MapPath("Data") + "\\" + fileName;
                        file.SaveAs(saveLocation);
                    }
                }
            }
        }
        catch (Exception ex)
        {
            Trace.Write(ex.Message);
            Response.StatusCode = 500;
            Response.StatusDescription = ex.Message;
            Response.End();
        }
    }
    </script>
    <html xmlns="http://www.w3.org/1999/xhtml">
    <head runat="server">
        <title>Upload</title>
    </head>
    <body>
        Hello
    </body>
    </html>

    10. Launch the App and press the button.

    11. You should see in the Debug Window that the image is uploaded twice.

     

    By the way, in the meantime I have tested the code against a different server with the same Upload.aspx file. It worked fine i.e. the images were uploaded only once. It means that the problem is not fully replicable and the behaviour also depends on the server/website settings. Unfortunately, I'm not in charge of the server and I don't know how it is configured.

    I would greatly appreciate if someone could point me in the right direction. At this moment I run out of ideas.

    Thanks,

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Tuesday, November 26, 2013 4:20 PM
  • Using this exact application, I cannot reproduce this problem on either Windows 8 or 8.1.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Tuesday, November 26, 2013 8:43 PM
    Moderator
  • Thanks Matt for trying. It sees the problem is related to some settings on the webserver and/or the website but I have no idea what they may be.

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Tuesday, November 26, 2013 9:25 PM
  • With some trial and error I figured out that the problem is related to the size of a file I'm trying to upload. The file is uploaded twice if its size exceeds 60 KB. I can't find any property that would allow me to set the maximum size of the uploaded file.

    Does anybody knows how to change the maximum allowed size of files for uploading by the UploadOperation class?

    Thanks,

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Wednesday, November 27, 2013 2:25 PM
  • This is weird as BackgroundUploader is meant for large uploads. I can't test this at the moment but please make sure to bump this post after the weekend and I'll dig into this more completely.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Wednesday, November 27, 2013 3:24 PM
    Moderator
  • Thanks Matt for looking at that.

    Just for the record: I have found a way to change the limit of uploaded files (web.config below). The behaviour has not changed though - the files are still uploaded twice.

    web.config

    <?xml version="1.0" encoding="UTF-8"?>
    <configuration>
        <system.web>
    	<httpRuntime maxRequestLength="102400" executionTimeout="300"/>
        </system.web>
        <system.webServer>
            <directoryBrowse enabled="true" />
            <security>
                 <requestFiltering>
                     <requestLimits maxAllowedContentLength="102400000" />
                 </requestFiltering>
            </security>
        </system.webServer>
    </configuration>

    Thanks,

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Thursday, November 28, 2013 8:49 PM
  • Hi Matt,

    I have obtained full access to the web server where the images are uploaded as well as the Upload.aspx is located. I can check properties of the website/virtual directory/folders etc.

    What properties do you think I should look at first?

    Thanks for all your support!

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Monday, December 2, 2013 2:58 PM
  • The image I used is 283 Kb and is only uploaded one time.  Please send me your project and I'll test using that.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Monday, December 2, 2013 3:51 PM
    Moderator
  • Thanks Matt.

    How should I send the project to you? I can't attach it here in the forum as it has credentials to the real web site. It is important to connect to this specific web site as when I connect to other web sites, everything works fine.

    Is there any secure place I can post the project? I also have to obtain an approval from my admin to do that.

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Monday, December 2, 2013 4:00 PM
  • You can email me:  msmall at Microsoft.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Monday, December 2, 2013 4:31 PM
    Moderator
  • This is interesting.

    First, I can only see the issue when I work against your server.  I think this is because my own server is local and can receive/respond much faster than a non-local server. 

    What's happening is that the initial post to the server does not contain credentials, which is normal.  The post also contains the entire image encoding embedded into the request.  The first part of the message reaches the server, but not the entire message containing the uploaded image.  The server receives the request, and instantly responds, without waiting for the entire message to come through.  The response is a 401 - not authorized - which is normal when the upload site needs authentication.

    At this point, although the response has been received, the upload is still taking place on the client.  This is why you see the first upload status.  When the first upload has completed, the post must take place again, but this time it must include authentication, and it must include the image again.  This is when you see the second upload to the server.

    The size of the image helps confuse the problem as it can send the smaller image faster and in less frames than the bigger image. This is why you might see the smaller image appear to be uploaded only once. Also, I would sometimes see my own tests seem to upload twice as well but I could not reliably repeat the procedure.

    I believe that that the only real resolution to this is to pass the basic authentication to the server in the first request.  You can do this by properly encoding the username/password and passing it in as a header into the initial request.  I don't know the exact mechanism to do this for the backgroundtransfer, though.

    EDIT: BackgroundUploader has a method called "SetRequestHeader" where you can set the basic authentication header.  This won't work with NTML/Kerberos.

    Please let me know if you have any questions about this.


    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.



    Wednesday, December 4, 2013 5:10 PM
    Moderator
  • Thanks Matt, it's working!  Great insight and explanation on what's going on.

    I'm aware of the SetRequestHeader method. This is the code I'm using to set the Basic Auth and it's working like a charm, thanks to you!

    // Create the BackgroundUploader object.
    BackgroundUploader uploader = new BackgroundUploader();
    string credentials = String.Format("{0}:{1}", userName, password); 
    string authorization = String.Concat("Basic ", Convert.ToBase64String(Encoding.UTF8.GetBytes(credentials))); 
    uploader.SetRequestHeader("Authorization", authorization);
    

    The image is uploaded only once :)

    Leszek


    Wiki: wbswiki.com
    Website: www.wisenheimerbrainstorm.com

    Wednesday, December 4, 2013 6:07 PM