none
Web Service to upload large files RRS feed

  • Question

  • Hi Guys

    I'm want to create a Web Service that uploads large files (excess of 100MB). Can some help me with the best way I can do that, either C# or VB.NET.

    Thanks 

    Wednesday, January 7, 2015 12:44 PM

Answers

  • I've done this before with a web service. It wasn't that hard. As jdweng alludes to, you need a web service and a client that knows how to repeatedly transfer the file sending it in chunks.

    The web service needs to take the file name and a byte array. It needs to return the number of bytes it received. The client calls the service endpoint passing in the name of the file and an array populated with x number of bytes. This should/could be tuned based on the size of the file and available bandwidth. 10 MB is a good upper limit but that may not be practical over a really slow connection... you could potentially time-out the call. The service endpoint receives the byte array and the file name. It creates/opens the file and appends the byte array onto the end in a synchronous manner. (You need to do it synchronous to avoid any potential race conditions with multiple calls overlapping.) When the server side is done, it returns the number of bytes written to the caller. The web service should close the file handle after each call since there is no guarantee that there will be another call.

    The caller then uses the return value to increment the offset pointer into the source file. When the call returns, the caller increments into the source file by the number of bytes returned then refills the buffer with the next chunk. It then re-calls the web service sending the same file name and the new byte array. The process continues until all of the bytes in the source file have been transferred to the server.

    I uploaded 4GB disk images to a Sharepoint server doing this so it works great and totally avoids the HTTP max payload problem.

    Good luck!

    Wednesday, January 7, 2015 3:20 PM

All replies

  • You may have issues with files larger than ~20MByte.  Often there are limitations that are set that you can't modify (or by the Network server) .  You may want to consider splitting the file into 10MByte chunks and then recombine at the Web Server.

    jdweng

    Wednesday, January 7, 2015 1:59 PM
  • I've done this before with a web service. It wasn't that hard. As jdweng alludes to, you need a web service and a client that knows how to repeatedly transfer the file sending it in chunks.

    The web service needs to take the file name and a byte array. It needs to return the number of bytes it received. The client calls the service endpoint passing in the name of the file and an array populated with x number of bytes. This should/could be tuned based on the size of the file and available bandwidth. 10 MB is a good upper limit but that may not be practical over a really slow connection... you could potentially time-out the call. The service endpoint receives the byte array and the file name. It creates/opens the file and appends the byte array onto the end in a synchronous manner. (You need to do it synchronous to avoid any potential race conditions with multiple calls overlapping.) When the server side is done, it returns the number of bytes written to the caller. The web service should close the file handle after each call since there is no guarantee that there will be another call.

    The caller then uses the return value to increment the offset pointer into the source file. When the call returns, the caller increments into the source file by the number of bytes returned then refills the buffer with the next chunk. It then re-calls the web service sending the same file name and the new byte array. The process continues until all of the bytes in the source file have been transferred to the server.

    I uploaded 4GB disk images to a Sharepoint server doing this so it works great and totally avoids the HTTP max payload problem.

    Good luck!

    Wednesday, January 7, 2015 3:20 PM
  • Thanks I will consider that.
    Thursday, January 8, 2015 6:30 AM
  • Hi, thanks I get the idea, I will work on it and see how it goes.
    Thursday, January 8, 2015 6:31 AM