locked
How is large data sent over RESTful web services in .NET? RRS feed

  • Question

  • User1978547804 posted

    Hi,

    I'm designing a Web based application that needs to be available as a RESTful Web service to clients over the Internet. The size of the data to be sent as a response can be very huge ranging from 1 GB to about 50 GB. I believe it is not possible to send such a huge response over as a Web response.

    What are some possible ways in which I can send such data across? Are there any industry-standard approaches to sending such data across? Or do I need to share it on some FTP site once such a request is received?

    Appreciate any inputs on this.

    TIA,

    Jy

    Thursday, August 30, 2018 12:39 PM

All replies

  • User970350993 posted

    You can do like this 

    https://www.codeproject.com/Articles/1034347/Upload-large-files-to-MVC-WebAPI-using-partitionin

    what type of data are you going to send as response ?

    Friday, September 21, 2018 6:09 AM
  • User765422875 posted

    APIs that move heavy amounts of data <g class="gr_ gr_7 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar multiReplace" id="7" data-gr-id="7">consume</g> a lot of memory to materialize results which can lead to high memory usage, paging, and sluggish performance. With this in mind, you want to implement a streaming approach. Assuming your <g class="gr_ gr_5 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" id="5" data-gr-id="5">datasource</g> can stream results, you can keep more memory available on the server by moving data with IEnumerable or IQueryable to the client.

    The projects I'm on that require massive data ingestion leverage services like Azure Event Hubs. What you decide to do comes down to the architecture and overall system requirements.

    Friday, March 1, 2019 7:52 PM