none
Sending and returning a continuous large stream in one WCF operation/call RRS feed

  • Question

  • Hi,

    This question is related to two others that I posted in trying to come up with a working implementation:


    https://social.msdn.microsoft.com/Forums/vstudio/en-US/98e1e8f3-2934-446f-b56e-82d4e0ff0a46/how-do-i-call-client-operation-for-streaming-service-with-messagecontract-that-gets-and-returns-a?forum=wcf#d5d32cc1-60ad-4d21-9a6e-41c9c74cc70e

    https://social.msdn.microsoft.com/Forums/vstudio/en-US/fc4ddd03-1caa-445d-9596-b1344f944a7e/why-is-my-sessionfulnettcpbinding-streaming-service-instance-not-retaining-session-state?forum=wcf#fc4ddd03-1caa-445d-9596-b1344f944a7e


    I would like to send a stream of data(huge file > 2GB), to a WCF service, process it then return the processed data as a stream, without buffering the entire stream in memory then sending it back.

    I know the traditional approach of streaming data in and out(outside WCF) involves

    1. At the consuming end, sending a stream to a void method with an input Stream and an output Stream as parameters.
    2. At the consuming end, also having a receiving stream to get the incoming,processed output Stream
    3. in the method, processing that input Stream then writing the processed bytes to the output Stream
    4. those processed bytes are what is received through the output Stream

    This way, the stream flow is not broken.

    E.g from a Microsoft sample:

     void CopyStream(System.IO.Stream instream, System.IO.Stream outstream)
            {
                //read from the input stream in 4K chunks
                //and save to output stream
                const int bufferLen = 4096;
                byte[] buffer = new byte[bufferLen];
                int count = 0;
                while ((count = instream.Read(buffer, 0, bufferLen)) > 0)
                {
                    outstream.Write(buffer, 0, count);
                }
            }
    I would like to do the same, only that it will be a return type. Is that possible? How do I do it?

    Using WCF, you cannot have more than one parameter(or message contract object with property) of Stream type when you want to use transferMode = "Streaming"

    Something like this:

    Stream CopyStream(System.IO.Stream instream) { //read from the input stream in 4K chunks //and save to output stream const int bufferLen = 4096; byte[] buffer = new byte[bufferLen]; int count = 0;

    //Some operation to generate and ensure a continuous outstream

    Stream outsream = //Some operation to generate and ensure a continuous outstream while ((count = instream.Read(buffer, 0, bufferLen)) > 0) { outstream.Write(buffer, 0, count); } }

    -----

    I also tried the following for the same requirement, which failed. If anyone knows why please share your advice?

    I also tried NetTcpBinding with SessionMode set to SessionMode.Allowed, hoping to have a session that I can start,send the stream data to the service, get the results in a separate stream then using the OperationContext, retrieve whatever property will be associated with with that service instance. But it did not retain the session information, see below:

    According to MSDN documentation I should also set InstanceContextMode = InstanceContextMode.PerSession and ConcurrencyMode = ConcurrencyMode.Multiple (see last paragraph)

    Contract:

        ServiceContract(SessionMode = SessionMode.Allowed)]
        public interface IMyService
        {
            [OperationContract]
            void Init(StreamMetaData metaData);
     
            [OperationContract]
            StreamMetaData GetMetaData();
     
        }
    Implementation


        [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession,ConcurrencyMode = ConcurrencyMode.Multiple)]
        public class MyService : IMyService
        {
            private StreamMetaData _streamMetaData;
            private StreamMetaData StreamMetaData 
            {
                get { return this._streamMetaData; }
                set { this._streamMetaData= value; }
     
            }
     
            public void Init(StreamMetaData metaData)
            {
                if (metaData == null)
                    throw new ArgumentNullException("metaData");
                try
                {
                    this.StreamMetaData = metaData;
                }
                catch (Exception ex)
                {
                    throw ex;
                }
     
            }
     
            public StreamMetaData GetMetaData()
            {
                return this.StreamMetaData;
     
            }
        }
    Client Side
            MyServiceNetTcp.MyServiceClient client= new MyServiceNetTcp.MyServiceClient ();
     
            client.Open();
     
            //metaData is an instance of StreamMetaData initialised somewhere in the code
            client.Init(metaData);
     
            var currentMetaData = client.GetMetaData(); //Returns null







    • Edited by kenchanX Monday, March 16, 2015 4:45 AM
    Saturday, March 14, 2015 11:53 AM

Answers

  • Hello kenchanX,

    Try this approach:

    1. Create two methods one for sending and one for receiving large stream of data using Transfer mode Streamed which you already know about.
    2. Now on client side call could call the sending method async and adding an event on async completed which will be called when your method has finished sending the data. Please refer this article on how to achieve this. https://msdn.microsoft.com/en-us/library/ms730059(v=vs.110).aspx
    3. Now on async completed call the async method to receive the large continuous stream. Because you will using Streamed Transfer mode it should not store the whole data in memory as your requirement.
    Friday, April 3, 2015 9:14 AM

All replies

  • Hi,

    Windows Communication Foundation (WCF) can send messages using either buffered or streamed transfers. In the default buffered-transfer mode, a message must be completely delivered before a receiver can read it. In streaming transfer mode, the receiver can begin to process the message before it is completely delivered.

    To stream data, the OperationContract for the service must satisfy two requirements:

    1. The parameter that holds the data to be streamed must be the only parameter in the method. For example, if the input message is the one to be streamed, the operation must have exactly one input parameter. Similarly, if the output message is to be streamed, the operation must have either exactly one output parameter or a return value.
    2. At least one of the types of the parameter and return value must be either Stream, Message, or IXmlSerializable.

    For more information, you could refer to:

    https://msdn.microsoft.com/en-us/library/ms789010%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396

    https://msdn.microsoft.com/en-us/library/ms731913(v=vs.110).aspx

    Regards

    Monday, March 16, 2015 2:11 AM
    Moderator
  • Hi, Shawn. I am well aware of that( refer to my code. I have also added two past, related questions). But that isnt my question

    • Edited by kenchanX Monday, March 16, 2015 4:46 AM
    Monday, March 16, 2015 4:23 AM
  • Hello kenchanX,

    Try this approach:

    1. Create two methods one for sending and one for receiving large stream of data using Transfer mode Streamed which you already know about.
    2. Now on client side call could call the sending method async and adding an event on async completed which will be called when your method has finished sending the data. Please refer this article on how to achieve this. https://msdn.microsoft.com/en-us/library/ms730059(v=vs.110).aspx
    3. Now on async completed call the async method to receive the large continuous stream. Because you will using Streamed Transfer mode it should not store the whole data in memory as your requirement.
    Friday, April 3, 2015 9:14 AM