locked
Issue with concurrent WCF service calls RRS feed

  • Question

  • Hi,

    I am having trouble making concurrent client calls to my IIS hosted WCF service.

    The service is wsHttpBinding with InstanceContextMode = InstanceContextMode.PerSession and ConcurrencyMode = ConcurrencyMode.Multiple.

    In my client I make a call to the service (which executes a time consuming method) - this call is made using a background worker thread.  This service method is running a model and periodically generating images.  Whilst this method is running I want to be able to make additional calls from my client to a service method which returns these images.  I make these additional calls on a DispatchTimer that is set to fire every 10 seconds.

    After a few succesfull attempts at picking up the images, further calls to the service seem to lock, i.e. a call is made to the service but the method doesn't appear to get called.  Only once the modelling has finished (on the background worker) does the method get called.

    To see if this was a lock issue on shared resources, I changed to code to return a string (and as such, makes no use of shared resources) but the same thing happens - a bunch of succesfull calls are made and then it locks - and only resumes once the modelling has finished.

    Any ideas why this is happening?

    Many thanks






    Friday, March 5, 2010 5:07 PM

Answers

  • Hi raptorSW,

    I got the same behavior, the GetImage can only occured 4-5 times. I'm not sure if it has limitation on RM concurrency.

    Maybe we could consider redesign the client service interaction pattern. From my understanding, we need make a lengthy call, during the length call, the service return data constantly. In this case, we could
    * Make lengthy operation as oneway. Because service instancemode is perSession, we could use a Flag field on service to indicate whether lengthy operation finished, in GetImage function, check Flag status, if lengthy operation finished, return specific data or return a custom Fault exception to let client know.

    * Use duplex service, let service send data to client directly.
    http://msdn.microsoft.com/en-us/library/ms731064.aspx

    Please have a try and let me know if you encountered problem.

    Thanks,
    Mog Liang
    • Proposed as answer by Mog Liang Friday, March 12, 2010 6:46 AM
    • Marked as answer by raptorSW Wednesday, March 24, 2010 11:47 AM
    Friday, March 12, 2010 1:30 AM

All replies

  • Hi,

    Although you have set the ConcurrencyMode = ConcurrencyMode.Multiple. You have to set higher values for maxConcurrentSessions and maxConcurrentCalls. The default values of them are 10 and 16. (I think..). Set them to higher values.

    Please have a look at this article that demonstrates about throttling.


    Regards
    Dnana
    Sunday, March 7, 2010 6:00 AM
  • Thanks for the reply.

    I have tried upping the maxConcurrentCalls from 16 to 64 but this made no difference.

    Both calls are being made on the same proxy.  The first call is the modelling run (which is time consuming and the call is not one way - hence the client thread waits until the service operation is completed) and whilst this is going on any calls to the service on the same proxy initially work but then lock (the number of successful calls prior to locking is variable).  So the maximum number of concurrent calls in this case is 2.

    Since I'm using the same proxy then presumably the maxConcurrentSessions is not affected?  I get my initial single session when I first log on to the service - multithreaded calls on this same session/proxy will not be restricted by maxConcurrentSessions (but would do if I created a new proxy at every call).

    What I do find confusing it the variability on the number of successful calls before blocking occurs - which could rule out the throttling ServiceBehaviour settings??

    Any further ideas?

    Thank you.


    Sunday, March 7, 2010 8:56 PM
  • Hi,

    Refer to this White paper
    http://support.microsoft.com/kb/821268

    The default maxWorkerThreads value is machineProcesserNumber * 2. It's possible that the running threads exceeds the limitation, and then wcf refuesd further requests. To solve the problem, you may change the default maxWorkerThreads setting, or limit the max concurrent processing jobs number.

    Thanks,
    Mog Liang
    • Proposed as answer by Anup Hosur Monday, March 8, 2010 11:17 AM
    Monday, March 8, 2010 9:33 AM

  • I tried increasing maxWorkerThreads but I'm still getting the same issue.

    Here's a bit more information.

    I've created a separate test client that consists of the following:

     static void Main(string[] args)
     {
                using (ChannelFactory<IDstService> channel = new ChannelFactory<IDstService>("DstService.service"))
                {
                    modelFinished = false;
                    channel.Credentials.UserName.UserName = "user";
                    channel.Credentials.UserName.Password = "password";
                    serviceProxy = channel.CreateChannel();
                    serviceProxy.InvokeService();              
                    BackgroundWorker bw = new BackgroundWorker();
                    bw.DoWork +=new DoWorkEventHandler(bw_DoWork);             
                    bw.RunWorkerAsync();
                    serviceProxy.DoLengthyOperation(); //This is the length modelling call - it's not oneway so execution remains here until finished
                    modelFinished = true;
                    System.Threading.Thread.Sleep(5000);  //allow bw_DoWork to finish
                    channel.Close();
                }
    }

     private static void bw_DoWork(System.Object sender,
                System.EventArgs e)
            {          
                while(true)
                {
                    Debug.WriteLine("Calling");
                    serviceProxy.GetImage();
                    Debug.WriteLine("Completed");
                    System.Threading.Thread.Sleep(4000);
                    if (modelFinished)
                        break;
                }
            }


    This is the output in the Debug window:

    Calling
    Completed
    Calling
    Completed
    Calling
    The thread 0x264c has exited with code 0 (0x0).
    The thread 0xd70 has exited with code 0 (0x0).
    Completed

    As can be seen, the first couple of calls complete and then it hangs on the third - not completing until serviceProxy.DoLengthyOperation() finishes.

    My throttling is set as follows:

     <serviceThrottling maxConcurrentCalls="100" maxConcurrentSessions="20" />

    The machine.config file maxWorkerThreads is set as follows:

    <processModel
            maxWorkerThreads="100"
            minWorkerThreads="20"
    />


    The lengthy modelling operation is performed by an external tool that is called at the service via COM.  This modelling tool is using multiple threads when modelling (which might be relevant!?).

    I'm at a loss as to what to try next!? Please help.

    Thank you.



    Monday, March 8, 2010 11:34 AM
  • Hi raptorSW,

    Since the serviceopertion "DoLengthyOperation" is two-way, why not letting it return Modeled Image directly?

    For client call block issue, note that you used sync pattern to call service on proxy, then the proxy will queue the call, send one request after another response return, even access proxy from different threads. To let proxy be able to call service concurrently, you could use Async pattern call, please see this article
    http://msdn.microsoft.com/en-us/library/ms730059.aspx

    Thanks,
    Mog Liang
    Tuesday, March 9, 2010 1:46 AM
  • Although "DoLengthOperation" is two-way, it is continuously generating images (as well as, more importantly, performing some intense modelling). 


    Anyway, I seem to now have it working although I'm not entirely sure why.  After trying a whole host of things I tried disabling Reliable Sessions and hey presto - it worked.  Now there is no blocking at all of the background worker calls.

    So could somebody explain exactly why disabling RS has solved the problem?  And is this normal behaviour?, i.e. using a wsHttpBinding, InstanceContextMode.PerSession, ConcurrencyMode = ConcurrencyMode.Multiple and reliable sessions enabled will cause sequential calls on the proxy.

    If so, is there no way to allow reliable sessions if you want to make concurrent calls on the same proxy?

    I found the following post which describes the same problem I have encountered (specifically, point 4a in the first post)

    http://social.msdn.microsoft.com/forums/en-US/wcf/thread/95daa492-f4a2-474e-9772-7b10f9f5eda2/

    Also, regarding the advise to implement async callbacks - my understanding was that they do not work with wsHttpBinding?

    Thanks.



    Tuesday, March 9, 2010 10:50 AM
  • Hi,

    As Ram said in post
    http://social.msdn.microsoft.com/forums/en-US/wcf/thread/95daa492-f4a2-474e-9772-7b10f9f5eda2/
    First sending a bunch of non-blocking messages to enlarge the Service RM transmission window, then the RM would handle messages in concurrent way.

    #"regarding the advise to implement async callbacks - my understanding was that they do not work with wsHttpBinding"
    No, this way cannot overcome the RM issue, sorry for misdirect you.


    Mog Liang
    Wednesday, March 10, 2010 2:14 AM

  • Thanks for the advice Mog.

    I do send several non-blocking calls to the service prior to starting the lengthy operation.  But this appears to make no difference.

    I haven't been able to get a clear picture in my head as to why RM is blocking - could somebody kindly explain this for me?

    And is this behaviour expected?  I have not been able to find any form of documentation stating that if a WCF service is configured as follows:

    1)  wsHttpBinding
    2)  Session.Required
    3)  InstanceContextMode = InstanceContextMode.PerSession
    4)  ConcurrencyMode = ConcurrencyMode.Multiple
    5)  Reliable Session enabled

    Then concurrent calls to the service on the same client proxy will get blocked due to the reliable session.   As said, disabling RS solves the issue - but I'd prefer to have it enabled.  But since I cannot find confirmation as to whether this is normal behaviour I cannot determine whether disabling RS is my only option (as it seems sending a bunch of non blocking calls makes no difference - at least in my case).

    Thanks.
    Wednesday, March 10, 2010 11:23 AM
  • Hi,

    Your config is alright, I could make it work on my machine, here is my test sample
    http://cid-8d29fb569d8d732f.skydrive.live.com/self.aspx/.Public/RM%5E_Blocking.zip
    Please note that, we need disable the reliable session Ordered function in case of lengthy operation blocking the later short opertions.

    Thanks,

    Mog Liang
    Thursday, March 11, 2010 3:37 AM
  • Hi Mog,

    Many thanks for the test sample.

    I ran it but it exhibited the same behaviour as my code, i.e. a few concurrent calls are successfully made but then it locks.  I disabled RS and then it worked fine.

    So it's strange it worked for you but not for me.  Could you try re-running it with a longer period of time on the DoLengthyOperationAsync - I used 30 seconds.  Five or so concurrent calls are successful and then after that they are blocked until DoLengthyOperationAsync finishes.

    I tried increasing the number of non-blocking calls (I called client.GetImage() 10 times prior to calling client.DoLengthyOperationAsync(30)) but this made no difference.

    Thanks.
    Thursday, March 11, 2010 11:28 AM
  • Hi raptorSW,

    I got the same behavior, the GetImage can only occured 4-5 times. I'm not sure if it has limitation on RM concurrency.

    Maybe we could consider redesign the client service interaction pattern. From my understanding, we need make a lengthy call, during the length call, the service return data constantly. In this case, we could
    * Make lengthy operation as oneway. Because service instancemode is perSession, we could use a Flag field on service to indicate whether lengthy operation finished, in GetImage function, check Flag status, if lengthy operation finished, return specific data or return a custom Fault exception to let client know.

    * Use duplex service, let service send data to client directly.
    http://msdn.microsoft.com/en-us/library/ms731064.aspx

    Please have a try and let me know if you encountered problem.

    Thanks,
    Mog Liang
    • Proposed as answer by Mog Liang Friday, March 12, 2010 6:46 AM
    • Marked as answer by raptorSW Wednesday, March 24, 2010 11:47 AM
    Friday, March 12, 2010 1:30 AM