none
about wcf client connect timeout

    Question

  • hello,

          I am writting program using wcf,I set ReceiveTimeout = TimeSpan.FromMinutes(10),then I call service,then the client have long time not to call,when I recall service,it show following:

    The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:09:58.7343750'

    how can I know the connection status of wcf?

    Monday, June 18, 2007 2:44 PM

Answers

  • After talking with some of our devs on this issue, I think I was mistaken in saying it is the SendTimeout you were hitting.  What you really need to set is the ReceiveTimeout on the Service side binding.  Here's why:

    When using TCP, the communication is sessionful.  In general, when the Client creates a channel to the Service, it uses up some Service side resources.  The Service has a ReceiveTimeout on the binding to configure how long to allow the Client to keep a channel open to the Service.  The ReceiveTimeout doesn't have anything to do with a receive action duration.  Further, the Client doesn't know what the ReceiveTimeout is on the Service, so when it faults because of ReceiveTimeout being exceeded, all it can tell you is what the local timeouts were.  Just in case this timeout was caused by something local to the Client, the exception gives you what the SendTimeout was.

    So, to have a NetTcp Service allow a Client to keep a channel open for longer than 10 minutes, you need to increase the ReceiveTimeout on the Service side's binding.

     

    HTH,

     

    -James

    Tuesday, June 19, 2007 8:14 PM

All replies

  • You are probably hitting some session timeout.  It can be configured to values larger than the default of 10 minutes.  More info about this can be found in these posts:

    http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1489527&SiteID=1

    http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1266767&SiteID=1

     

    Are you using Reliable Sessions by any chance?  There's an inactivity timeout in the Reliable Session Binding Element.

     

    -James

    Monday, June 18, 2007 9:31 PM
  • thank you for your response, I use nettcpbinding , and  I set InstanceContextMode to InstanceContextMode.PerCall and reliable to false
    Tuesday, June 19, 2007 2:07 AM
  • I believe you are hitting the Send Timeout.  I think I reproduced this issue with the following code:

     

    Code Snippet

    using System;

    using System.Collections.Generic;

    using System.Text;

    using System.ServiceModel;

    using System.ServiceModel.Channels;

    using System.Threading;

    namespace Timeouts

    {

        class Program

        {

            static void Main(string[] args)

            {

                string address = "net.tcp://localhost";

                NetTcpBinding binding = new NetTcpBinding("NetTcp_TestTimeouts");

                TimeSpan timeToWait = TimeSpan.FromSeconds(70.0);

                try

                {

                    ServiceHost host = new ServiceHost(typeof(HelloService), new Uri(address));

                    host.AddServiceEndpoint(typeof(IHelloContract), binding, address);

                    host.Open();

                    Console.WriteLine("Service Host is {0}, press enter to continue.", host.State);

                    Console.ReadLine();

     

                    ChannelFactory<IHelloContract> factory = new ChannelFactory<IHelloContract>(binding, new EndpointAddress(address));

                    IHelloContract clientChannel = factory.CreateChannel();

                    ((IChannel)clientChannel).Open();

                    Console.WriteLine("[Client]: Send message 1 to HelloService.");

                    clientChannel.Hello(1);

                    for (int i = (int)timeToWait.TotalSeconds; i > 0; --i)

                    {

                        Console.Clear();

                        Console.WriteLine("[Client]: Waiting for {0} minute. \n binding Receive Timeout = {1}\n binding Close Timeout = {2}\n binding Send Timeout = {3}\n binding Open Timeout = {4}", timeToWait, binding.ReceiveTimeout, binding.CloseTimeout, binding.SendTimeout, binding.OpenTimeout);

                        Console.WriteLine(i);

                        Thread.Sleep(TimeSpan.FromSeconds(1.0));

                    }

                    Console.WriteLine("[Client]: Send message 2 to HelloService.");

                    clientChannel.Hello(2);

                    Console.WriteLine("[Client]: done.");

                }

                catch (Exception e)

                {

                    Console.WriteLine(e);

                }

                Console.ReadLine();

            }

        }

        [ServiceContract]

        public interface IHelloContract

        {

            [OperationContract]

            void Hello(int id);

        }

        public class HelloService : IHelloContract

        {

            public void Hello(int id)

            {

                Console.WriteLine("[Service]: Hello. Received {0}", id);

            }

        }

    }

     

    I configured the binding in App.config:

    Code Snippet

    <?xml version="1.0" encoding="utf-8" ?>

    <configuration>

      <system.serviceModel>

        <bindings>

          <netTcpBinding>

            <binding name="NetTcp_TestTimeouts" closeTimeout="00:00:10" openTimeout="00:00:20"

    receiveTimeout="00:00:30" sendTimeout="00:00:40">

              <reliableSession ordered="false" enabled="false" />

            </binding>

          </netTcpBinding>

        </bindings>

      </system.serviceModel>

    </configuration>

     

    The output of the program looked like this:

    [Client]: Waiting for 00:01:10 minute.
     binding Receive Timeout = 00:00:30
     binding Close Timeout = 00:00:10
     binding Send Timeout = 00:00:40
     binding Open Timeout = 00:00:20
    1
    [Client]: Send message 2 to HelloService.
    System.ServiceModel.CommunicationException: The socket connection was aborted. T
    his could be caused by an error processing your message or a receive timeout bei
    ng exceeded by the remote host, or an underlying network resource issue. Local s
    ocket timeout was '00:00:39.7770000'. ---> System.IO.IOException: The write oper
    ation failed, see inner exception. ---> System.ServiceModel.CommunicationExcepti
    on: The socket connection was aborted. This could be caused by an error processi
    ng your message or a receive timeout being exceeded by the remote host, or an un
    derlying network resource issue. Local socket timeout was '00:00:39.7770000'. --

     . . .

     

    Hope this helps!

     

    -James

    Tuesday, June 19, 2007 6:03 PM
  • After talking with some of our devs on this issue, I think I was mistaken in saying it is the SendTimeout you were hitting.  What you really need to set is the ReceiveTimeout on the Service side binding.  Here's why:

    When using TCP, the communication is sessionful.  In general, when the Client creates a channel to the Service, it uses up some Service side resources.  The Service has a ReceiveTimeout on the binding to configure how long to allow the Client to keep a channel open to the Service.  The ReceiveTimeout doesn't have anything to do with a receive action duration.  Further, the Client doesn't know what the ReceiveTimeout is on the Service, so when it faults because of ReceiveTimeout being exceeded, all it can tell you is what the local timeouts were.  Just in case this timeout was caused by something local to the Client, the exception gives you what the SendTimeout was.

    So, to have a NetTcp Service allow a Client to keep a channel open for longer than 10 minutes, you need to increase the ReceiveTimeout on the Service side's binding.

     

    HTH,

     

    -James

    Tuesday, June 19, 2007 8:14 PM
  • Thank you James for your reply.

    I was just trying to figure out why I got the same results in my app, and discovered this thread just as I realized it was the receiveTimeout that was causing the problem (after setting it to a ridiculously low setting and seeing the connection drop.)

    I'm glad I know what's going on, thank you for the detailed response.

     

    I wouldn't have guessed this was a receiveTimout issue because of the property's name.  It's very misleading.  I initially did not expect it to be a receiveTimeout issue because I figured it must be a timeout switch for receiving a message being transmitted - or something along those lines... may not have made sense.  Needless to say, the name was misleading.

    Thank you for looking into this and letting us know...

     

    Although, if I'm having to significantly up the receiveTimeout (am expecting a long period of idleness) am I not using the best solution?  My situation is that a client will connect to a service, and may have a period of idleness up to 62 hours (3 days), but the client must be prepared to receive a callback at any time during that period.  So it may be idle for a while... is this the right solution?  I know this opens up security issues, but that's my situation, and this is the only solution I could come up with.

    This is my first WCF - be easy on me!  This stuff is new to me...

     

     James Osborne - MSFT wrote:

    After talking with some of our devs on this issue, I think I was mistaken in saying it is the SendTimeout you were hitting.  What you really need to set is the ReceiveTimeout on the Service side binding.  Here's why:

    When using TCP, the communication is sessionful.  In general, when the Client creates a channel to the Service, it uses up some Service side resources.  The Service has a ReceiveTimeout on the binding to configure how long to allow the Client to keep a channel open to the Service.  The ReceiveTimeout doesn't have anything to do with a receive action duration.  Further, the Client doesn't know what the ReceiveTimeout is on the Service, so when it faults because of ReceiveTimeout being exceeded, all it can tell you is what the local timeouts were.  Just in case this timeout was caused by something local to the Client, the exception gives you what the SendTimeout was.

    So, to have a NetTcp Service allow a Client to keep a channel open for longer than 10 minutes, you need to increase the ReceiveTimeout on the Service side's binding.

     

    HTH,

     

    -James

    Wednesday, December 12, 2007 4:29 PM
  • Yeah, having a channel open for 3 days is far from ideal.  Not only the security risk, but it doesn't scale well if you end up having several clients opening long duration channels.

    You might consider having the client run a service to receive the callback.  Then the client can just call the remote service and close.  When your remote service has something, it can open a channel to your "clientService" and send the message.  Of course, this may pose all kinds of interesting deployment issues if you need to broadly distribute this client code externally. 

     

    -James

     

    Wednesday, December 12, 2007 6:04 PM
  • I like that idea, but in my situation it would be a deployment headache.  Actually, I just found out that there will be much more communication going on than I expected.  The service will be sending a handful of messages about every hour, so the receiveTimout can be small.  However, I'm now worried - only the service will be sending to the client, and the client will likely not need to respond.  So am I now going to run into (let me guess) a sendTimeout error?  Just wondering...

    Do I have to ensure that they both are talking to one another and not just one way?

    Testing it out right now, so I'm about to find out Smile

    Thanks a lot for your help James!

     

    -Josh

    Wednesday, December 12, 2007 10:03 PM
  • I don't think the sendTimeout will be an issue.  SendTimeout is per message, so if you don't get an ack for a message within SendTimeout, then it'll throw.  You don't have to get the response from the server, just the ack (http 200 code, I think).

    I could be wrong, I'm curious about the results of you testing.

     

    -James

    Wednesday, December 12, 2007 10:17 PM
  • I believe your right!  It seems to be working fine.  I set the sendTimeout to very low and it's still not timing out (now that I've got more communication from the service to the client and upping the receiveTimeout.)  So that answers my question: sendTimeout is not raised when the service has not received anything from the client (whereas the receiveTimeout is raised when the service hasn't sent anything to the client.) 

    I could be wrong but that's what I'm getting so far.  At any rate, it's working!  Yay!

    Thanks again!

     

    Josh

     

    Thursday, December 13, 2007 12:03 AM
  • Is there a timeout value that controls how long an attempt-to-connect can take before failing?

     

    I'd like to make this fairly short, so I can have the client try a few alternatives ...

     

    Josh also.

     

    Wednesday, May 21, 2008 6:21 PM
  • I think that's OpenTimeout.  Note that if you don't specifically Open the channel, like:

    ((IChannel)clientProxy).Open()

    before sending a message (ie: clientProxy.Ping("hello")Wink then that first call to the service will Open the channel, using the OpenTimeout.  If SendTimeout is small, the OpenTimeout won't be honored because SendTimeout will exceed first.  So, for more robust code, it's a good idea to specifically Open the channel first.

     

    -James

     

    Wednesday, May 21, 2008 6:30 PM
  • Hi James,

    I have read your reply here and I have a question:

    I have same problem but I use DuplexChannel. The client sends a request to the server and when the server has something to answer to the client it uses the callback. If there more than 20 minutes (approximately) passed then I have an exception:

    The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:01:00'. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host.

    Is there is also any timeout for the callback? It doesn't make sense to me because the aim of the callback is to send answers not immediately but only when the server has an answer...

    I have to say that in our project the server may have no messages for client even for a few days...

     

    Thanks,

    Boris.

    Wednesday, August 20, 2008 9:27 AM
  • When using the composite duplex channel, make sure that there were no errors processing the message at the service.  If there were errors and the service couldn't get the message up the binding stack to your application, then the errors won't be able to be returned on the response sequence.  They generally are returned on the http back-chanel for the request sequence, but since the WSDualHttpBinding has a one-way binding element right beneath the CompositeDuplex binding element, the http errors would be dropped and the Client would simply experience a timeout.

     

    Now, assuming you are using the standard WSDualHttpBinding, and assuming the server processes the request just fine, but it's just a long running service method, you could be hitting any number of timeouts.  From what you described, the timeout is happening on the Client right?  I would increase the Send and Receive timeouts on the binding.  Then look at the ReliableSessionBinding element (you may have to cast as a custom binding to get to this.)  Increase the InactivityTimeout.  Additionally, you might want to do this on the Service side binding, too.

     

    -James

    Wednesday, August 20, 2008 4:23 PM
  • Hi James,

    Thanks for your answer.

    First of all we use netTcpBInding. The exception that I mentioned was on the server side when server tryed to send a message to client, after not sending for several minutes (about 20 minutes).

    I have configured the receive timeout (on the sever side) to be infinite. I'm still testing if it helped...

    But even if this helped - is this the write way to set the timeout to be infinite? Maybe there is another, better solution ?

    Also, I don't know if this related to timeouts, but I have another problem - the server is running as windows service. Sometimes the process of the service  just disappears as if it was killed in Task Manager. Maybe you have any ideas what I can check?

    I tryed to use windbg but there I see that there was an exception in kernel32.exe and I can't see what happened. I suspect that WCF is related somehow because only when the system is idle and the client doesn't connect to server, this happens.

     

    Thanks,

    Boris.

     

    Wednesday, August 20, 2008 4:39 PM
  • I'm sorry, when you mentioned Duplex channel, I thought you were using the WSDualHttpBinding.  My assumptions were wrong.  Unless you are using a CompositeDuplexBindingElement, I would call this a request-reply channel.  There isn't an endpoint at the Client listening, is there?  Assuming no, this would mean that the Client basically makes a call and blocks while waiting for the Service to return the result.  The Service will keep the channel open for as long as its ReceiveTimeout is configured for.  If you never want the Service to time out, then set it to infinite.  I don't think this design is ideal, but it depends on your application.  If the information the Service returns doesn't depend too much on the information in the request, I might consider throttling the Client.  Just handle the exception and retry a given number of times, or implement your own timeout.

     

    As for the other problem, it sounds like the app pool is recycling.  If this is happening while your service is processing a message and kills the service, I would call that a serious bug with the Windows Service.  But if your service isn't processing a message or if it doesn't have an open session, this is normal behavior.  It should re-activate once a message is sent by a client.

     

    Wednesday, August 20, 2008 5:11 PM
  • I'm sorry if I wasn't clear. I meant that we create duplex proxy and use callback in order to send answers from server to client, so the client doesn't block.

    What do you mean by "throttling the client" ? Can you please give me a little more information?

     

    Reagarding the second problem - the service doesn't processing a message, but the fact that the service becomes stopped seems to me very strange. And I can't reactivate it in the code...

    Thursday, August 21, 2008 6:50 AM
  • Throttling client - Basically, you just wrap the call that occasionally throws in a try-catch block.  Then in the catch, you might log something, increment a counter, whatever.  Wrap that whole thing in a do-while block.  Only exit if the call succeeded or if a timeout that you implement inside the loop is exceeded.  This basically makes an "impatient" client that just keeps retrying failed calls, hoping one would get through.

     

    When you set the ReceiveTimeout on the Service to infinite, is it still timing out?

     

    For the second problem - What is the result of the Windows Service process ending?  If a new Client tries accessing the service, do you get an endpoint not found exception? 

     

    -James

    Thursday, August 21, 2008 4:22 PM
  • James, thanks for the explanation!

    The "infinite" value in ReceiveTimeout doesn't really solve the problem. Something happens (probably any timeout) and dispose method of object is called. In the dispose method we close the channel and the factory that created the channel.

    When closing there is an exception (for both):

    System.ServiceModel.CommunicationException: A TCP error (10038: An operation was attempted on something that is not a socket) occurred while transmitting data. ---> System.Net.Sockets.SocketException: An operation was attempted on something that is not a socket

    at System.Net.Sockets.Socket.Send(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)

    at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout)

    --- End of inner exception stack trace ---

    Do you have any ideas?

    Just to remind you, all this happens when there are no messages pass form client to server and no in the second direction.

     

    For the second problem - yes, when the server is unavailable, endpoint not found exception occurs.

    I tried to trace the WCF activities and used the trace viewer to check for anything "interesting". I found that before the service stopped there was an exception:

    "A connection has exceeded the idle timeout of this connection pool (00:02:00) and been closed."

    I think it was the reason for service process ending.

    Do you know how this idle timeout can be changed? Where?

    As I understand this value is for TCP connection in general and not specifaclly for WCF ...

     

    Thanks,

    Boris.

    Thursday, August 21, 2008 4:44 PM
  • Yeah, I'm pretty sure that second problem is the TCP idle timeout, but I don't know how to change that.

    The other error looks really odd.  I would check to make sure you aren't trying to close an aborted channel.  But usually the exception would come from WCF, unless you aren't using the WCF object model.

     

    I'm trying to loop some other people in on this thread.

     

    -James

     

    Thursday, August 21, 2008 4:55 PM
  •  

    Maybe I try to close an aborted channel but it shouldn't be aborted (at least I haven't aborted it) and also we check the state before the close.

     

    I attach snippet of our code , maybe it will help...

    private ChannelFactory<T> m_ChannelFactory;

    private T m_Channel;

    public void Dispose()

    {

    IClientChannel channel = m_Channel as IClientChannel;

    SafeCloseAndDispose(channel);

    SafeCloseAndDispose(m_ChannelFactory);

    }

    public static void SafeCloseAndDispose(ICommunicationObject channel)

    {

    if (channel == null) return;

    if (channel.State == CommunicationState.Closed) return;

    if (channel.State == CommunicationState.Closing) return;

    if (channel.State == CommunicationState.Faulted) return;

    try

    {

    channel.Close();

    }

    catch (CommunicationObjectFaultedException ex)

    {

    EventLog.WriteEntry("", "Aborting channel, exception1: " + ex);

    channel.Abort();

    }

    catch (TimeoutException ex)

    {

    EventLog.WriteEntry("", "Aborting channel, exception2: " + ex);

    channel.Abort();

    }

    catch (Exception ex)

    {

    EventLog.WriteEntry("", "Exception: " + ex);

    }

    }

     

    Waiting for your reply.

     

    Thanks,

    Boris.

     

    Thursday, August 21, 2008 5:20 PM
  • Back to the timeouts: Did you adjust the timeouts on both locations?  Since you are using duplex, the Client acts as a service as well for listening on the callback.  This could also time out.

     

    -James

     

    Friday, August 22, 2008 8:46 PM
  • We use mex, so we have a configuration file that is common for server and client.

    You say that Client acts as a service and also could time out - I thinks that this is what happens. But what switch sholud I change? Is this also "ReceiveTimeout" ?

    It's very strange to me - if client times out and server can't send messages to client via callback, than what the advantages of callback over usual, not duplex channel?

     

    Thanks,

    Boris.

    Monday, August 25, 2008 7:02 AM
  • When you say you use mex, do you mean your client is a generated proxy?  Because I don't believe the ReceiveTimeout would get published as metadata for the client.  You would have to configure that manually.

     

    -James

    Monday, August 25, 2008 6:00 PM
  • Yes, the client is a generated proxy.

    OK, I will try to change the ReceiveTimeout in code...

     

    Regarding the termination of the service  - can you suggest me something? Maybe the app pool you told me before about...

    Can it be any problem with WCF that causes the process of the service to terminate?

     

    Thanks,

    Boris.

     

    Tuesday, August 26, 2008 1:52 PM
  • If it's WCF that is causing it to close, we should be able to do something about it.  To find out, I think enabling tracing on the service could help.  Instructions for this are here: http://msdn.microsoft.com/en-us/library/ms732023.aspx, but basically, you just add a section to web.config, then you'll get a file that can be viewed with the Service Trace Viewer (http://msdn.microsoft.com/en-us/library/ms732023.aspx).  Then you can find the last few traces and find out how the service was terminated. 

     

    -James

    Tuesday, August 26, 2008 5:04 PM
  • Thanks, James.

    I already have tryed the tracing and have found no problems.

    I think that the termination doesn't happen because of WCF problem...

    I will continue to investigate...

    Thanks again for your help.

     

    Boris.

     

    Thursday, August 28, 2008 7:47 AM
  • Hi there,

    I seem to have some issues with WCF and the open timeout.  We are using the netTCP protocol.  We use multiple services that talk to eachother through an event manager which is basically a pub/sub system using one way publish calls and one way callbacks for the subscriptions.  The services run on a total of 3 servers all connected to the same gigabit network switch and belonging to the same domain.  All the servers are 8 core machines with 16 gig of RAM, gigabit network cards and plenty of fast disk space... i.e. pretty beefy machines.

    On a regular basis I see errors where the open command times out.  Now I'm not sure what happens during a open, but 1 minute should be more than enough for two servers on the same switch to negotiation communication.  We are not using reliable messaging.

    This is one of the error dumps from such a time out:

    Description:
    Failed to publish a message to the event manager: System.TimeoutException: The open operation did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout. ---> System.TimeoutException: The socket transfer timed out after 00:01:00. You have exceeded the timeout set on your binding. The time allotted to this operation may have been a portion of a longer timeout. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
       at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
       at System.ServiceModel.Channels.SocketConnection.ReadCore(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, Boolean closing)
       --- End of inner exception stack trace ---
       at System.ServiceModel.Channels.SocketConnection.ReadCore(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, Boolean closing)
       at System.ServiceModel.Channels.SocketConnection.Read(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout)
       at System.ServiceModel.Channels.DelegatingConnection.Read(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper& timeoutHelper)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper& timeoutHelper)
       at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
       --- End of inner exception stack trace ---

    Server stack trace:
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
       at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at AwsEventManagerClientLib.WxEventPublisher.IEventPublisher.TriggerEvent(String eventNamespace, String subscriptionId, Dictionary`2 EventData)
       at AwsEventManagerClientLib.EventHelper.PublishEvent(String eventNamespace, String subscriptionId, Dictionary`2 eventData) in C:\Users\misler\WBProjects\DataServices\Libraries\AwsEventManagerClientLib\Release\v1.0.0.0\Source\AwsEventManagerClientLib\EventHelper.cs:line 174
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="RemoteFileMonitor" />
        <EventID Qualifiers="0">2</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2009-01-20T16:02:20.000Z" />
        <EventRecordID>67092</EventRecordID>
        <Channel>AWS</Channel>
        <Computer>dc4-gis-emgr1.dc4.awscorp.com</Computer>
        <Security />
      </System>
      <EventData>
        <Data>Failed to publish a message to the event manager: System.TimeoutException: The open operation did not complete within the allotted timeout of 00:01:00. The time allotted to this operation may have been a portion of a longer timeout. ---&gt; System.TimeoutException: The socket transfer timed out after 00:01:00. You have exceeded the timeout set on your binding. The time allotted to this operation may have been a portion of a longer timeout. ---&gt; System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
       at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
       at System.ServiceModel.Channels.SocketConnection.ReadCore(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, Boolean closing)
       --- End of inner exception stack trace ---
       at System.ServiceModel.Channels.SocketConnection.ReadCore(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, Boolean closing)
       at System.ServiceModel.Channels.SocketConnection.Read(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout)
       at System.ServiceModel.Channels.DelegatingConnection.Read(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.SendPreamble(IConnection connection, ArraySegment`1 preamble, TimeoutHelper&amp; timeoutHelper)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.DuplexConnectionPoolHelper.AcceptPooledConnection(IConnection connection, TimeoutHelper&amp; timeoutHelper)
       at System.ServiceModel.Channels.ConnectionPoolHelper.EstablishConnection(TimeSpan timeout)
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
       --- End of inner exception stack trace ---

    Server stack trace:
       at System.ServiceModel.Channels.ClientFramingDuplexSessionChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.CallOnceManager.CallOnce(TimeSpan timeout, CallOnceManager cascade)
       at System.ServiceModel.Channels.ServiceChannel.EnsureOpened(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
       at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&amp; msgData, Int32 type)
       at AwsEventManagerClientLib.WxEventPublisher.IEventPublisher.TriggerEvent(String eventNamespace, String subscriptionId, Dictionary`2 EventData)
       at AwsEventManagerClientLib.EventHelper.PublishEvent(String eventNamespace, String subscriptionId, Dictionary`2 eventData) in C:\Users\misler\WBProjects\DataServices\Libraries\AwsEventManagerClientLib\Release\v1.0.0.0\Source\AwsEventManagerClientLib\EventHelper.cs:line 174</Data>
      </EventData>
    </Event>


    Any idea what could be causing open timeout issues like these?

    Thanks
    Marcel

    Tuesday, January 20, 2009 4:57 PM
  • The ReceiveTimeout as explained to be a server side issue, explains a lot.

    We have an Axis2 Web Service and a WCF Client with a Stub generated by a similar WCF Server. The protocols and the bindings where then massage until we had long lastuing sessions with outr AXIS2 Service. That is to say, our Axis2 Web Service will (at a given level) not timeout for 8 hour, I garantee that.

    1. we need long lasting sessions, AND we need a client AND/OR a service never timing out ! and we need it Synchron, not Asynchron or with Callbacks (Policy and Legacy Apps Integration figures).

    2. Szenarios: The person can login and establish a long lasting session with a service; this session gets terminated if the client does not send a second request within 8 hours (basically axis2 has adjusted this request time out to 30 seconds, but that is not practically) i.e. if you look at a transaction mask just filled in from a response object from this our context full service/objects/service, and then your logged-in session times out while you explain a thing on the phone to your customer; that is not practical. This situation is under control by a timeout value to be adjusted in AXIS2.xml for your service using service group id's in SOAPSESSION scope.

    3. BUT the other way arround is alos a night mar (as explained in this thread). You ask for a long lasting calculation from your service or DB transaction and now, your client has to wait (maybe you do just server side debugging, the client has to wait as well) not remote debugging in a nice-ALL-IN-1-beautifull-MS-WORLD but in a heterogeneous WCF -> AXIS2 world, where you have to connect your IDE via asocket to the JVM running in debug mode. That is to say:your WCF client shall send the request, be decouppled via a TCP/IP SOAP/XML protocoll, and it should just wait and not receive any timeouts at all (as it does right now).

    In what James Osborn explains so nicely in its first reply, I must admit now, I have no clue how to adjust something at the server in fact at AXIS2 which I do not know so deeply.  

    James, is this on a WCF service a TCP/IP kind of Timeout or would you look more at the WCF service engine logic, at our pace at the AXIS2 engine - more closer; Where at WCF Services is this feature implemented triggering a time-out in the client?. How would you explain this to an AXIS2 engine developer? What is not allowed to cancel or send back via wire to a waiting client by an AXIS2 service or by an WCF service?;

    But I really wonder now; Why did I never see this with a TCP/IP Monitor inbetween CL and SV?

    If the Axis2 server is the active partner making my client doing a receive timeout, then we should see this when we use a TCP Monitor? Right?

    So I am now really wondering about any explanations by this Forum why my client times out while a long calculation takes place at the server; I can see the clients request in the monitor (headers and body bytes) but nothing comes back from the server through this monitor, but my client times out anyway?

    What shall I adjust and where ? :-)

    can you prove and show it using kind of a TCP Monitor, that any event happening on the server is communicated by TCP/IP back to the client to make this just waiting WCF client TIME-OUT?

    Jsoef.Stadelmann
    @axa-winterthur.ch


    Sepp

    Tuesday, June 01, 2010 12:49 PM
  • Initially I am missing good explanations in all kind of client server szenarios explaining what all this time-out values will do. (be aware sending and receiving can lead to confusion given justtheb standpoint from which you are lookig, bee it from the client toward the server or be it from the server toward the client) I guess sometimes we talk about the same thing and mean different things, and sometimers we talk about different things and mean the same thing. And both leads to confusion;

    Well I have read through this thread, and I miss the following.

    We see a time out at the client

    But voices say there is a parameter to be adjusted at the service. right?

    Given this is true and given the time out value at the server is doubbled ...

    Is it then true that the client shall time out after the doubbled waiting time period given at the server?

    In this case can we then say there is a relation from server back to client,
    that is to say, in this case the server informs/signals the client the time-out-event.

    If that is true?! quwestion: how is this time-out-event signalled back to the client?

    my own observations using Ethereal/wirechark inbetween WCF 3.5 Client and Apache/Axis2-1.2 server with long lasting sessions is as such;

    The client times out if the server does not response within 1 minute; Just because the server is busy or on debugging at a break!

    With Ethereal, a protocol analyzer, one can observed the following

    No.   Time            Source              Destination       Protocol   Info
    1      0.000000      166.9.88.58      166.9.121.130   TCP         1573 > 8080 [SYBN] Seq=0 Len=0 MSS=1460 WS=2
    2      0.000337      166.9.121.130   166.9.88.58      TCP         8080 > 1573 [SYN, ACK] Seq=0 Ack=1 Win=62780, Len=0 MSS=1460 WS=0
    3      0.000377      166.9.88.58      166.9.121.130   TCP         1573 > 8080 [ACK] Seq=1 Ack=1 Win=65700 Len=0
    4      0.000475      166.9.88.58      166.9.121.130   TCP         [TCP segment of reassembled PDU]
    5      0.002110      166.9.121.130   166.9.88.58      HTTP        HTTP/1.1 100 Continue
    6      0.002175      166.9.88.58      166.9.121.130   TCP         [TCP segment of reassembled PDU]
    7      0.002178      166.9.88.58      166.9.121.130   TCP         [TCP segment of reassembled PDU]
    8      0.002196      166.9.88.58      166.9.121.130   HTTP        POST /axis2/services/SpezplaService HTTP/1.1
    9      0.003114      166.9.121.130   166.9.88.58      TCP         8080 > 1537 [ACK] Seq=26 Ack=1933 Win=62780 Len=0
    10   60.009453     166.9.88.58      166.9.121.130    TCP         1573 > 8080 [FIN, ACK] Seq=1933 Ack=26 Win=65672 Len=0
    11   60.010096     166.9.121.130   166.9.88.58       TCP         8080 > 1573 [ACK] Seq=26 Ack=1934 Win=62780 Len=0

    You can see, the request is made in a bit more then 3 ms, then we wait for the server to response. But that will not happen because the server is busy for 5 minutes, or it is in a debugger session at a break, single stepping through code;

    And then ou can see with No 10 and 11, that the client times-out; sends the message to the screen, and about the same time the client sends a [FIN, ACK] to the server, meaning we terminate, and the server respones with an [ACK]

    So it is clearly the client which times out, and now please tell me what I have to adjust on my client side custome binding with a HTTPTransport and Text to post-pone this client-originating time-out to a later time, lets say 30 minutes.

    The error message I receive after 1 minute of waiting is:

    The request channel timed out while waiting for a reply after 00:00:59.9844001.
    Increase the timeout value passed to the call to Request or increase the SendTimeout value on the Binding.
    .... etc...

     

    I would like to know - why changing my TimeSpan to 30 minute has no effect?

    Sepp

        public FKTmapController(String theEndPointAddress, String theProxyAddress)
        {
    
          EndpointAddress epa1 = new EndpointAddress(theEndPointAddress);
    
          //--- CustomBinding programmatic configuration --------------------------------------------
          CustomBinding binding;
    
    
          // 1. Create a custom binding that contains binding elements (BE)(Order Important)
          //TransactionFlowBindingElement tfBE = new TransactionFlowBindingElement();
    
          //*******************************************************************
          //****** ORDER in STACK (insertion Point in stack) is IMPORTANT *****
          //*******************************************************************
          
          // 2. [optional]
          //ReliableSessionBindingElement reliableSession = new ReliableSessionBindingElement();
          //reliableSession.Ordered = true;
    
          // 3. [optional] 
          // SecurityBindingElement securityBE = new SecurityBindingElement();
          
          // 5. [optional]
          // Create a BE for the required Encoder
          TextMessageEncodingBindingElement textBE = new TextMessageEncodingBindingElement();
          textBE.WriteEncoding = Encoding.UTF8;
    
          //BinaryMessageEncodingBindingElement binBE = new BinaryMessageEncodingBindingElement();
          //MtomMessageEncodingBindingElement mtomBE = new MtomMessageEncodingBindingElement();
    
          
          // 6.
          // Create a BE for the required Transport and define the proxy for TCP Monitoring
          //                    ***************************************
          HttpTransportBindingElement httpTransportBE = new HttpTransportBindingElement();
          httpTransportBE.AllowCookies = true;
          //httpTransport.KeepAliveEnabled = false;
          
          if (theProxyAddress.Length > 0)
          {
            httpTransportBE.UseDefaultWebProxy = false;
            httpTransportBE.ProxyAddress = new Uri(theProxyAddress);
          } else {
            httpTransportBE.UseDefaultWebProxy = true;
          }
    
    
          //--------------------------------------------------------------------------------
          // create the binding collection consisting the individual binding elements
          //--------------------------------------------------------------------------------
          //
          // create the customBinding and pass a stack of BindingElements
          binding = new CustomBinding(textBE, httpTransportBE);
          // after this steps we have the binding and we have the endpoint address
          // now lets give this two to the client stub - see below in event procedures
    
    
          binding.OpenTimeout.Add(new TimeSpan(0,1,0));
          binding.ReceiveTimeout.Add(new TimeSpan(0,1,0));
          binding.SendTimeout.Add(new TimeSpan(0,30,0));
          binding.CloseTimeout.Add(new TimeSpan(0,1,0));
    
          
          //--- ChannelFactory creation(s) just needs a binding ----------------------------
          // new try
          //ChannelFactory<Services.stub> factory = 
          //  new ChannelFactory<Services.stub>(binding);
    
          //in use **********************************************
          ChannelFactory<Services.stubChannel> factory2 =
            new ChannelFactory<Services.stubChannel>(binding);
    
          // new try
          //ChannelFactory<Services.stubClient> factory3 =
          //  new ChannelFactory<Services.stubClient>(binding);
    
    
    
          //--- CustomBehavior ---- A D D  I N T E R C E P T O R S ---------------------------------
          //  Add a custom behavior which adds our message interceptors
          //factory.Endpoint.Behaviors.Add(new WcfUtils.CustomBehavior());
          factory2.Endpoint.Behaviors.Add(new WcfUtils.CustomBehavior());
          //factory3.Endpoint.Behaviors.Add(new WcfUtils.CustomBehavior());
    
    
          //--- EndpointAddress(es). ----------------------------------------------------------------
          // you can define a real servers EP URI here and have the 
          // Transport.ProxyAddress of the binding set to the proper 
          // new Uri("http://C035619:54888") of the i.e. TCP Monitor;
          // some trials ...
          EndpointAddress10 epa10 = EndpointAddress10.FromEndpointAddress(epa1);
          EndpointAddress endPointAddress = epa10.ToEndpointAddress();
    
    
          //--- FKTmapManager -----------------------------------------------------------------------
          //  Through FKTmapManagerChannel we call our services()
          //FKTmapManager = factory.CreateChannel(endPointAddress);
          FKTmapManagerChannel = factory2.CreateChannel(endPointAddress);
          //FKTmapManagerClient = factory3.CreateChannel(endPointAddress);
        }
    
    

     

     

     

             

     

     

     


    Sepp
    Monday, July 12, 2010 4:26 PM
  • That is how one has to define time-outs, not as shown in my previous code when I just wonder why it does not work.

          //--------------------------------------------------------------------------------
          // create the binding collection consisting the individual binding elements
          //--------------------------------------------------------------------------------
          //
          // create the customBinding and pass a stack of BindingElements
          binding = new CustomBinding(textBE, httpTransportBE);
    
    	  // adjust the various time-out's
          binding.OpenTimeout = new TimeSpan(0,1,0);       //********* default 1 minute ******
          binding.SendTimeout = new TimeSpan(2, 0, 0);      //********* extrem 2 hours ******
          binding.ReceiveTimeout = new TimeSpan(2, 0, 0);     //********* extrem 2 hours ******
          binding.CloseTimeout = new TimeSpan(0,1,0);       //********* default 1 minute ******
    
    

    Trimming a binding that way makes that the the binding finds its way with all values properly when we call

    //in use **********************************************
       ChannelFactory<Services.stubChannel> factory2 =
        new ChannelFactory<Services.stubChannel>(binding);
    

    After all, to make a long story short;

          // that is where your timeout values must be to take effect latest ....
          factory2.Endpoint.Binding.OpenTimeout = new TimeSpan(0, 1, 30);
          factory2.Endpoint.Binding.ReceiveTimeout = new TimeSpan(0, 30, 0);
          factory2.Endpoint.Binding.SendTimeout = new TimeSpan(0, 30, 0);
          factory2.Endpoint.Binding.CloseTimeout = new TimeSpan(0, 1, 0);
    
    

    But again 

    binding.OpenTimeout.Add or

    factory2.Endpoint.Binding.Add  will not do the job, and I just wonder what gets added and where?

     


    Sepp
    Tuesday, July 13, 2010 12:37 PM