none
Sync Framework over WCF: uploading large data sets

    Вопрос

  • I am trying to upload ~20,000 rows via Sync Framework, but I am getting a CommunicationException. Smaller changesets never encounter exceptions.

    There are two messages I get:

    • An error (The request was aborted: The request was canceled.) occurred while transmitting data over the HTTP channel.
    • The underlying connection was closed: A connection that was expected to be kept alive was closed by the server.

    I have tried to debug this by looking at the WCF traces, but it has no information about what went wrong. The only information that it has about my DataCacheSyncService is that a ServiceHost was constructed, then opened, then closed.

    Here is my client-side binding configuration:

          <wsHttpBinding>
            <binding name="dataCacheSyncBinding"
                     closeTimeout="00:01:00"
                     openTimeout="00:01:00"
                     receiveTimeout="00:10:00"
                     sendTimeout="00:10:00"
                     bypassProxyOnLocal="false"
                     transactionFlow="false"
                     hostNameComparisonMode="StrongWildcard"
                     maxBufferPoolSize="2147483647"
                     maxReceivedMessageSize="2147483647"
                     messageEncoding="Text"
                     textEncoding="utf-8"
                     useDefaultWebProxy="true"
                     allowCookies="true">
              <readerQuotas maxDepth="2147483647"
                            maxStringContentLength="2147483647"
                            maxArrayLength="2147483647"
                            maxBytesPerRead="2147483647"
                            maxNameTableCharCount="2147483647" />
              <reliableSession ordered="true"
                               inactivityTimeout="00:10:00"
                               enabled="false" />
              <security mode="None" />
            </binding>
          </wsHttpBinding>
    Here is my server side binding configuration:
          <wsHttpBinding>        
            <binding name="dataCacheSyncBinding"
                     closeTimeout="00:01:00"
                     openTimeout="00:01:00"
                     receiveTimeout="00:10:00"
                     sendTimeout="00:10:00"
                     bypassProxyOnLocal="false"
                     transactionFlow="false"
                     hostNameComparisonMode="StrongWildcard"
                     maxBufferPoolSize="2147483647"
                     maxReceivedMessageSize="2147483647"
                     messageEncoding="Text"
                     textEncoding="utf-8"
                     useDefaultWebProxy="true"
                     allowCookies="true">
              <readerQuotas maxDepth="2147483647"
                            maxStringContentLength="2147483647"
                            maxArrayLength="2147483647"
                            maxBytesPerRead="2147483647"
                            maxNameTableCharCount="2147483647" />
              <reliableSession ordered="true"
                               inactivityTimeout="00:10:00"
                               enabled="false" />
              <security mode="None" />
            </binding>
          </wsHttpBinding>
    7 мая 2012 г. 18:00

Ответы

  • Hey Karl,

    You have to set the maxRequestLength under system.web in your web.config file:

    <httpRuntime useFullyQualifiedRedirectUrl="true|false"
                 maxRequestLength="size in kbytes"
                 executionTimeout="seconds"
                 minFreeThreads="number of threads"
                 minFreeLocalRequestFreeThreads="number of threads"
                 appRequestQueueLimit="number of requests"
                 versionHeader="version string"/>

    Make sure that the batch size you are using won't be bigger than this value.  That should do the trick...

    Ian

    • Помечено в качестве ответа Karl Dickman 21 мая 2012 г. 16:16
    17 мая 2012 г. 18:26
  • This time the 404 was because configuration/system.webServer/security/requestFiltering/requestLimits@maxAllowedContentLength was set too low.

    Future readers: this security setting is in place for a reason, so think long and hard before changing it.

    • Помечено в качестве ответа Yi-Lun Luo 21 мая 2012 г. 1:56
    18 мая 2012 г. 22:55

Все ответы

  • The problem is very definitely some kind of misconfiguration of the sync service or IIS. If I reconfigure synchronization to work directly with the database server rather than communicating through WCF, I experience no synchronization problems at all. (All the records are uploaded in about ten seconds.)
    7 мая 2012 г. 21:34
  • try enabling the Sync Framework Tracing so you can get more information on the sync events firing during the sync...
    7 мая 2012 г. 23:48
  • Hello, I see you configured the timeouts to 10 minutes. If that's not enough, try to increase the timeouts further. In addition, increase httpRuntime.executionTimeout: http://msdn.microsoft.com/en-us/library/e1f13641.aspx. You may also need to increase several timeout settings on IIS.

    You can also post the question on http://social.msdn.microsoft.com/Forums/en-US/category/sync if you think it's more related to Sync Framework, and post it on http://forums.iis.net/ if you think it's more related to IIS.


    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    If you have feedback about forum business, please contact msdnmg@microsoft.com. But please do not ask technical questions in the email.

    8 мая 2012 г. 1:55
  • Thanks for the suggestions.

    When debugging the services, the breakpoints I set on ApplyChanges never get hit: the CommunicationException actually gets thrown somewhere in IIS before getting to the code that I wrote. (Sync Tracing was a good suggestion, though. It's a useful tool.)

    I agree that there's something wrong with the service configuration, but in this case it's not the timeouts. I am encountering the CommunicationException immediately, rather than after ten minutes. I have cross-posted to the IIS forums, per Yi-Lun Luo above.

    8 мая 2012 г. 16:58
  • Hey Karl,

    You have to set the maxRequestLength under system.web in your web.config file:

    <httpRuntime useFullyQualifiedRedirectUrl="true|false"
                 maxRequestLength="size in kbytes"
                 executionTimeout="seconds"
                 minFreeThreads="number of threads"
                 minFreeLocalRequestFreeThreads="number of threads"
                 appRequestQueueLimit="number of requests"
                 versionHeader="version string"/>

    Make sure that the batch size you are using won't be bigger than this value.  That should do the trick...

    Ian

    • Помечено в качестве ответа Karl Dickman 21 мая 2012 г. 16:16
    17 мая 2012 г. 18:26
  • Now what I'm getting is "There was no endpoint listening at http://localhost/ebp/Services/EDT/DataCacheSync.svc that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details." The inner exception on this is a 404 from the server.

    I thought I had previously resolved the 404 issue.

    Also, I had already set my maxRequestLength to 2147483647 some time ago.

    18 мая 2012 г. 21:51
  • Tried to see if I could avoid the 404 by switching to a basic HTTP binding. ~20,000 rows is really not that much data, so why is WCF being so balky?

    It seems like I've tried everything. Set maxRequestLength on httpRuntime, set high values for the maxReceivedMessageSize and all the reader quotas in the WCF config. What am I missing here?

    18 мая 2012 г. 22:08
  • This time the 404 was because configuration/system.webServer/security/requestFiltering/requestLimits@maxAllowedContentLength was set too low.

    Future readers: this security setting is in place for a reason, so think long and hard before changing it.

    • Помечено в качестве ответа Yi-Lun Luo 21 мая 2012 г. 1:56
    18 мая 2012 г. 22:55
  • IByrne's suggestion about maxRequestLength (see above) may have solved the problem.
    21 мая 2012 г. 16:17