none
Unable to send UDP packets larger than the MTU with Windows Build 1809 using C# UdpClient RRS feed

  • Question

  • Short version of below - we can't seem to successfully send UDP packets larger than 1472 bytes when running Build 1809 of Windows, though it worked OK in previous versions.

    We have an existing C# application (really a set of applications) that runs on a local wired network that periodically sends out UDP packets to communicate status to other applications on different computers on the network. Some of these packets are small, but some are fairly large. This application is written in C# using .NET 4.5.1 and uses the UdpClient class to broadcast and receive broadcasts. Prior to testing with Windows 10 build 1809 / Windows Server 2019 build 1809, everything worked fine - we sent and received the larger packets without a problem. However, starting with build 1809, it appears that we can no longer successfully send the large packets. Here is some of the testing that we did:

    System 1: Windows 10 Build 1803 (MTU is 1500)

    System 2: Windows 10 Build 1809 (MTU is 1500)

    I wrote a test program that sends out a small (200 bytes or so) and large (8000 bytes or so) UDP packet using UdpClient, while also listening for those packets. Here is what happens when I send from each system:

    -Send from System 1:

    -System 1 sees both packets
    
    -System 2 sees both packets

    -Send from System 2:

     -System 1 sees only the small packet
    
     -System 2 sees both packets

    This happens every single time - the large packets never arrive successfully. Further testing revealed that the magic number was 1472 bytes. That or less worked, and more than that failed. This is why I suspect that something with fragmenting/MTU is not working correctly. We had not seen this problem before, so I fired up Wireshark to take a look at what might be going on. However, and this is where it gets weird, starting Wireshark on the Build 1809 system suddenly makes it able to send the packet, even if I exit Wireshark. Rebooting the system, though, reverts it to its original state of being unable to send successfully. I should note that I always see "Fragmented IP Protocol" for these packets in Wireshark on the receiving end of things regardless of whether they are working correctly or not.

    I did some reading online and found that RDP over UDP had a major overhaul in Build 1809, but I did not see anything about UDP packets larger than the MTU having a problem, nor was I able to find anyone else reporting this particular issue.

    We have made no code changes to this part of our code in a long time - it worked on Windows 7, Server 2012R2, Server 2016, and Windows 10 prior to build 1809. Is there something new in Build 1809 that requires us to set a flag somewhere or configure something on the network adapter? I don't know what all Wireshark does when it boots up, but it seems to "fix" things somehow, so if anyone has an idea what that might be specifically that could help as well.

    As an additional note - I did make sure my network drivers were up to date, and one of the systems was a Windows 10 VM running 1803 that I upgraded to 1809.  It worked OK as 1803 and immediately stopped working correctly after the update to 1809.

    Monday, March 25, 2019 12:17 PM

Answers

  • I already posted this in the Windows 10 networking forum. Going to re-post here as well, in case someone finds this thread instead.

    ----------------------------------------------------------------------------------------------------------------------------

    I encountered this problem recently at my company when we updated a number of our systems to 1809 and 1903. I did a bunch of debugging and analysis and I think I know what the problem is and how to work around it.

    Problem is that the UDP packets larger than MTU in size (larger than 1472 bytes of max payload), will be corrupted and dropped by the network stack. If you analyze the difference between data sent and data received, you will find that the first IP frame contains correct data. Subsequent frames, however, will have two bytes of data corrupted. This will generally repeat once for each subsequent set of 1472 bytes of data in the reassembled datagram. I did see a couple of other corruption patterns, depending on the data I was sending, but this was generally the most common one.

    Moreover, the corrupted two bytes have direct correlation to the data being sent. So, the values will always be the same for the same set of data. My theory was that the origin of this corruption is the UDP checksum calculation going wrong. Generally this checksum computation is done in the protocol stack. Many NICs, however, support UDP/TCP checksum offloading.

    I realized this optimization was the cause of the problem when I ran a packet sniffer and saw that the data was correct before it was sent out on the wire and was clearly received corrupted on the other end.

    My conclusion is that this optimization (UDP Checksum Offload) was broken in the 1809 release and continues to remain broken in 1903. To work around it for now, the optimization needs to be disabled (seems to be enabled by default). This can be done in the Device Manager. Find your network adapter card in the list and go into Advanced tab. We only deal with IPv4, but IPv6 might also be affected, so you might need to disable both.

    Hope this helps and hope the information will be passed along to the devs and officially fixed.


    • Proposed as answer by SashaLivshin Tuesday, June 18, 2019 8:09 PM
    • Edited by SashaLivshin Tuesday, June 18, 2019 8:14 PM typo
    • Marked as answer by nola2172 Monday, June 24, 2019 7:58 PM
    Tuesday, June 18, 2019 8:08 PM

All replies

  • Not a VS issue. The networking forum is here

    AFAIK Win10 has some special hack for Wireshark, this is why it may cause different behavior.

    -- pa

    Monday, March 25, 2019 3:59 PM
  • I actually had posted this in the Windows 10 Networking forum, and the Microsoft person there said to post here.
    Monday, March 25, 2019 4:23 PM
  • Hmm. Well... but nevertheless this is not a VS issue. SDK, maybe... Have you tried some latest insider preview version like 19H1 ?

    Or, reproduce the issue with a native program?

    -- pa


    • Edited by Pavel A Monday, March 25, 2019 7:28 PM
    Monday, March 25, 2019 7:24 PM
  • I just downloaded Packet Sender, and I see the same issue when using it.  On the 1809 end I hear when either side sends, but on the 1803 end I only hear when the 1803 side sends.
    Tuesday, March 26, 2019 5:16 PM
  • Hi nola,

    The forum is about using visual studio(designer, reported control, editor and etc in VS IDE). For your case is more related to C#development, so we will move this post to Visual C# forum for better support.

    Thank you for your understanding.

    Best Regards,

    Dylan


    MSDN Community Support Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com

    Wednesday, March 27, 2019 9:09 AM
  • This is not limited to C#.  I am having the same issue in my Delphi application after upgrading to 1809.  Any UDP packet I send that is larger than the MTU will fail to be transmitted.  Downgrading to 1803 and everything starts working again.  

    There are reports of Remote Desktop RDP failing over UDP links after upgrading to 1809 as well, so I am guessing the issues may be from the same root cause but that is just my theory and not proven fact. (I cant post links here but google "windows 1809 UDP MTU").

    Monday, April 22, 2019 6:09 PM
  • I already posted this in the Windows 10 networking forum. Going to re-post here as well, in case someone finds this thread instead.

    ----------------------------------------------------------------------------------------------------------------------------

    I encountered this problem recently at my company when we updated a number of our systems to 1809 and 1903. I did a bunch of debugging and analysis and I think I know what the problem is and how to work around it.

    Problem is that the UDP packets larger than MTU in size (larger than 1472 bytes of max payload), will be corrupted and dropped by the network stack. If you analyze the difference between data sent and data received, you will find that the first IP frame contains correct data. Subsequent frames, however, will have two bytes of data corrupted. This will generally repeat once for each subsequent set of 1472 bytes of data in the reassembled datagram. I did see a couple of other corruption patterns, depending on the data I was sending, but this was generally the most common one.

    Moreover, the corrupted two bytes have direct correlation to the data being sent. So, the values will always be the same for the same set of data. My theory was that the origin of this corruption is the UDP checksum calculation going wrong. Generally this checksum computation is done in the protocol stack. Many NICs, however, support UDP/TCP checksum offloading.

    I realized this optimization was the cause of the problem when I ran a packet sniffer and saw that the data was correct before it was sent out on the wire and was clearly received corrupted on the other end.

    My conclusion is that this optimization (UDP Checksum Offload) was broken in the 1809 release and continues to remain broken in 1903. To work around it for now, the optimization needs to be disabled (seems to be enabled by default). This can be done in the Device Manager. Find your network adapter card in the list and go into Advanced tab. We only deal with IPv4, but IPv6 might also be affected, so you might need to disable both.

    Hope this helps and hope the information will be passed along to the devs and officially fixed.


    • Proposed as answer by SashaLivshin Tuesday, June 18, 2019 8:09 PM
    • Edited by SashaLivshin Tuesday, June 18, 2019 8:14 PM typo
    • Marked as answer by nola2172 Monday, June 24, 2019 7:58 PM
    Tuesday, June 18, 2019 8:08 PM
  • I tested this on build 1809 of Windows 10, and it does appear to fix things there.  However, I am also running build 1809 of Windows Server 2019, and I have disabled every offload that I could and it still won't send larger packets through.  That being said, I think even for Windows Server, this is probably pointing in the right direction.
    Monday, June 24, 2019 7:58 PM