locked
Signal R Over Load Balancers RRS feed

  • Question

  • User914698260 posted

    Hello friends,

    I am experiencing an issue with our SignalR Self Hosted Setup. Keep in mind that our solution works in our development environment without a load balancer flawlessly. With that in mind I'm pretty confident that our live environment setup with the load balancer and ssl termination is causing the problems. I've also tried with SignalR Scalout with the same results.

    Client -> Load Balancer (With SSL Termination) -> Self Hosted Signal R Server on a Windows Server on port 80. 

    The client successfully connects. Once the clients try's to send a message to the server. We get a "reconnect" for every message that the client is trying to send.  The messages fail to reach the hub for processing. Messages originating from the signal r server going out the the clients are successful. Its only the reverse that doesn't seam to make it. The only error I am managing to get is the listed bellow. 

    Is there a hook I can add to the self host signal r application to catch all incoming http/ws requests. Or any other path you guys may think of that I could take to identify why my messages are not making it to the hub function.

    SignalR.Transports.WebSocketTransport Error: 0 : OnError(37fcae7f-b91a-4c49-bda5-62801b575997, System.Net.WebSockets.WebSocketException (0x80004005): An internal WebSocket error occurred. Please see the innerException, if present, for more details. ---> System.Net.HttpListenerException (0x80004005): The I/O operation has been aborted because of either a thread exit or an application request
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at System.Net.WebSockets.WebSocketHttpListenerDuplexStream.<ReadAsyncCore>d__30.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---   
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

    Thursday, September 27, 2018 5:06 PM

Answers

  • User914698260 posted

    After more investigation and comparing a working and broken environment. I came to the conclusion that the messages from the client to the server get lost on the firewall or load balancer as it does not reach the server. After reviewing the incoming traffic over the PORT on the servers I can see that the working environment has both traffic both ways. On the broken environment the messages from client never reach the server. 

    Thanks for you help Fei Han

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Tuesday, October 2, 2018 2:09 PM

All replies

  • User61956409 posted

    Hi LucasHoby,

    The client successfully connects. Once the clients try's to send a message to the server. We get a "reconnect" for every message that the client is trying to send. 

    To troubleshoot the issue, you can try to override OnConnected event and do logging in this event to check if the client indeed connect to your hub server. Besides, you can try to enable tracing in your client application so that you can check the diagnostic information.  

    I've also tried with SignalR Scalout

    Which backplane(Azure Service Bus, Redis and SQL Server) you are using?

    With Regards,

    Fei Han

    Friday, September 28, 2018 3:30 AM
  • User914698260 posted

    Hello Fei Han, 

    Thank you for your reply, I have since put this aside since other critical task took my attention. But I am back at it today.

    to troubleshoot the issue, you can try to override OnConnected event and do logging in this event to check if the client indeed connect to your hub server. Besides, you can try to enable tracing in your client application so that you can check the diagnostic information.  

    I currently have tracing on with the settings provided by the msdn documentation. That is how I was able to get that unknown error I have in my initial post.

    <sharedListeners>
    <add name="SignalR-Transports" type="System.Diagnostics.TextWriterTraceListener" initializeData="transports.log.txt" />
    <add name="SignalR-Bus" type="System.Diagnostics.TextWriterTraceListener" initializeData="bus.log.txt" />
    <add name="SignalR-Init" type="System.Diagnostics.TextWriterTraceListener" initializeData="init.log.txt" />
    </sharedListeners>

    Which backplane(Azure Service Bus, Redis and SQL Server) you are using?

    I am using the SQL Server Backplane.

    I find it odd that my messages from the server work and not the reverse. I can see my socket messages trying to send from the client but it's soon followed by the reconnect function from the client. 

    Thanks

    Monday, October 1, 2018 7:35 PM
  • User914698260 posted

    After more investigation and comparing a working and broken environment. I came to the conclusion that the messages from the client to the server get lost on the firewall or load balancer as it does not reach the server. After reviewing the incoming traffic over the PORT on the servers I can see that the working environment has both traffic both ways. On the broken environment the messages from client never reach the server. 

    Thanks for you help Fei Han

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Tuesday, October 2, 2018 2:09 PM