none
Multi-threaded Windows Service Hangs sometimes RRS feed

  • Question

  • I am developing using Visual Studio Professional 2015.

    I have created a windows service with 2 threads.  It connects to a serial port and talks to several pieces of equipment connected to a 485 network.  Thread 1 polls the equipment continuously.  Thread 2 sets up a tcpListener on a specific port.  When it "hears" a command on that port, it sets an interrupt flag that thread 1 looks at.  Thread 1 then stops its polling.   Thread 2 then sends commands over the serial port and receives data back and writes it to the tcpport.

    Problem:

    Periodically, when it receives the diagnostic commands (command on tcpport) it hangs and desktop application gets socket time outs.

    Any ideas how I can start to attack debugging this as it is a desktop exe that sends the command to the tcp port and a service exe that listens and responds ?

    thanks

    Thursday, February 28, 2019 8:52 PM

Answers

  • When a multithreaded program hangs sometimes, it is usually due to a race condition. This means that more than one thread is accessing a common resource (such as a variable in memory) and from time to time this causes the content of the shared resource to be corrupted. This can happen when the two threads change it at the same time, or when one thread reads a value that was in the middle of being changed by another, so a "bad" value is read. The bad value causes the program to malfunction. The easiest remedy is to place a lock around the code that accesses the common resource, but it is not always obvious which portions of the code might be affected.

    Another issue that may occur with multithreaded code is called a "deadlock". This happens when one thread locks a resource A and tries to access a resource B, while another thread has locked resource B and attempts to access resource A. This causes both of them to stop and the program "hangs". One way to prevent deadlocks is to ensure that all threads always lock resources in the same order, for example, if you know that you will need resources A and B always lock first A and then B. Unfortunately, in a complex program, this is easier said than done.

    So, how do you debug it? One thing that you can do is "sprinkle" the program with debugging statements, which write into the logger of your choice (for instance, Log4Net is a popular one). When the application is in production, you configure the logger to ignore the statements, or at least the most verbose ones, so that they don't affect the performance of your program. But when you need to debug it, you enable these statements. When the program hangs, you look at the log to see what was the last thing that each of your threads was doing. This should give you an approximation about where the problem lies. If needed, you can then add more logging around that area and repeat until you locate the problem.

    Thursday, February 28, 2019 9:17 PM
    Moderator