none
High performance alternatives of ThreadPool.QueueUserWorkItem Method (QUWI) RRS feed

  • Question

  • I am on framework 2.0 and currently using  ThreadPool.QueueUserWorkItem for my multi-threaded app.

     

    In my application the QueueUserWorkItem is getting hit tens of thousands of times per second and my testing now shows it is causing a high CPU usage on a top of the line multicore hardware.

     

     

    The implementation goes something like this:

     

     

     

    internal void StartProcessing()//This can be called upto tends of thousands of times per second. CPU usage goes down when I hit this function at a lower rate.

    {

    lock (MyQueueEvets)

    {

    if ( !bWorkInProgress ) //To prevent using up thread pool by one trading item

     

    bWorkInProgress = ThreadPool.QueueUserWorkItem( new WaitCallback(this.StartWork), MyQueueEvets.Dequeue() );

    }

     

    }

     

     

     

    private void StartWork()

    {

    lock( MyQueueEvets )

    {

    //SomeBusinessLogic() //So even if I comment this function out the CPU usage remains the same. 

    if ( MyQueueEvets.Count == 0 )

    bWorkInProgress = false;

    else

    bWorkInProgress = ThreadPool.QueueUserWorkItem( new WaitCallback(this.StartWork), MyQueueEvets.Dequeue() );

    }

    }

     

     

    I tried setting min and max thread but that worsens the CPU usage. I am not doing any direct IO so I always set IO parameter to 1 when setting min and max threads.

     

     

    Are there any alternatives to acheive lower CPU and low latency for the kind of scenaraios I need this for?

     

     

    Monday, August 15, 2011 2:49 PM

Answers

  • Given that IIS, SQL, and Exchange all use the ThreadPool.QueueUserWorkItem I think it can handle the work load. Is it perfect? No. That's why it receives performance improvements in nearly every release of .Net. Is it scalable? yes.

    You're saying that when a lot of work comes in the CPU is getting used. In my opinion, that's a good thing. The CPU is there to be used. If no wasted work is happening (ie. polling, calculations that don't matter), that means that the system is making excellent use of the resources at hand.

    The purpose of the ThreadPool is to make use of the CPU.

    If you're only performance problem is that the CPU usage is high, then it sounds like you're running an ideal system.

    If you feel that the system is having performance problems, and they occur during times of high CPU usage, the real problems could be false cache swaps, or Page Faults.

    I work hard to get my applications to take up all of the CPU. They get more work done faster when they do.

    • Marked as answer by Paul Zhou Wednesday, August 24, 2011 6:14 AM
    Friday, August 19, 2011 9:09 PM
  • If you have Visual Studio 2010 Premium or Ultimate, you can run the Concurrency Visualizer and see what's blocking the UI thread. Perhaps the UI thread is getting locked for human noticable amounts of time around the lock for MyQueueEvents.
    • Marked as answer by Paul Zhou Wednesday, August 24, 2011 6:14 AM
    Monday, August 22, 2011 6:04 PM

All replies

  •  

    Hi,

     

    You can try to remove "lock(MyQueueEvets)" since you have locked it when start the method.

     Moreover, you can use WinDbg or SOS.dll to determine what causes the high CPU usage.

     


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, August 19, 2011 3:14 AM
  • Thanks,

     

    lock (MyQueueEvets) happends once in STartProcessing() and again in StartWork().

     

    Do you mean I can lock it only once in StartProcessing() and do not need to lock it in StartWork() ??

    Friday, August 19, 2011 2:41 PM
  • Have you considered using threads with a shared blocking queue instead of threadpool. This may reduce some of the overhead. Then you can also lock the threads to different physical processor to reduce switching over processors on multi-processor hardware.

    Friday, August 19, 2011 2:54 PM
    Moderator
  • You shouldn't need to have a queue ontop of the ThreadPool queue. From the comments in the code, it looks like this happens because you're afraid that the ThreadPool will create too many threads. If the work items execute for less than a half a second, the ThreadPool won't create more threads than there are cores on the computer. The ThreadPool will wait for a half a second for a thread to return to the ThreadPool before spinning up another thread.

    You will probably get more bang for your buck to find out why the work items are taking so long (are they being blocked by synchronous I/O).

    Does the lock in StartWork() really come before SomeBusinessLogic() (I'm assuming that's where the reall work gets done).

    Lock statements will spin for a little while before actually blocking a thread. So you could be getting a lot of CPU from the spinning going on in the lock statements.

    Friday, August 19, 2011 8:12 PM
  • So I did a few experiments before posting.

     

    I left everything as is, and only commented out

    bWorkInProgress = ThreadPool.QueueUserWorkItem( new WaitCallback(this.StartWork), MyQueueEvets.Dequeue() );

     

    in StartWork() and that brought down the CPU

     

    Another experiment I did was to comment out everything except the 

    bWorkInProgress = ThreadPool.QueueUserWorkItem( new WaitCallback(this.StartWork), MyQueueEvets.Dequeue() );

    in StartWork but the CPU remained pegged.

     

    The problem is not latency but CPU pegging.

     

    Its just that when QuereUserItem gets hit a lot the CPU pegs, even if I comment out the delegate implementation the QuereUserItem raises. This led me to think it is just QueueUserItem unable to handle the hits so frequently. I dont think it is the business logic because as I mentioned, even when I comment out all business logic, CPU still pegs uneffected.

    Friday, August 19, 2011 8:26 PM
  • Given that IIS, SQL, and Exchange all use the ThreadPool.QueueUserWorkItem I think it can handle the work load. Is it perfect? No. That's why it receives performance improvements in nearly every release of .Net. Is it scalable? yes.

    You're saying that when a lot of work comes in the CPU is getting used. In my opinion, that's a good thing. The CPU is there to be used. If no wasted work is happening (ie. polling, calculations that don't matter), that means that the system is making excellent use of the resources at hand.

    The purpose of the ThreadPool is to make use of the CPU.

    If you're only performance problem is that the CPU usage is high, then it sounds like you're running an ideal system.

    If you feel that the system is having performance problems, and they occur during times of high CPU usage, the real problems could be false cache swaps, or Page Faults.

    I work hard to get my applications to take up all of the CPU. They get more work done faster when they do.

    • Marked as answer by Paul Zhou Wednesday, August 24, 2011 6:14 AM
    Friday, August 19, 2011 9:09 PM
  • Yes, high CPU is'nt bad per se but my application has a UI and there are performance issues at the time of high CPU.
    Monday, August 22, 2011 3:57 PM
  • So is it possible the real problem would be that your process has too many threads, causing over subscription of the cores, which would prevent the GUI thread from getting the time it needs on one of the cores?
    Monday, August 22, 2011 4:31 PM
  • Number of threads it shows is 45.

    I dont set max and min threads, Threadpool does it. 

    Monday, August 22, 2011 5:33 PM
  • If you have Visual Studio 2010 Premium or Ultimate, you can run the Concurrency Visualizer and see what's blocking the UI thread. Perhaps the UI thread is getting locked for human noticable amounts of time around the lock for MyQueueEvents.
    • Marked as answer by Paul Zhou Wednesday, August 24, 2011 6:14 AM
    Monday, August 22, 2011 6:04 PM
  • I will try the concurrency visualizer. I did'nt know of this tool until this post. Will learn how to use it.
    Monday, August 22, 2011 9:08 PM