Do you see Tasks as the preferred method for scheduling work from .net 4?

Question

Hi,
Do you see Tasks as the preferred mechansim for scheduling work from .net 4?

I understand that Tasks behind the scenes utilize the thread pool but my understanding is that the Task Manager will by default create a number of work threads equal to the number of cores on the machine. If say you needed to call a number of functions that have some IO or network bottleneck would it be better scheduled through the thread pool as by utilizing the thread pool directly would create more threads and then might complete work quicker?

Tasks are a lot more intuitive to work with than the thread pool directly and threads so seems a better alternative?

Thanks,

Alex

Answer 1

Hi Alex,

Yes, Tasks will be the preferred way to schedule asynchronous work in .NET 4.  The statement Task.Factory.StartNew will be recommended over ThreadPool.QueueUserWorkItem.

The default TaskScheduler does not, itself, decide how many threads to use.  It is built on top of the ThreadPool, which uses a hill-climbing algorithm to adjust to the number of threads that maximizes work item completion rate.  Typically, blocking workloads (to due I/O, etc.) will cause the work completion rate to drop, causing the ThreadPool to inject more theads.  Therefore, even if some of your Tasks block, you shouldn't see any improvement by using the ThreadPool directly.

Thanks,
Danny

Answer 2

Hi Danny,
Thanks for the reply.

Sorry a few more simple questions!

1) Just to confirm - there is a number of posts and articles that say the task scheduler by default creates a number of threads equal to the number of cores on the machine. Am I correct in thinking that this is just the default number and is then adjusted by the thread pool if necessary as you describe above?
2) On a dual core machine if I create 8 tasks that will each take about 30 seconds to complete and are just calculating something so no IO etc how many threads would you expect would be created by the task manager by default - 2?
3) Is there any documentation available yet about the task scheduler/manager?

Thanks,

Alex

Answer 3

Hi Alex,

Re: (1)
You're probably referring to posts/articles from around or before the June 2008 CTP timeframe.  Back then, TPL used its own independent scheduler, which created a number of threads equal to twice the number of available cores, I believe.  A while back, we started building TPL on top of the .NET 4 ThreadPool, which had been enhanced with some of the features from our earlier scheduler.  So now, the TaskScheduler in TPL does not directly manipulate the default or current number of threads used; all that stuff is managed by the ThreadPool.

Re:(2)
It depends on the ThreadPool's algorithms, and I don't know exactly how those work.  If your 8 Tasks are all CPU-bound, I would probably expect the hill-climbing to adjust to around 2-4 threads on a dual-core, if there was nothing else going on on the machine.  If your Tasks run for a long time, the ThreadPool's starvation logic might kick in and inject more threads.  This is all speculation though.  It's hard to determine how many threads the ThreadPool will use, and that's sort of the point.  It is designed to figure out and use the optimal number of threads for arbitrary workloads.

Re: (3)
It sounds like you're more interested in ThreadPool-related information.  You might start here:
http://blogs.msdn.com/ericeil/archive/2009/04/23/clr-4-0-threadpool-improvements-part-1.aspx

Hope this helps,
Danny

Answer 4

As Danny mentioned, the CLR thread pool has a "starvation detection" feature that's intended to prevent long-running work from "starving" other work that may be trying to run.  We have a background thread that checks every 1/2 second to see if any work has completed recently.  If not, then we add a thread (but only if there's more work available). 

So if you queue 8 tasks that take 30 seconds each, I would expect that you would fairly quickly end up with 8 threads running all tasks concurrently.

This is probably not a big problem if you've only got 8 tasks - but for larger numbers of very long-running tasks it can seriously degrade performance.  To avoid this, you can break each task into many smaller tasks. 

I really need to finish my blog post about thread injection....

Answer 5

Hello,

I have a similar situation, I have about 20,000 tasks to finish, basically it's a queue of database records that I bring from the database to a datatable, and then I start iterating through each record.

Each task takes about 0.2 second, most of my machines where I run the processes are dual core machines. With Parallel Extension CTP (june 08), I used the TaskManager along with the TaskManagerPolicy to explicitly set the number of threads executing per processor to somewhere between 5 to 10 threads, it was not a hardware kill for such light tasks. And it was pretty quick.

Now there is no such settings in the new .net 4.0 beta, so if I use the new Task.Factory.StartNew, would I get a similar or better performance to what I have now with relying on the threadpool to adjust itself, or is it a better to customize the taskScheduler to force it to use a specific number of threads per core (I will need an example for that if this is the case).

thanks!
-Tamer

Answer 6

Hi Tamer-

Right, the ThreadPool should adapt to the optimal number of threads for processing your 20,000 tasks.  Over time it may ramp up and down, and in fact if you pay very close attention, you'll likely see the number of threads steadily wobble over time as the ThreadPool tests the waters of adding and removing threads to see what kind of difference it makes in throughput in order to tune accordingly.

Answer 7

Hi Tamer-

Right, the ThreadPool should adapt to the optimal number of threads for processing your 20,000 tasks.  Over time it may ramp up and down, and in fact if you pay very close attention, you'll likely see the number of threads steadily wobble over time as the ThreadPool tests the waters of adding and removing threads to see what kind of difference it makes in throughput in order to tune accordingly.

So what if I do want to force the scheduler to schedule more tasks? I did a quick test for 5000 tasks and the CPU only maxed out at about 30% on a dual core machine, I would like to take advantage and push it somewhere between 60% to 80% of CPU usage.

for comparison, these 5000 tasks took about 488 seconds (8 minutes). With the 2008 June CTP (Task Manager and Task manager policy) it finished faster since I was able to explicitly push the number of threads per core.

thanks,
Tamer

Answer 8

Hi Tamer-

How much faster was it able to finish with the June CTP?

If you do want to control the number of threads used, you can do so through the static ThreadPool methods SetMinThreads and SetMaxThreads; the former will force the ThreadPool to quickly ramp up to at least the specified number of threads if there are any work items waiting to be processed.

Answer 9

Hi Tamer-

How much faster was it able to finish with the June CTP?

If you do want to control the number of threads used, you can do so through the static ThreadPool methods SetMinThreads and SetMaxThreads; the former will force the ThreadPool to quickly ramp up to at least the specified number of threads if there are any work items waiting to be processed.

Stephen: I had in production the SetMinThreads and SetMaxThreads before moving to the 08 June CTP.  I have the june CTP in production now and it's running well.

Today, I had a head-to-head comparison between June CTP TPL and .net 4.0 TPL, .net 4.0 proved to be faster overall, here are some numbers:
on a job that contained 2500 tasks and a dual core machine:

June TCP: set at 5 threads per core, it took an average of 3 minutes to finish. I verified this against production performance (same june CTP) and it does match.

.net 4.0 : this was wild experience, I had several tests on it, and here is how long it took, each number represents the "number of seconds" it took for each run, they were all executed within the same process, in a repeated fashion, I think the concept of self learning threadpool was applied:
162, then 89 (almost 50% faster), then 59, 64,50,49, 60,46,53,40,49,46,43,44,42 .......

So by removing the high numbers at the begining, the avg was about 47 seconds (which is great and much faster than June CTP). In this experience I did not capture the CPU usage.

I redid the test on .net 4.0 again, to capture the CPU this time:
153 seconds with avg cpu 6%, then down to 81 seconds with avg cpu 20%, then with almost the same CPU, I got avg of 65 seconds 3 times, and then it went down to the 50's seconds with slightly heigher cpu usage, then something weired happened:

it jumped to 50% cpu usage, but took longer: 135 seconds (up from 55), and then it dropped to 85 seconds, with almost same CPU, and then the macjic happened: high cpu (avg 70% to 80% usage) with time dropping to low 40's and 30's in some instances, matching my first test with .net 4.0

So at this point I'm happy with the performance, I will contine using .net 4.0 for this test and will keep my eyes on it.

thanks!