All Threads Run On A Single Processor

Question

I am trying to generate client load against my server.  In the client I have 100 threads.  Each thread is in an infinite loop issuing web requests.   create to get a connection,  get response to get the response,  close the response and do it again.   I am running the client on a hyperthreaded hexacore machine.   So it looks like 12 processors.   But only one processor is ever running and I am pegged at 8% CPU (1 processor worth).   Occasionally the load will switch processors,  but still only runs on one processor.   You can see it in the graph below.   CPU 9 has the load,  then CPU 9 drops and CPU 2 picks it up.  Then CPU 9 gets it back and then flip flops with CPU 4.

I have tried Parallel.ForEach,  I have tried creating my own threads,  I have tried queuing work items to the thread pool.   Same result in all cases.   I am not specifying any affinity, parallelism - anything.  This is a simple console application. I thought maybe the connections were being limited,  but I set the maximum number of connections on the service point equal to the number of threads I have,  and the actual calls to the service take less than 1 millisecond.  If I was blocking getting a connection I would expect requests to take much longer.

What am I doing wrong?

_sjz.

---

sjz

Answer 1

It looks like your threads are doing IO operations. If you issue IO operations (blocking operation) to a thread and if the thread is blocked (waiting for connection or response), the thread is returned to thread pool instead of wasting time in waiting. Then the thread may take another operation. Suppose the other operation is alse blocked, the thread is again returned to thread pool which can take up other operations.

You can see that the same thread can take up more than one operations. So, it may look like all the CPUs are not utilized. But, CLR is clever enough - it knows what is best at a given time. It may schedule the threads to run on seperate processors or it may schedule multiple threads to run on one processor only depending upon which is more appropriate at that time. Hence, the output that your task manager is showing, there is nothing wrong in it.

---

Please mark this post as answer if it solved your problem. Happy Programming!

Answer 2

IO operations utilize very little of the available CPU resources.  To see your threads on the Task Manager display, insert a long running, CPU intensive loop on the thread.

Answer 3

sjz.

I didn't understand correctly. You want to stress test your server or your client? If you send e.g. 1000 requests does it differ if they have been sent by 10 threads, 20 threads, 50 threads, ...? CPU cores are saturated when there is CPU-Bound job but your program is doing I/O operation. So most of the time your threads are sleeping. That's why you don't see any significant increase in CPU usage. In fact it is a false viewpoint to think making CPUs busy is a good thing.

Adavesh:

If you issue IO operations (blocking operation) to a thread and if the thread is blocked (waiting for connection or response), the thread is returned to thread pool instead of wasting time in waiting

Your guess about the answer to sjz's question is almost true (Although the correct answer is that sjz's job is I/O-bound not CPU-bound). But your statement is completely false about returning pooled threads to thread-pool. See the following example:

ThreadPool.QueueUserWorkItem((x) =>
{
	Console.WriteLine("I'm thread {0}", Thread.CurrentThread.ManagedThreadId);
	Thread.Sleep(5000); // current pooled thread blocks here for 5 seconds
	Console.WriteLine("I'm thread {0}", Thread.CurrentThread.ManagedThreadId);
});

Both Console.WriteLine() will write the same number.

Pooled threads return to thread-pool only when their job (the execution of the delegate which has been given to them) is finished.

If sjz is using APM methods for sending web request and receiving the response it is then after calling BeginXXX() method, the pooled thread issuing the I/O operation will be freed and when the callback is called, another random thread is employed from the thread-pool.

Answer 4

Hi, 

If you are creating thread to simulate the user, which is not a ThreadPool thread, You will have 100 threads being created in your case, but the question is, is all threads are performing any action at all the time?

As I see you are using hyperthreaded, looks like there are 2 CPU's, Hexacore at one time there will be virtually 12 threads runs, if there is a work at all. But in many cases threads have to wait to get response in some form, could be user input, waiting for connection, reading from file...etc In these cases ContextSwitch, when time slice expires windows context switches to another thread, happens. So only CPU is enough to perform requests. 

If i look at my system, dualcore , though there are 865 threads are running but no work is happening since all of these threds are waiting for some response.

In case you try to implement by simulating 100 requests using threads from ThreadPool, even system will not create 100 threads but will finish with limited number of threads.

And,

Using AsParallel does not guarantee it always partitions the process and run concurrently, PLINQ infrastructure analyzes the overall structure of the query and if query is likely to yield speedups with parallelization, PLINQ partitions the source and assigns to Task and run concurrently.

for eg: 

    var guidList = new List<Guid>();
            for (int i = 0; i < 10000; i++)
            {
                guidList.Add(Guid.NewGuid());
            }
            var groupedList = guidList.AsParallel().GroupBy(f => f).Where(g => g.Count() > 1);
            Console.ReadKey();

If i < 10000000 it uses multiple cores to finish but if i < 1000 it doesn't.

You can make each request perform CPU intensive operation behind to simulate the CPU usage, but in reality is this happens? 

You can also think of using available test tools to test stress on webserver right? look at this KB for more information

http://support.microsoft.com/kb/231282 

I hope this helps you....

---

If this post answers your question, please click "Mark As Answer". If this post is helpful please click "Mark as Helpful".

Answer 5

Adavesh:

If you issue IO operations (blocking operation) to a thread and if the thread is blocked (waiting for connection or response), the thread is returned to thread pool instead of wasting time in waiting

Your guess about the answer to sjz's question is almost true (Although the correct answer is that sjz's job is I/O-bound not CPU-bound). But your statement is completely false about pooled threads. Who has told that when a pooled thread is blocked it returns to thread pool?! dsds

First thing is - Whatever method you choose, TPL, ThreadPool.Que.. or APM - all use ThreadPool in the background. And my answer was common to all these methods but not just ThreadPool.QueueUserItem (I hope you are understanding). If you take ThreadPool.QueueUsetItem specifically, then you are correct that the thread will be blocked when a blocking instraction is issued. But This not the case with other two. They pick a thread from ThreadPool for doing operation. If the operation is blocked, then the thread will return to ThreadPool. (Dig more into the internals of TPL & APM).

Second thing is - When  you are commenting on someone's reply, make sure your statements do not hurt them. No one is perfect in this world, So I may not be perfect and my answer may be wrong. BTW, did I tell anywhere that "somebody told me when a pooled thread ..." ???

---

Please mark this post as answer if it solved your problem. Happy Programming!

Answer 6

This is the incorrect statement.

If the operation is blocked, then the thread will return to ThreadPool.

I'm sorry Adavesh. I didn't mean to offend or hurt you. I hope I can compensate someway in your good posts.

Answer 7

Another thing that might be happening here is that by default the WebClient (and most tools that use web requests) follow proper etiquette and won't send more than 2 simultaneous requests to a server no matter how many requests you spawn.

See also: http://stackoverflow.com/questions/866350/how-can-i-programmatically-remove-the-2-connection-limit-in-webclient

---

My blog: blog.jessehouwing.nl

Answer 8

Thanks all for some comments.   Not sure I have an answer yet.  So let me add some details.

1) i believe i had already removed the 2 connection limit and had mentioned this in my original post.  I had done this using the service point manager default connection limit as the first line of my client,  but after reading, tried it with an app.config as well just to be sure.  No difference there.   And the reason I discounted this before is because my web calls finish so quickly.  It doesn't seem like threads are queuing/blocking waiting to get a connection.

2) I am not using APM so no Begin, End for get request stream or get response.  I was thinking this is just a simple client program,  more threads seemed simpler - at least at the onset.   I can switch to that to see if it makes a difference,  but I have my doubts.

3) As I mentioned in the original post I have tried creating my own threads as well (no thread pool),  and just letting them run.  Same result.

4) Everyone seems convinced that I am IO bound.  I reduced the number of threads to two which again use 100% of just one CPU.  So I only need two threads to saturate one CPU.  Running 12 copies of the program consumes all 12 CPUs.   My web calls all finish in about 1/100 th of a millisecond (machines are side by side on the same switch).   So I am not convinced that I am IO bound.

I'll try the APM thing later today and report back.

---

sjz

Answer 9

Can you share the code you're using, if only the outline? That way we can try to reproduce it or maybe just spot the issue. Just commenting with more options that might be the issue isn't helping you nor us.

---

My blog: blog.jessehouwing.nl

Answer 10

This is a dumbed down version (lot's of cutting , whitespace removed, etc).  Didn't run this - so it is more of an outline.

public class Program {
    public class State {
        private string url;
        private int numIterations;
        public State(string url, int numIterations) {
            this.numIterations = numIterations;
            this.url = url;
        }
        public void Run() {
            for (int i = 0; i < this.numIterations; i++) {
                WebRequest request = (WebRequest) HttpWebRequest.Create(url);
                request.Method = "POST";
                request.ContentType = "text/plain";
                using (Stream stream = request.GetRequestStream()) {
                    using (StreamWriter streamWriter = new StreamWriter(stream)) {
                        streamWriter.WriteLine("Test " + i);
                    }
                }
                WebResponse response = request.GetResponse();
                response.GetResponseStream().Close();
            }
        }
    }
    static void Main(string[] args) {
        int    NUM_THREADS = 2;
        int    NUM_ITERATIONS  = 100000;
        string APP_URL = "http://myserver:54321/myapp/echo";
        ServicePointManager.DefaultConnectionLimit = NUM_THREADS * 2;
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(new Uri(APP_URL));
        servicePoint.UseNagleAlgorithm = false;
        servicePoint.Expect100Continue = false;
        // create state for MAX_CONNECTION threads
        Thread thread = null;
        State[] state = new State[NUM_THREADS];
        for (int i = 0; i < state.Length; i++) {
            state[i] = new State(APP_URL, NUM_ITERATIONS);
            thread = new Thread(new ThreadStart(state[i].Run));
            thread.Start();
        }
        // imprecise - but join on the last thread
        thread.Join();
    }
}

---

sjz

Answer 11

Hi sjz,

How's it going now? Do you have any updates about the previous issue?

---

Bob Shen [MSFT]
MSDN Community Support | Feedback to us

Answer 12

No progress.  I was hoping someone would take a look at the code and say "You fool !!!   You forgot to do ... blah".   Alas, the thread went dark.   I will try to convert to APM,  but that really should not be necessary.   There is something more fundamental going on here.

---

sjz

Answer 13

I'll have to agree with the previous posters, you have low CPU utilization because you're IO bound.  If you just want to stress test your web server you should look at the throughput of requests, not the CPU utilization, to see if you're doing your job.  If your goal is to stress test the machine sending the requests, you should come up with a more CPU intensive task since getting web requests consumes very little CPU time.

Answer 14

I'm sorry,  but the process is NOT IO bound.   2 threads consume 100% of the CPU on one core.     I can add more threads but it all stays on one core.    If I instead run multiple copies of the program (each with 2 threads) then I can consume all the cores,   24 threads = 100% CPU across the whole machine.    But 24 threads in a single process only runs on a single core.    So not IO bound.

---

sjz

Answer 15

Well if all you're doing is network communication with another entity then you SHOULD be IO bound, and if you're not you should get a profiler and see where your code is spinning because your overhead is apparently too high.