locked
Bug in Linq AsParallel RRS feed

  • Question

  • Hi guys,

    I have found a strange behavior in AsParallel method. It looks like a bug. This behavior was tested on Win7 and Win8 64 bit machines with 4 cores processor (Intel i5-3437U).

    So... I have a code of simple console application:

        class Program
        {
            static void Main(string[] args)
            {
                List<int> list = new List<int>() { 1, 1, 1, 10000, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 };

                list.AsParallel().ForAll(
                    i =>
                    {
                        var sw = Stopwatch.StartNew();
                        Thread.Sleep(i);
                        sw.Stop();
                        Console.WriteLine(string.Format("Sleep time: {0}\tThreadId: {1}", sw.ElapsedMilliseconds, Thread.CurrentThread.ManagedThreadId));
                    });

                Console.WriteLine("Finish");
                Console.ReadLine();
            }
        }

    Here is the result of the code running:

    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 12
    Sleep time: 20  ThreadId: 13
    Sleep time: 22  ThreadId: 6
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 13
    Sleep time: 1   ThreadId: 6
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 6
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 13
    Sleep time: 2   ThreadId: 10
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 13
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 13
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 13
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 13
    Sleep time: 1   ThreadId: 12
    Sleep time: 10001       ThreadId: 6
    Sleep time: 2   ThreadId: 6
    Sleep time: 1   ThreadId: 6
    Sleep time: 1   ThreadId: 6
    Finish

    As you can see, AsParallel creates 4 threads (according to processors count) with ids: 6, 10, 12, 13.

    But at some point the only one thread (6) is remaining live!

    Any thoughts?

    Best Regards,

    Aleksey


    Sunday, April 13, 2014 8:04 AM

Answers

All replies

  • Seems that the collection is partitioned between threads, and it works as expected. See also the Partitioner class.

    Consider another way of parallelization:

    Parallel.ForEach(

           list,

           i =>

           {

                  var sw = Stopwatch.StartNew();

                  Thread.Sleep( i );

                  sw.Stop();

                  Console.WriteLine( string.Format( "Sleep time: {0}\tThreadId: {1}", sw.ElapsedMilliseconds, Thread.CurrentThread.ManagedThreadId ) );

           } );

    Sunday, April 13, 2014 8:14 PM
  • The same result for Parallel.ForEach...

    So, according to your answer AsParallel method will work optimal in case that all the provided action will have the same execution time for each item in the collection.

    The provided code is just an example, illustrating the problem. In the real world I use the AsParallel method to make a parallel parsing of files. I have expected that the method will make the smart parallelization, and in case the parsing of one of the files will take much longer then the others, it will not stick the other files parsing. I mean, my expectation was that one thread will work on that file, but the other threads will parse all the other files. But it doesn't work this way.

    Partitioning of the collection is explaining the behavior, but don't you think it is a "little" problematic implementation?

    Wednesday, April 16, 2014 8:06 AM
  • The same result for Parallel.ForEach...

    [...]

    Do you mean that the output is the same? Show your results. The long thread (10000 ms) should be the last in the list, so that you can use Parallel.ForEach when the partitioning made by AsParrallel is not suitable for particular problems.

    Wednesday, April 16, 2014 8:42 AM
  • Here is the code:

    List<int> list = new List<int>() { 1, 1, 1, 10000, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1 };

                Parallel.ForEach(
                    list,
                    i =>
                    {
                        var sw = Stopwatch.StartNew();
                        Thread.Sleep(i);
                        sw.Stop();
                        Console.WriteLine(string.Format("Sleep time: {0}\tThreadId: {1}", sw.ElapsedMilliseconds, Thread.CurrentThread.ManagedThreadId));
                    });

                Console.WriteLine("Finish");
                Console.ReadLine();

    And here is the result:

    Sleep time: 1   ThreadId: 9
    Sleep time: 2   ThreadId: 8
    Sleep time: 1   ThreadId: 10
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 8
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 10
    Sleep time: 2   ThreadId: 9
    Sleep time: 1   ThreadId: 8
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 9
    Sleep time: 2   ThreadId: 12
    Sleep time: 2   ThreadId: 10
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 9
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 12
    Sleep time: 1   ThreadId: 9
    Sleep time: 1   ThreadId: 11
    Sleep time: 1   ThreadId: 9
    Sleep time: 1   ThreadId: 12
    Sleep time: 10000       ThreadId: 8
    Sleep time: 1   ThreadId: 8
    Sleep time: 2   ThreadId: 8
    Finish


    As you can see the last to list items have waited till the thread 8 will be free.

    It looks like the same partitioning implementation here

    Wednesday, April 16, 2014 12:22 PM
  • Have you tried to use the Partitioner class that Viorel mentioned? Partitioner.Create(list, true).AsParallel() should be what you need in your case.

    And no, this isn't a bug. Dynamic partitioning has higher overhead, you don't want to use that by default for AsParallel because then every simply PLINQ query such as Sum will be significantly slower.

    Recommended reading: http://msdn.microsoft.com/en-us/library/dd997411(v=vs.110).aspx 

    Thursday, April 17, 2014 7:07 AM