locked
Questions about PLINQ RRS feed

  • Question

  • Hi,

    I have a couple of questions about PLINQ - one general and one specific -

    (1) -> I have been using LINQ (and now PLINQ) extensively and in particular like the functional abstraction it gives you in terms of just saying what tasks you want parallelized (not how) and it is done for you - and then I see all these posts about Imperative function calls like Tasks.Create, Tasks.Wait(), Parallel.ForAll() etc - so my question is - am I missing out on really cool API functions under the hood in TPL, when I have been just going gaga over PLINQ - in other words what are the class of problems for which I must leave PLINQ because its ineffective and then code imperatively using such calls - i understand these are broad questions - but necessary to wrap my mind around these new technologies...

    (2) -> Specifically, I have some PLINQ code where I am reading sequentially from a System.Data.DataReader and creating a lazy list using yield return - whose consumption I will then attempt to parallelize after calling .AsParallel() on the list - my question is have I really parallelized the consumption of the list or its production? - would multiple tasks be working to get me as many records from the database - or is it one task getting all the records and many tasks then consuming it?


    Thanks in advance...

    jaydd
    Thursday, September 10, 2009 6:35 PM

Answers

  • Hi jaydd,

    (2) If you are reading from a DataReader, then parallel execution functionality would have to be built into that DataReader.

    (1) You can use PLINQ to implement some simple task parallelism scenarios, but it PLINQ not as convenient as TPL to use for this purpose. Also, if you have tasks that have more complex relationships (e.g., that wait on each other), PLINQ will be even less appropriate.

    Igor Ostrovsky (blog)
    • Marked as answer by jaydd Sunday, September 13, 2009 5:08 PM
    Friday, September 11, 2009 11:50 PM

All replies

  • Hi jaydd,

    (1) PLINQ is aimed at data-parallel computing - do the same thing on a set of inputs in parallel. Tasks focus on task-parallel computing - do different things in parallel.

    (2) The production of the list will not be parallelized. Only the part that follows AsParallel() can run in parallel. So, one thread is getting records from the data base, and then multiple threads consume the records.

    It is best to have a range of tools in your toolbox, and use each for the tasks that it is best suited for.

    And, of course, experiment, measure, and choose the implementation that works best for your scenario.

    Igor Ostrovsky (blog)

    Thursday, September 10, 2009 10:34 PM
  • Hi Igor, Thanks,

    (2) is there a way to parallelize the production of records? assuming the ordering is not important

    (1) about the distinction above between data-parallel and task-parallel computing - does it not disappear if I consider tasks as 'data'; so in the example below I could use PLINQ to do different tasks in parallel? (admittedly the tasks must belong to the same family of method signatures)

                List<Action<int>> Jobs = new List<Action<int>>
                {
                    (int n) => Console.WriteLine("{0};{1}",n,n),
                    (int n) => Console.WriteLine("{0};{1}",n,n*n),
                    (int n) => Console.WriteLine("{0};{1}",n,n/2)
                };

                int tasks = Jobs.AsParallel().
                            Select(n =>
                                {
                                    n.Invoke(System.Threading.Thread.CurrentThread.ManagedThreadId);
                                    return 1;
                                }).Count();

    Thanks..


    jaydd
    Friday, September 11, 2009 5:15 PM
  • Hi jaydd,

    (2) If you are reading from a DataReader, then parallel execution functionality would have to be built into that DataReader.

    (1) You can use PLINQ to implement some simple task parallelism scenarios, but it PLINQ not as convenient as TPL to use for this purpose. Also, if you have tasks that have more complex relationships (e.g., that wait on each other), PLINQ will be even less appropriate.

    Igor Ostrovsky (blog)
    • Marked as answer by jaydd Sunday, September 13, 2009 5:08 PM
    Friday, September 11, 2009 11:50 PM