Self Replicating Task

Question

In the CTP there was an option to Create a Task with the option SelfReplicating, I can't seem to find that option in .NET 4.  Has this functionality been lost from the Task based API.

Answer 1

Hi Andy,

Yes, that option has been removed.  You might try the ParallelExtensionsExtras, where you'll find a self replicating task sample: Extensions->TaskFactoryExtensions->TaskFactoryExtensions_SelfReplicating.cs

Danny

Answer 2

Andy, can you elaborate on why you're interested in SelfReplicating?  We found that most cases that would benefit from this option could actually be built as well or better on top of some of the Parallel.* APIs; I'd be curious to see if that's true as well for your scenarios.  Thanks!

Answer 3

Sorry for the late reply I literally posted the message as I went on vacation.  

I saw the parallel while blog implementation, and I just felt that SelfReplicatingTask would have been more appropriate.  Whilst I can see Parallel.ForEach works it doesn't seem like a particularly natural way of doing things.

 

Answer 4

Thanks Danny I'll check it out.

Answer 5

Hi, What about a scenario in which you have a queue of work to be done. The psudocode is as follows. 

Thread: 
	Work work = null; 
	while (TryGetWork(out work)) 
		DoStuffWithWork(work); 

Now I create a task that will do this, but I want to create one for each of the core's on the machine. 

A self replicating task is perfect for this. Why have you taken it out? What construct would you currently use for this type of scenario? 

Thanks 

Regards 

Craig.

Answer 6

Hi Craig,

I'd probably use Parallel.ForEach for that scenario:

IEnumerable<Work> GetAllWork()
{
    Work work = null;
    while (TryGetWork(out work))
        yield return work;
}

void Main()
{
    Parallel.ForEach(GetAllWork(), work =>
    {
        DoStuffWithWork(work);
    });
}

One of the reasons we took out official support for self replicating tasks is that we didn't see any scenarios that couldn't be solved just as well with existing constructs.

Thanks,
Danny

Answer 7

Hi Danny,

I like Tasks, maybe because  that's what i am used to.

I doub't I will find a specific example where it's extremely difficult to write using Parallel.* constructs.

I must add that in my opinion being able to write Tasks that scale to the number of cores automatically is in my mind a very useful construct.

This is the case when you have a single producer and many consumers. You want the consumers to replicate (they take longer).

I will adapt, or write something, but probably adapt since that's cheaper.

Anyway, thanks for listening.

Regards

Craig.

Answer 8

Hi Craig-

Thanks for the feedback.  Please keep it coming.

One nice thing about the Parallel loops is that they automatically handle not only scaling up (as replicable tasks did), but also scaling down.  If the thread injection/retirement logic notices that fewer threads will actually result in better throughput (for example due to too much preemption and oversubscription), or if multiple parallel operations run in parallel, the loops will be able to load balance much better than could replicable tasks.

For single producer, many consumer, a simple example of that can be coded relatively easily using Task, Parallel.ForEach, and BlockingCollection, e.g.

var bc = new BlockingCollection<Work>();
var producer = Task.Factory.StartNew(() =>
{
    try
    {
        while(NotDone()) bc.Add(ProduceWorkItem());
    }
    finally { bc.CompleteAdding(); }
});
var consumer = Task.Factory.StartNew(() =>
{<br />    var options = new ParallelOptions { MaxDegreeOfParallelism = MAX_CONSUMERS };
    Parallel.ForEach(bc.GetConsumingEnumerable(), options, work =>
    {
        ProcessWork(work);
    });
});
Task.WaitAll(producer, consumer);

Thanks,
Steve