How to call parent child task multiple time

Question

Hi,

I want to process multiple files and each file has to go thru various steps(parsing, scrubbing etc.) I have written the logic as below: I know this is not correct as it's failing while I try to capture the exception being thrown by a file on its step(parsing, scrubbing ).

I need to know how can I implement this logic in TPL(Task parallel Programming)

Try

Do While totalCountofFilesinQ > 0

               Try

                    Dim filetoProcessonSeparateTask As String

                    filetoProcessonSeparateTask = "File" + counterfilesProcessed.ToString()

                    TaskManager = task.Factory.StartNew(Function()

            Console.WriteLine("Processing for File: " + filetoProcessonSeparateTask + " started ")

                                                            Dim fileParsing As Task(Of String) = task(Of String).Factory.StartNew(Function()

                                                                                                                                      FileParsingprocess(filetoProcessonSeparateTask)

                                                                                                                                  End Function, TaskCreationOptions.AttachedToParent, token)

                                                            Console.WriteLine("Sentry Task Manager Process.")

                                                            Dim mapScrub As Task(Of String) = Task(Of String).Factory.StartNew(Function()

                                                                                                                                   Mapscrubprocess(filetoProcessonSeparateTask)

                                                                                                                               End Function, TaskCreationOptions.AttachedToParent, token)

                                                        End Function

                                                        )                                

 

                    counterfilesProcessed += 1

                    totalCountofFilesinQ = totalCountofFilesinQ - 1

                    Console.WriteLine("---------------------------------")

                    Console.WriteLine("File Processed >> :  " + filetoProcessonSeparateTask.ToString())

                     

            Catch ex as exception

       End Try

      

Loop

Catch aex As AggregateException

            Console.BackgroundColor = ConsoleColor.Red

            Console.WriteLine("Task Manager has noticed the below errors thrown by child processes: " & aex.InnerExceptions.Count())

            For Each ex In aex.InnerExceptions

                Console.WriteLine("Exception thrown by child processes: " & ex.Message)

            Next

End try

Can anybody look into this and let me know how I can do this or what I am not doing the capture the exception which are being thrown by child task while processing of file at any stage.

I really need this. Please reply.

Answer 1

First, I think you're posting to incorrect forum, your code doesn't use TPL Dataflow (a library new in .Net 4.5), just TPL.

Regarding your code, I think it's quite convoluted and it's not even right, assuming the second phase has to follow the first phase, because you're starting the second phase without waiting for the first phase to finish.

And you're not catching exceptions correctly. Exceptions from Tasks are surfaced then you Wait() on them or when you access their Result. If you don't do that and the Task is garbage collected, it throws an exception on the finalizer thread (in .Net 4.0, .Net 4.5 changes this).

I think the best way to rewrite this would be to use Parallel.For and don't create any child tasks, just run the methods directly in the loop:

Parallel.For(0, totalCountofFilesinQ,
                Sub(i)
                    Dim filetoProcessonSeparateTask = "File" + i.ToString()

                    Console.WriteLine("Processing for File: " + filetoProcessonSeparateTask + " started ")

                    FileParsingprocess(filetoProcessonSeparateTask)

                    Console.WriteLine("Sentry Task Manager Process.")

                    Mapscrubprocess(filetoProcessonSeparateTask)

                    Console.WriteLine("---------------------------------")
                    Console.WriteLine("File Processed >> :  " + filetoProcessonSeparateTask)
                End Sub)

If you do this, you can enclose the whole loop in a Try-Catch and catch the AggregateException.

Answer 2

Thanks for the reply. My child task (i.e. various steps in fileprocessing which are fileparsing, Scrub. etc.) are dependenet on the previous one. If the File Parsing is completed successfully only then the Scrub task should continue.  This should happen parallely for all files in the queue. And the exception should be caught for each processing.

Is it possible usign the Parallel..For

Please reply.

Answer 3

Also will the below code run on two different tasks or just one task.

FileParsingprocess(filetoProcessonSeparateTask)
Console.WriteLine("Sentry Task Manager Process.")
Mapscrubprocess(filetoProcessonSeparateTask)

Say there are 5 files so only 5 task will be open parallel. Is it correct?

Answer 4

If an exception is thrown in one of the iterations, no new iterations will be started. It seems that's not what you want, so you need to catch the exception inside the loop code. This way, the processing for one file won't continue if an exception is thrown, but other files won't be affected.

The processing won't happen for all files at once, because that's usually not the most efficient approach. The library will process the files in parallel, but not all of them at once.

And yes, if you have five files, there will be at most five tasks, calling a normal method inside a task doesn't create another task.

Answer 5

Can you please redirect me the sample piece for code, I am very new in Parallel Programming. This is what i want to build. Thanks a lot in advance

Step1: I got the request for 5 files (e.g.) Some of these client data files will be small and some might be too large. So If a large file is running, it should not block the resources for other files procesing to get started. Being said that all 5 files should be put on different task to start their processing. 

Step 2: If any exception is encountered by any one file on its path, it should not stop the processing of other files which are running, the program should report gracefully that out of 5 files being sent in the queue 3 are processed, 2 are not due to some exception.

Can you please point me to some code link which looks near to what I am looking for. I am really on a tight deadline with this project. Please reply.

Answer 6

Does svick's solution not work for you?  The Parallel.For approach sounds like it does what you want (you'd just use a try/catch inside the delegate passed to Parallel.For rather than around than outside of the call to Parallel.For)... you could also use PLINQ, for example:

IEnumerable<string> filePaths = ...;
string results = filePaths.AsParallel().Select(filePath =>
{
    try
    {
        var parsed = Parse(filePath);
        Process(parsed);
        return "Successfully processed " + filePath;
    }
    catch(Exception exc)
    {
        return "Processing " + filePath + " failed with error: " + exc.Message;
    }
});
foreach(var result in results)
    Console.WriteLine(result);