none
Is there a way to disable the Parallel.For's in the Concurrency Visualizer?

    Question

  • I am using the Concurrency Visualizer to help identify sections of code that can be cleaned up.  The Scenario object on MSDN is great for marking the beginning and end of processing regions. Unfortunately, the Visualizer is also picking up the Parallel.For and PLINQ markers as well.  Is there any way to get the Visualizer to ignore the internal Scenario markers so that I can just focus on my own?  If I want to see where a Parallel loop is being executed, I'll insert the marker myself ;)

     

    Thanks,

    Flevine


    flevine
    Wednesday, June 09, 2010 4:52 PM

Answers

  • Hi Flevine,

    Unfortunately there isn't a way to disable Scenario Markers for Parallel.For loops and PLINQ queries.  I'm curious to hear more about your scenario.  In what context are you using Scenario Markers to instrument your code?

    -James

    • Marked as answer by flevine Wednesday, June 09, 2010 6:18 PM
    Wednesday, June 09, 2010 5:24 PM

All replies

  • Hi Flevine,

    Unfortunately there isn't a way to disable Scenario Markers for Parallel.For loops and PLINQ queries.  I'm curious to hear more about your scenario.  In what context are you using Scenario Markers to instrument your code?

    -James

    • Marked as answer by flevine Wednesday, June 09, 2010 6:18 PM
    Wednesday, June 09, 2010 5:24 PM
  • James,

      Thanks for the reply.  I have been working on a basic processing pipeline for text files.  Our application needs to process multiple large text files (~200-500 mb each).  Using File.ReadAllLines() isn't an option because we don't want to read the entire file into memory, and calling ReadLine() repeatetly is just too slow.  Also, each line is delimited, but I don't want to use string.split().  Each record has ~300 fields in it, and I'm only interested in 5-10, so it doesn't make sense to incur the string.split() cost.  Instead, I scan the record for the delimiters manually and pull out the fields that I need.  Certainly not as elegant as string.split(), but it's a whole lot faster (at least the profiler says it is!).

    Anyways, I used Tasks, MemoryMappedFile's and BlockingCollections to create a simple pipeline to process the records:

    Task 1 = Load chunks of the file into memory using MemoryMappedFile() and ReadLine() the lines from memory.  Every 50,000 lines or so, I put the lines that have been read into a BlockingCollection<string[]>.

     

    Task 2 = Take the string[]'s from the BlockingCollection and transform them into the records that I want.

     

    Before using the Scenario Markers, it was very hard to figure out what was going on in the thread view.  Sure, you can click on the colors and see who is blocking what, but it was still hard to figure out.  Once I started using scenario markers, things became much more clear.  I have 2 Scenario markers in my code, one that marks the 'Reading Blocks' section of code, and the other that marks the 'Making Records'.  In my previous tests, I had been pulling out a small number of fields from each record.  When I tried pulling 30-40 fields per record, I noticed that the gap between the end of reading the file and the end of making the records was very long... and it was all on the same thread.  A simple change in the structure of Task2 to process the string[] using a Parallel.For worked great, and I'm now happily making records on all 4 cores.  Unfortunately, my once clean thread view now has a bunch of Parallel.For's on it ;(

    Maybe in a future version or service pack we can remove the markers, or make it an option?


    flevine
    Wednesday, June 09, 2010 6:17 PM
  • Flevine,

    Thanks for the detailed response!  We are aware (thanks to feedback such as yours) that there are areas upon which we can improve Scenario Markers for future releases.  We may or may not end up being able to spend time on this specific feature, but every bit of feedback we can get helps to inform our thinking.

    Thanks and best of luck!

    -James

    Wednesday, June 09, 2010 9:26 PM
  • James - no problem.  It's not really and issue unless you get lots of markers in the same run.  When that happens, the visualizer stops displaying them (similar to the 32,000 points in a graph in Excel!).  If there's going to be an upper limit on the number of markers that can be displayed, I would like to have control of all of markers, rather than Parallel.For and PLINQ auto-inserting them for me.  Markers are the best way that I have found (so far) to get a good grip on the timing and phasing of the threads, and I would definitely vote for them to be included directly in a future release.  They have saved me a ton of time already.


    flevine
    Thursday, June 10, 2010 2:37 AM