locked
Drop Folders - Design on how to parse RRS feed

  • Question

  • User770790503 posted

    I have a large amount of data that I want to parse through, each in a seperate file.  I'd like to create a multi-threaded application to parse these files, but I'm brain dead on a general approach on how to do this.

    I've already written the code to do the parsing, however, it's taking about 1-5 seconds per file.  How can I make sure that each thread doesn't interfere with the other?  Simply move the folder before parsing?

    Monday, April 26, 2010 9:33 AM

Answers

  • User-952121411 posted

    Good point brought up by Steve (I did not see any hardware details in op).  Unless you have a dual core+ setup or multiple single core processors, you are probably going down the wrong path with multithreading.  In a single core processor, only one task runs at any point in time.  Just make sure you have the hardware to support an actual performance gain from multithreading an application. If you have more than (1) single core processor (i.e. server) than this would be ok as well.  From purely a hardware view, take a look at the 'Single Core' and 'Multi Core' sections of the following link for more info:

    http://zone.ni.com/devzone/cda/tut/p/id/6424



    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, April 26, 2010 10:35 AM

All replies

  • User-952121411 posted

    Take a look to this thread:

    C#-APP: Multi-threading file processing:

    http://bytes.com/topic/net/answers/750241-c-app-multi-threading-file-processing

    ...and this MSFT article has some recommendations as well:

    How to synchronize access to a shared resource in a multithreading environment by using Visual C#:

    http://support.microsoft.com/kb/816161

    Monday, April 26, 2010 9:59 AM
  • User187056398 posted

    I'd like to create a multi-threaded application to parse these files,
     

    Why?  On a single processor machine, it will take longer to complete the task.

    Monday, April 26, 2010 10:20 AM
  • User770790503 posted

    This is a multi processor machine.  I've done this in the past, but it was 3 years ago and it doesn't pop out.


    I'll look at the links, thank you.

    Monday, April 26, 2010 10:30 AM
  • User-952121411 posted

    Good point brought up by Steve (I did not see any hardware details in op).  Unless you have a dual core+ setup or multiple single core processors, you are probably going down the wrong path with multithreading.  In a single core processor, only one task runs at any point in time.  Just make sure you have the hardware to support an actual performance gain from multithreading an application. If you have more than (1) single core processor (i.e. server) than this would be ok as well.  From purely a hardware view, take a look at the 'Single Core' and 'Multi Core' sections of the following link for more info:

    http://zone.ni.com/devzone/cda/tut/p/id/6424



    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, April 26, 2010 10:35 AM