none
How to update multiple xml files simultaneously by thread RRS feed

  • Question

  • Supposed in a list i have stored many xml files path. I need to update each xml files in separate thread.  So how could i develop a multi threading program which will update each xml file in separate thread. So when i will run progrthen all xml files will be updating parallel.  Also race condition should not occur.

    I will use xdocument class to load xml file and in foreach i will add or update xml element. A sample code attached hete which show how i am updating xml file. So i just need to know how could i update multiple xml files at parallel.  Please help me with sample code.  Thanks

    {
                        XDocument xmlDoc = XDocument.Load(path);
    
                        //iterate in return data from db
                        foreach (var data in returndata)
                        {
                            //query 10QK xml data based on section & lineitem
                            var items = (from item in xmlDoc.Descendants("TickerBrokerStandardDateLineitemValue")
                                         where item.Element("TabName").Value.Trim() == data.Section
                                         && item.Element("StandardLineItem").Value.Trim() == data.LineItem
                                         select item).ToList();
    
    
                            foreach (var item in items)
                            {
                                //element will be inserted or updated in xml when match found
                                if (item.Element("SectionID") == null)
                                {
                                    //if SectionID element does not exist then it will be added in xml having ID 
                                    item.Add(new XElement("SectionID", data.SectionID));
                                }
                                else
                                {
                                    //if SectionID element exist then it will be updated with db value
                                    item.Element("SectionID").SetValue(data.SectionID);
                                }
    
                                //element will be inserted or updated in xml when match found
                                if (item.Element("LineItemID") == null)
                                {
                                    //if LineItemID element does not exist then it will be added in xml having ID 
                                    item.Add(new XElement("LineItemID", data.LineItemID));
                                }
                                else
                                {
                                    //if LineItemID element exist then it will be updated with db value
                                    item.Element("LineItemID").SetValue(data.LineItemID);
                                }
                                
                                if (data.Approved == 1)
                                {
                                    //if approved then xfundcode will be updated
                                    if (item.Element("XFundCode") != null)
                                    {
                                        //if XFundCode element exist then it will be updated with db value
                                        item.Element("XFundCode").SetValue(data.ApprovedXFundCode);
                                    }
                                }
                                else if (data.Approved == 0)
                                {
                                    //if unapproved then xfundcode will be empty
                                    if (item.Element("XFundCode") != null)
                                    {
                                        //if XFundCode element exist then it will be updated with db value
                                        item.Element("XFundCode").SetValue(string.Empty);
                                    }
                                }
                            }
                        }
    
                        xmlDoc.Save(path);
    

    Friday, December 6, 2019 8:05 PM

Answers

  • Consider this approach too:

    List<string> listOfFilePaths = new List<string> { @"C:\MyFolder1\File1.xml", @"D:\File2.xml", . . . };
    
    Parallel.ForEach( listOfFilePaths, path => ProcessOneFile( path ) );
    

    where ProcessOneFile is your thread-safe function that processes one file.

    • Marked as answer by Sudip_inn Tuesday, December 10, 2019 8:01 AM
    Friday, December 6, 2019 8:45 PM
  • What you have to do is look at each line of code in the loop, and figure out whether it is using a shared resource in a way that could conflict with other threads.  There's no magic shortcut for that.

    In your case,you're sharing the "returndata" object that contains your data table records.  If that is a live connection to a database, then YOU BET that's going to conflict, because each time a thread pulls another piece of data, it advances the server connections, affecting the other threads.

    But if returndata is a list or array -- a static data structure containing all of the records in your data set -- then there is no problem.  Each thread gets its own pointer into the list, and you aren't changing the list.


    Tim Roberts | Driver MVP Emeritus | Providenza &amp; Boekelheide, Inc.

    • Marked as answer by Sudip_inn Tuesday, December 10, 2019 8:00 AM
    Monday, December 9, 2019 9:41 PM

All replies

  • Consider this approach too:

    List<string> listOfFilePaths = new List<string> { @"C:\MyFolder1\File1.xml", @"D:\File2.xml", . . . };
    
    Parallel.ForEach( listOfFilePaths, path => ProcessOneFile( path ) );
    

    where ProcessOneFile is your thread-safe function that processes one file.

    • Marked as answer by Sudip_inn Tuesday, December 10, 2019 8:01 AM
    Friday, December 6, 2019 8:45 PM
  • I think processonefile functions will process one file at a time. Then what will be advantage?  I need to update multiple files  at same time.

    I guess parallel foreach may create race condition. If my assumption is wrong then please tell me the reason why race condition will not occur?

    If i use lock inside parallel foreach then i can process single file at a time and objectives of parallel  foreach will not be fulfill. Looking for your further guideline. Thanks

    • Edited by Sudip_inn Saturday, December 7, 2019 6:36 PM
    Saturday, December 7, 2019 6:28 PM
  • Hi Sudip_inn,
    Using Parallel.ForEach method will call multiple threads.
    if there is no lock, it will have contention for resources. Besides, it may have other uncontrollable situations. So this is a not a good choice.
    Best Regards,
    Daniel Zhang

    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, December 9, 2019 8:52 AM
  • so how could i update multiple xml file using multi threading without locking ?
    Monday, December 9, 2019 6:34 PM
  • What you have to do is look at each line of code in the loop, and figure out whether it is using a shared resource in a way that could conflict with other threads.  There's no magic shortcut for that.

    In your case,you're sharing the "returndata" object that contains your data table records.  If that is a live connection to a database, then YOU BET that's going to conflict, because each time a thread pulls another piece of data, it advances the server connections, affecting the other threads.

    But if returndata is a list or array -- a static data structure containing all of the records in your data set -- then there is no problem.  Each thread gets its own pointer into the list, and you aren't changing the list.


    Tim Roberts | Driver MVP Emeritus | Providenza &amp; Boekelheide, Inc.

    • Marked as answer by Sudip_inn Tuesday, December 10, 2019 8:00 AM
    Monday, December 9, 2019 9:41 PM