none
Trying to read nested xml into Dataset RRS feed

  • Question

  • Hello,

    I am trying to load this xml into a dataset. I don't get any hard errors but it just doesn't finish. I have let him execute the function ReadXml for over an hour without him getting done at all. The xml is like 6 MB in size so it should not be an issue of size. However the fact that it throws no error confuses me and I am not sure whats the issue.

    Anyone here has an Idea?

    I am trying to read it into dataset that way:

                   //doc type is XmlDocument

                   var doc = JsonConvert.DeserializeXmlNode(json, rootNode);

                    DataSet ds = new DataSet();

                    using (var xmlReader = new XmlNodeReader(doc))
                    {
                        ds.ReadXml(xmlReader);

                       //he doesn't throw an error but he does not get out of that function either. I have let him run it for over an hour with no result at all.

                    }

    Best regards


    Dev86 Technical Blog

    Thursday, August 1, 2019 11:39 AM

All replies

  • Hello,

    At least for me, I see no valid XML when viewing the page, only a lot of numbers.


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Thursday, August 1, 2019 1:18 PM
    Moderator
  • Hi,

    A DataSet is known te be quite slow for loading a large amount of data like that. It takes a long time because the DataSet will infer the schema from the XML.

    Do you need the result to be a dataset?

    Otherwise deserializing the XML file to objects from classes could also be an option?

    Kind regards,

    Johnny Hooyberghs 

    Thursday, August 1, 2019 3:50 PM
  • Hi Q-tec, 

    Thank you for posting here.

    Based on your description, I need more information to reproduce your problem.

    >> JsonConvert.DeserializeXmlNode(json, rootNode);

    Could you provide some code about ‘json’ and ‘rootNode’ ?

    Besides, Could you provide the XML with the correct format? It will help us to do the test.

    We are waiting for your update.

    Best Regards,

    Xingyu Zhao

    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, August 2, 2019 5:33 AM
    Moderator
  • Hi guys,

    sorry for the delay. It was my bad. I did totally forgot that the xml is probably not in a good shape for the function.

    I could provide the json but I don't think it will help much.

    Maybe I should explain a bit more.

    I am trying to write a third party application for a software. The company is providing "database dumps" in form of yaml files. Which is really really but. Don't ask me why they use yaml as a dump format.

    However in order to import it into my database I at least need xml for the import. I was looking for a way to easily convert the yaml content into xml. But all the solutions did not statisfy me. The only reliable thing i found was a converter to json. So I decided to convert it to json and then to xml.

    The xml itself has a structure. The root node is blueprints. Sadly the child nodes are the names of the blueprints and within each child are their properties (allways the same).

    Most likely the dataset import fuction is trying to create a table for each child of the root node (since the node names are allways different) which is the problem.

    So I would have to manually go though each child of "blueprints" and parse it manually.


    Dev86 Technical Blog

    Thursday, August 22, 2019 12:26 PM
  • Hi Q-tec, 

    Thanks for your feedback.

    According to your description, I make a simple test on my side, and I convert yaml to json and then convert json to xml in my test.

    Here’s my code:

            static void Main(string[] args)
            {
                var r = new StringReader(@"
                    scalar: a scalar
                    sequence:
                    - one
                    - two
                    ");
                var deserializer = new Deserializer();
                var yamlObject = deserializer.Deserialize(r);
    
                var serializer = new Newtonsoft.Json.JsonSerializer();
                StringWriter sw = new StringWriter();
                serializer.Serialize(sw, yamlObject);
                Console.WriteLine(sw.ToString());
                string jsonstr ="["+ sw.ToString() + "]";
                jsonstr = jsonstr.Replace("\"","'");
                jsonstr = "{\'root \':" + jsonstr + "}";
                XmlDocument xdoc = JsonConvert.DeserializeXmlNode(jsonstr); 
                Console.ReadKey();
            }

    Result :

    If I have any misunderstanding,

    >>The xml itself has a structure. The root node is blueprints. Sadly the child nodes are the names of the blueprints and within each child are their properties (allways the same).

    Please describe more clearly about what you need, and it will help us to make a test.

    Best Regards,

    Xingyu Zhao



    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    Tuesday, August 27, 2019 8:57 AM
    Moderator
  • Hello Zhao,

    thank you for your help. Sadly I did not have time to continue this private project of mine yet.

    I am trying to read each bpc entry from that xml. Sadly the format is horrible and somehow false. The json tool I used for translating the yaml to xml translated it the way you saw (link from first post.).

    I do understand why the dataset.read runs endless imply because the xml format is invalid and he is most likly creating tons of tons of tables.

    Sadly he is using the names as xml nodes.

    Either I would have to repair the xml layout from:

      <_x0036_81>
        <activities>
          <copying>
            <time>480</time>
          </copying>
          <manufacturing>
            <materials>
              <quantity>86</quantity>
              <typeID>38</typeID>
            </materials>
            <products>
              <quantity>1</quantity>
              <typeID>165</typeID>
            </products>
            <time>600</time>
          </manufacturing>
          <research_material>
            <time>210</time>
          </research_material>
          <research_time>
            <time>210</time>
          </research_time>
        </activities>
        <blueprintTypeID>681</blueprintTypeID>
        <maxProductionLimit>300</maxProductionLimit>
      </_x0036_81>

    to

      <bpc>
        <activities>
          <copying>
            <time>480</time>
          </copying>
          <manufacturing>
            <materials>
              <quantity>86</quantity>
              <typeID>38</typeID>
            </materials>
            <products>
              <quantity>1</quantity>
              <typeID>165</typeID>
            </products>
            <time>600</time>
          </manufacturing>
          <research_material>
            <time>210</time>
          </research_material>
          <research_time>
            <time>210</time>
          </research_time>
        </activities>
        <blueprintTypeID>681</blueprintTypeID>
        <maxProductionLimit>300</maxProductionLimit>
        <name>_x0036_81</name>
      <bpc>

    or I will have to iterate all blue print entries through myself and create the dataset.

    best regards

    Simon


    Dev86 Technical Blog

    Tuesday, October 1, 2019 11:28 AM