Need to pull out vendor reference info from XML

Question

User-718146471 posted

Guys, I am stuck.  I need to bring the reference URL in if there is one.  That XML looks like this:

  <cpe-item name="cpe:/a:1024cms:1024_cms:1.4.2:beta">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.4.2 beta</title>
    <references>
      <reference href="http://12net.jp">vendor website</reference>
      <reference href="http://wordpress.org/plugins/login-rebuilder/changelog/">product changelog</reference>
    </references>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.4.2:beta:*:*:*:*:*:*"/>
  </cpe-item>

Just to clarify, when I bring these in as import, I only care about the first reference.  Also, if the reference tag is not there, I need to tell my code to skip it.  SO the XML block would look like this with no ref URL:

  <cpe-item name="cpe:/a:1024cms:1024_cms:0.7">
    <title xml:lang="en-US">1024cms.org 1024 CMS 0.7</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:0.7:*:*:*:*:*:*:*"/>
  </cpe-item>

I am doing this with C#.net.  Currently, I read the data using this code:

        protected void btnUpload_Click(object sender, EventArgs e)
        {
            string cpetype = string.Empty, manuf = string.Empty, prod = string.Empty, vers = string.Empty, cpe = string.Empty;

            XmlDocument myDoc = new XmlDocument();
            myDoc.Load(FileUpload1.FileContent);

            // Add the namespaces:
            XmlNamespaceManager nsmgr = new XmlNamespaceManager(myDoc.NameTable);
            nsmgr.AddNamespace("ns6", "http://scap.nist.gov/schema/scap-core/0.1");
            nsmgr.AddNamespace("cpe-23", "http://scap.nist.gov/schema/cpe-extension/2.3");
            nsmgr.AddNamespace("ns", "http://cpe.mitre.org/dictionary/2.0");
            nsmgr.AddNamespace("meta", "http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2");
            nsmgr.AddNamespace("scap-core", "http://scap.nist.gov/schema/scap-core/0.3");
            nsmgr.AddNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
            nsmgr.AddNamespace("config", "http://scap.nist.gov/schema/configuration/0.1");

            XmlNodeList nodeList;
            nodeList = myDoc.DocumentElement.SelectNodes("//ns:cpe-list/ns:cpe-item", nsmgr);
            long conta = 0;

            foreach (XmlNode node in nodeList)
            {
                // Access to the name ATTRIBUTE of the <cpe-item> tag:
                Response.Write(String.Format("[{0:N0}] CPE: {1}  Title: {2}", conta, node.Attributes["name"].Value, node.FirstChild.FirstChild.Value));
                // Split out the InnerText for Manufacturer, Model, and Version
                // If version is empty, use - as place holder

                // Access to the <title> tag content:
                //Debug.WriteLine(String.Format("[{0:N0}] Title: {1}  Title: {2}", conta, node.SelectSingleNode("./title", nsmgr)));
                XmlNode titleNode = node.SelectSingleNode("./title", nsmgr);

                // splitting out values

                string s = node.Attributes["name"].Value;
                s = s.Replace("/", "");
                var array = s.Split(':');
                cpetype = s[1].ToString();
                manuf = s[2].ToString();
                prod = s[3].ToString();
                vers = s[4].ToString();
                cpe = "cpe:2.3:" + cpetype.ToString() + ":" + manuf.ToString() + ":" + prod.ToString() + ":" + vers.ToString() + "";

Answer 1

User-271186128 posted

Hi bbcompent1,

when I bring these in as import, I only care about the first reference.  Also, if the reference tag is not there, I need to tell my code to skip it.

According to your description, I suggest you could use XmlNode.SelectNodes Method to check whether the current node contains the reference node. You could refer to the following code:

            foreach (XmlNode node in nodeList)
            {
                //Use SelectNodes method to find reference node.
                XmlNodeList typeNode = xn.SelectNodes(".//references");
                //check whether current node contains the reference node
                if (typeNode.Count>0)
                {
                    string str = typeNode[0].InnerText;
                }
                else
                {
                    //if it doesn't contains the reference node. skip this node.
                    continue;
                }

                // Access to the name ATTRIBUTE of the <cpe-item> tag:
                Response.Write(String.Format("[{0:N0}] CPE: {1}  Title: {2}", conta, node.Attributes["name"].Value, node.FirstChild.FirstChild.Value));
                // Split out the InnerText for Manufacturer, Model, and Version
                // If version is empty, use - as place holder

                // Access to the <title> tag content:
                //Debug.WriteLine(String.Format("[{0:N0}] Title: {1}  Title: {2}", conta, node.SelectSingleNode("./title", nsmgr)));
                XmlNode titleNode = node.SelectSingleNode("./title", nsmgr);

                // splitting out values

                string s = node.Attributes["name"].Value;
                s = s.Replace("/", "");
                var array = s.Split(':');
                cpetype = s[1].ToString();
                manuf = s[2].ToString();
                prod = s[3].ToString();
                vers = s[4].ToString();
                cpe = "cpe:2.3:" + cpetype.ToString() + ":" + manuf.ToString() + ":" + prod.ToString() + ":" + vers.ToString() + "";

                }

Best Regards,
Dillion

Answer 2

User-718146471 posted

Dillon, typeNode.Count is always 0.  For your reference, this is the data file:

<?xml version='1.0' encoding='UTF-8'?>
<cpe-list xmlns:config="http://scap.nist.gov/schema/configuration/0.1" xmlns="http://cpe.mitre.org/dictionary/2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:scap-core="http://scap.nist.gov/schema/scap-core/0.3" xmlns:cpe-23="http://scap.nist.gov/schema/cpe-extension/2.3" xmlns:ns6="http://scap.nist.gov/schema/scap-core/0.1" xmlns:meta="http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2" xsi:schemaLocation="http://scap.nist.gov/schema/cpe-extension/2.3 http://scap.nist.gov/schema/cpe/2.3/cpe-dictionary-extension_2.3.xsd http://cpe.mitre.org/dictionary/2.0 http://scap.nist.gov/schema/cpe/2.3/cpe-dictionary_2.3.xsd http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2 http://nvd.nist.gov/schema/cpe-dictionary-metadata_0.2.xsd http://scap.nist.gov/schema/scap-core/0.3 http://nvd.nist.gov/schema/scap-core_0.3.xsd http://scap.nist.gov/schema/configuration/0.1 http://nvd.nist.gov/schema/configuration_0.1.xsd http://scap.nist.gov/schema/scap-core/0.1 http://nvd.nist.gov/schema/scap-core_0.1.xsd">
  <generator>
    <product_name>National Vulnerability Database (NVD)</product_name>
    <product_version>2.33</product_version>
    <schema_version>2.3</schema_version>
    <timestamp>2015-08-29T03:50:00.191Z</timestamp>
  </generator>
  <cpe-item name="cpe:/a:1024cms:1024_cms:0.7">
    <title xml:lang="en-US">1024cms.org 1024 CMS 0.7</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:0.7:*:*:*:*:*:*:*"/>
    <references>
      <reference href="http://12net.jp">vendor website</reference>
      <reference href="http://wordpress.org/plugins/login-rebuilder/changelog/">product changelog</reference>
    </references>
  </cpe-item>
  <cpe-item name="cpe:/a:1024cms:1024_cms:1.2.5">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.2.5</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.2.5:*:*:*:*:*:*:*"/>
  </cpe-item>
  <cpe-item name="cpe:/a:1024cms:1024_cms:1.3.1">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.3.1</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.3.1:*:*:*:*:*:*:*"/>
  </cpe-item>
  <cpe-item name="cpe:/a:1024cms:1024_cms:1.4.1">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.4.1</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.4.1:*:*:*:*:*:*:*"/>
  </cpe-item>
  <cpe-item name="cpe:/a:1024cms:1024_cms:1.4.2">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.4.2</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.4.2:*:*:*:*:*:*:*"/>
  </cpe-item>
  <cpe-item name="cpe:/a:1024cms:1024_cms:1.4.2:beta">
    <title xml:lang="en-US">1024cms.org 1024 CMS 1.4.2 beta</title>
    <cpe-23:cpe23-item name="cpe:2.3:a:1024cms:1024_cms:1.4.2:beta:*:*:*:*:*:*"/>
  </cpe-item>
</cpe-list>

and here is my complete source code:

using System;
using System.Data.SqlClient;
using System.Web.Configuration;
using System.Xml;

namespace CPEReportingTool
{
    public partial class ImportDictionary : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {

        }

        protected void btnUpload_Click(object sender, EventArgs e)
        {
            string cpetype = string.Empty, manuf = string.Empty, prod = string.Empty, vers = string.Empty, cpe = string.Empty;

            XmlDocument myDoc = new XmlDocument();
            myDoc.Load(FileUpload1.FileContent);

            // Add the namespaces:
            XmlNamespaceManager nsmgr = new XmlNamespaceManager(myDoc.NameTable);
            nsmgr.AddNamespace("ns6", "http://scap.nist.gov/schema/scap-core/0.1");
            nsmgr.AddNamespace("cpe-23", "http://scap.nist.gov/schema/cpe-extension/2.3");
            nsmgr.AddNamespace("ns", "http://cpe.mitre.org/dictionary/2.0");
            nsmgr.AddNamespace("meta", "http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2");
            nsmgr.AddNamespace("scap-core", "http://scap.nist.gov/schema/scap-core/0.3");
            nsmgr.AddNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
            nsmgr.AddNamespace("config", "http://scap.nist.gov/schema/configuration/0.1");

            XmlNodeList nodeList;
            nodeList = myDoc.DocumentElement.SelectNodes("//ns:cpe-list/ns:cpe-item", nsmgr);
            long conta = 0;

            foreach (XmlNode node in nodeList)
            {
                //Use SelectNodes method to find reference node.
                XmlNodeList typeNode = node.SelectNodes("./references");
                //check whether current node contains the reference node
                if (typeNode.Count > 0)
                {
                    string str = typeNode[0].InnerText;
                }
                else
                {
                    //if it doesn't contains the reference node. skip this node.
                    continue;
                }

                // Access to the name ATTRIBUTE of the <cpe-item> tag:
                Response.Write(String.Format("[{0:N0}] CPE: {1}  Title: {2}", conta, node.Attributes["name"].Value, node.FirstChild.FirstChild.Value));
                // Split out the InnerText for Manufacturer, Model, and Version
                // If version is empty, use - as place holder

                // Access to the <title> tag content:
                //Debug.WriteLine(String.Format("[{0:N0}] Title: {1}  Title: {2}", conta, node.SelectSingleNode("./title", nsmgr)));
                XmlNode titleNode = node.SelectSingleNode("./title", nsmgr);

                // splitting out values

                string s = node.Attributes["name"].Value;
                s = s.Replace("/", "");
                var array = s.Split(':');
                cpetype = s[1].ToString();
                manuf = s[2].ToString();
                prod = s[3].ToString();
                vers = s[4].ToString();
                cpe = "cpe:2.3:" + cpetype.ToString() + ":" + manuf.ToString() + ":" + prod.ToString() + ":" + vers.ToString() + "";

                try
                {
                    // Now check to see if the CPE is already in the database
                    string CheckDB = "SELECT COUNT(*) from CPEs where CPE = @cpeval";
                    string DBConnect = WebConfigurationManager.AppSettings["DBConn"];
                    SqlConnection conn = new SqlConnection(DBConnect);
                    SqlCommand cmd = new SqlCommand(CheckDB, conn);
                    cmd.Parameters.AddWithValue("cpeval", cpe.ToString());

                    SqlDataReader dr = cmd.ExecuteReader();
                    if (dr.HasRows)
                    {

                        // TODO: Update existing CPE record with information from NIST Dictionary
                    }
                    else
                    {
                        cpetype = s[1].ToString();
                        manuf = s[2].ToString();
                        prod = s[3].ToString();
                        vers = s[4].ToString();
                        cpe = node.Attributes["name"].Value.ToString();

                        // TODO: Insert data into CPE DB Respository of reporting tool
                    }
                }
                catch (Exception ex)
                {
                    Response.Write("Uh oh, something went wrong. " + e.ToString() + "");
                }
                conta++;
            }
        }
    }
}

Answer 3

User-718146471 posted

I am still working on this but I seem to be getting nowhere fast.  Here is the code now:

        protected void btnUpload_Click(object sender, EventArgs e)
        {
            string cpetype = string.Empty, manuf = string.Empty, prod = string.Empty, vers = string.Empty, cpe = string.Empty;

            XmlDocument myDoc = new XmlDocument();
            myDoc.Load(FileUpload1.FileContent);

            // Add the namespaces:
            XmlNamespaceManager nsmgr = new XmlNamespaceManager(myDoc.NameTable);
            nsmgr.AddNamespace("ns6", "http://scap.nist.gov/schema/scap-core/0.1");
            nsmgr.AddNamespace("cpe-23", "http://scap.nist.gov/schema/cpe-extension/2.3");
            nsmgr.AddNamespace("ns", "http://cpe.mitre.org/dictionary/2.0");
            nsmgr.AddNamespace("meta", "http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2");
            nsmgr.AddNamespace("scap-core", "http://scap.nist.gov/schema/scap-core/0.3");
            nsmgr.AddNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
            nsmgr.AddNamespace("config", "http://scap.nist.gov/schema/configuration/0.1");

            XmlNodeList nodeList, nodeList2;
            nodeList = myDoc.DocumentElement.SelectNodes("//ns:cpe-list/ns:cpe-item", nsmgr);
            nodeList2 = myDoc.DocumentElement.SelectNodes("//ns:cpe-list/ns:cpe-item/ns:references", nsmgr);
            long conta = 0;

            foreach (XmlNode node in nodeList)
            {
                foreach (XmlNode node2 in nodeList2)
                {
                    XmlNode typeNode = node2.SelectSingleNode("./reference", nsmgr);
                    if (typeNode != null)
                    {
                        string str = node2.Attributes["reference"].Value;
                    }
                    else
                    {
                        continue;
                    }
                }
                // Access to the name ATTRIBUTE of the <cpe-item> tag:
                Response.Write(String.Format("[{0:N0}] CPE: {1}  Title: {2}", conta, node.Attributes["name"].Value, node.FirstChild.FirstChild.Value));

Answer 4

User-271186128 posted

Hi bbcompent1,

typeNode.Count is always 0.

Since my previous sample, I haven't added the namespace property, so you can't find the relevant node.

Please try to use the following code, I tested on my side, it worked well.

        protected void btnUpload_Click(object sender, EventArgs e)
        {
            string cpetype = string.Empty, manuf = string.Empty, prod = string.Empty, vers = string.Empty, cpe = string.Empty;

            XmlDocument myDoc = new XmlDocument();
            myDoc.Load(FileUpload1.FileContent);

            // Add the namespaces:
            XmlNamespaceManager nsmgr = new XmlNamespaceManager(myDoc.NameTable);
            nsmgr.AddNamespace("ns6", "http://scap.nist.gov/schema/scap-core/0.1");
            nsmgr.AddNamespace("cpe-23", "http://scap.nist.gov/schema/cpe-extension/2.3");
            nsmgr.AddNamespace("ns", "http://cpe.mitre.org/dictionary/2.0");
            nsmgr.AddNamespace("meta", "http://scap.nist.gov/schema/cpe-dictionary-metadata/0.2");
            nsmgr.AddNamespace("scap-core", "http://scap.nist.gov/schema/scap-core/0.3");
            nsmgr.AddNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
            nsmgr.AddNamespace("config", "http://scap.nist.gov/schema/configuration/0.1");

            XmlNodeList nodeList;
            nodeList = myDoc.DocumentElement.SelectNodes("//ns:cpe-list/ns:cpe-item", nsmgr);
            long conta = 0;

            StringBuilder sb = new StringBuilder();
            foreach (XmlNode node in nodeList)
            {
                //Use SelectNodes method to find reference node.
                XmlNodeList typeNode = node.SelectNodes(".//ns:references",nsmgr);
                //check whether current node contains the reference node
                if (typeNode.Count > 0)
                {
                    string str = typeNode[0].InnerText;
                }
                else
                {
                    //if it doesn't contains the reference node. skip this node.
                    continue;
                }

                // Access to the name ATTRIBUTE of the <cpe-item> tag:
                sb.AppendLine(String.Format("[{0:N0}] CPE: {1}  Title: {2}", conta, node.Attributes["name"].Value, node.FirstChild.FirstChild.Value));

            }

            Response.Write(sb.ToString());
        }

Best regards,
Dillion

Answer 5

User-718146471 posted

Thanks Dillon, I will give this a try!

Answer 6

User-718146471 posted

Ok, I changed the code to typeNode[0].InnerXml, this gave me the URL of the vendor site but it also gave me several other things in the string builder.

<reference href="http://12net.jp" xmlns="http://cpe.mitre.org/dictionary/2.0">vendor website</reference><reference href="http://wordpress.org/plugins/login-rebuilder/changelog/" xmlns="http://cpe.mitre.org/dictionary/2.0">product changelog</reference>

How would you go about grabbing just the first reference href and drop the rest? The only part out of the string I need is the first URL.

Answer 7

User-271186128 posted

Hi bbcompent1,

<reference href="http://12net.jp" xmlns="http://cpe.mitre.org/dictionary/2.0">vendor website</reference><reference href="http://wordpress.org/plugins/login-rebuilder/changelog/" xmlns="http://cpe.mitre.org/dictionary/2.0">product changelog</reference>

How would you go about grabbing just the first reference href and drop the rest? The only part out of the string I need is the first URL.

As for this issue, you could use String.Substring method to get the first URL. Please refer to the following code:

                //Use SelectNodes method to find reference node.
                XmlNodeList typeNode = node.SelectNodes("./ns:references",nsmgr);
                //check whether current node contains the reference node
                if (typeNode.Count > 0)
                {
                    string str = typeNode[0].InnerText;
                    string str2 = typeNode[0].InnerXml.ToString(); //output: <reference href="http://12net.jp" xmlns="http://cpe.mitre.org/dictionary/2.0">vendor website</reference><reference href="http://wordpress.org/plugins/login-rebuilder/changelog/" xmlns="http://cpe.mitre.org/dictionary/2.0">product changelog</reference>

                    int startindex = str2.IndexOf("href");
                    int endIndex = str2.IndexOf("xmlns");

                    string url = str2.Substring(startindex + 6, endIndex - startindex - 8); //output: http://12net.jp
                }
                else
                {
                    //if it doesn't contains the reference node. skip this node.
                    continue;
                }

Besides, I suppose you could also try to use Regex.Match Method to get the first URL.

Best regards,
Dillion