locked
encoding xml in webrequest

    Question

  • I am trying trying to send xml in a webrequest object to a 3rd party system, who receives the request via an .asp page.  They sent us a sample asp script that utilizes the ms xmlhttp object.  In their sample code they encode the xml that is to be sent to them using utf-16.  When I try to encode the xml in my webrequest object using UnicodeEncoding, I never get a response.   If I change the encoding object to UTF8Encoding and the PI to utf-8 (or uncomment the StartDocument() method call), it works.  But the response is empty when it is utf-16 and Unicode.  The asp code works, but the following does not, any ideas? 

    HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(this._url);
    UnicodeEncoding encoding = new UnicodeEncoding(true, true);//I've tried false on both
    //UTF8Encoding encoding = new UTF8Encoding();
    byte[] byte1 = encoding.GetBytes(this._xml);//this._xml defined below
    request.ContentType = "text/xml";
    request.ContentLength = byte1.Length;
    request.Method = "POST";
    Stream newStream = request.GetRequestStream();
    newStream.Write(byte1, 0, byte1.Length);
    HttpWebResponse searchResults = (HttpWebResponse)request.GetResponse();
    Stream responseStream = searchResults.GetResponseStream();
    StreamReader reader =
    new StreamReader(responseStream, Encoding.Unicode);
    newStream.Close();
    string results = reader.ReadToEnd();

    .....

    System.IO.StringWriter sw = new System.IO.StringWriter();
    MemoryStream memoryStream =
    new MemoryStream();
    XmlTextWriter xmlTextWriter =
    new XmlTextWriter(memoryStream, Encoding.Unicode);
    xmlTextWriter.Formatting = Formatting.Indented;
    //xmlTextWriter.WriteStartDocument();
    xmlTextWriter.WriteProcessingInstruction("xml", "version='1.0' encoding='UTF-16'");
    xmlTextWriter.WriteStartElement("REQUEST");
    xmlTextWriter.WriteElementString("TITLEWORDS", "");
    xmlTextWriter.WriteElementString("TITLEWORDS_ALL", "");
    xmlTextWriter.WriteEndElement();
    //xmlTextWriter.WriteEndDocument();
    xmlTextWriter.Flush();
    StreamReader streamReader =
    new StreamReader(memoryStream, Encoding.Unicode);
    memoryStream.Position = 0;
    this._xml = streamReader.ReadToEnd();
    xmlTextWriter.Close();

    Sample asp code provided by 3rd party.

    Dim searchRequest, objHttp, objXML, objXMLDocument, objPI, objXMLElement
    Set objPI = searchRequest.createProcessingInstruction("xml", "version='1.0' encoding='UTF-16'")

    searchRequest.appendChild(objPI)

    ...

    Set objHttp = Server.CreateObject("Microsoft.XMLHTTP")

    objHttp.Open "POST", "http://3rdPartySite.com/3rdParyPage.asp", False

    objHttp.send (searchRequest)

    strRspXML = objHttp.responseText

    Wednesday, February 15, 2006 3:25 PM

Answers

  • This code is currently 1.1 and don't have plans to move to 2.0.  I will try Fiddler, to sniff out the request and see what the classic asp is sending.  I think my issue with removing the extra buffer and getting the ArgumentException might be perhaps due to the order in which I'm creating objects, especially with the byte array for the ContentLength property, and the writing of the xml to the request stream.  I am just prototyping now, I'm sure when we write this into production quality, I should be able to separate the functionality for re-use and remove the extra buffer.

    Thanks for your help.

    Thursday, February 16, 2006 4:08 PM

All replies

  • I think that ASP code doesn't really work as advertised—XMLHTTP uses UTF-8 as default encoding for send(), if any.

    That would explain why your code works with UTF-8 as well (though it's strange that the XML processor on the server side ignores the wrong encoding declaration in the ASP case). 

    BTW, you could write the XML directly to the request stream without another buffer in between.

    .NET 1.1

    XmlTextWriter writer =
      new XmlTextWriter(requestStream, new UTF8Encoding(false));

    .NET 2.0

    XmlWriterSettings settings = new XmlWriterSettings();
    settings.Indent =
    true
    ;
    settings.IndentChars =
    " "
    ;
    settings.Encoding =
    new UTF8Encoding(false
    );
    XmlWriter writer = XmlWriter.Create(requestStream, settings);

    Wednesday, February 15, 2006 6:03 PM
  • Interesting, I wonder if the developers at the 3rd party site realize that.  I haven't seen their asp page that receives the request so I can't comment on the ignoring.

    So in your suggestion, is the requestStream what I had as "newStream" in my code sample?  If so I keep getting ArgumentExceptions saying that the stream was not readable.  What are the performance impacts of the extra buffer in between?  When this goes to production it needs to perform at an enterprise level, ie many many requests per second during peak hours.

    Thanks

    Wednesday, February 15, 2006 7:47 PM
  • Yes, requestStream should have been called newStream.

    Concerning your ArgumentException—that's weird. An XmlWriter surely won't require a readable stream. BTW, are you using .NET 1.1 or .NET 2.0?

    Save from unnecessary complexity (you neither need it nor does it make the code in any way more maintainable), the rammification for using a buffer is certainly additional memory consumption, which means there's more work to do for the GC.

    Regarding the 3rd party site: You could use Fiddler to capture a request sent by the ASP code and check the encoding. If it was UTF-16, it should start with a byte order mark, either 0xFFFE or 0xFEFF.

     

    Wednesday, February 15, 2006 8:36 PM
  • This code is currently 1.1 and don't have plans to move to 2.0.  I will try Fiddler, to sniff out the request and see what the classic asp is sending.  I think my issue with removing the extra buffer and getting the ArgumentException might be perhaps due to the order in which I'm creating objects, especially with the byte array for the ContentLength property, and the writing of the xml to the request stream.  I am just prototyping now, I'm sure when we write this into production quality, I should be able to separate the functionality for re-use and remove the extra buffer.

    Thanks for your help.

    Thursday, February 16, 2006 4:08 PM