none
WSMAN XPRESS Remote Shell Compression RRS feed

  • Question

  • Hi All,

    According to protocol document [ms-wsmv] section 3.2.4.1.19 clients may send an rsp:CompressionType soap header (set to 'xpress') as part of the wst:Create request when creating a Remote Shell and the server must compress the data in the ReceiveResponse using the algorithm.
    The [ms-wsmv] document apparently has a compression example (section 4.5) - but the body is empty. The [ms-wsmv] document also refers to the algorithm as COMP_ALG_W2K3 in MS-XCA, however, there is no mention of that algorithm there, although LZZ algorithms are discussed. Having researched the topic a little, I found a reference to COMP_ALG_W2K3 in 4.1.10.5.19 CompressOrDecompressWin2k3 of [MS-DRSR] - that is LZ77 with DIRECT2 encoding.

    With that in mind, I created a WSMAN remote shell and specified the rsp:CompressionType as  'xpress' and send the command 'ipconfig /all'

    The key parts of the rsp:ReceiveResponse are shown below:

    <s:Envelope xml:lang="en-GB" ....
    <s:Body>
      <rsp:ReceiveResponse>
      <rsp:Stream Name="stdout" CommandId="8B1CD50B-F0BA-44B5-9718-1D4387274C58">HQAdAA0KV2luZG93cyBJUCBDb25maWd1cmF0aW9uDQoNCg==</rsp:Stream>
      <rsp:Stream Name="stdout" CommandId="8B1CD50B-F0BA-44B5-9718-1D4387274C58">MwAzACAgIEhvc3QgTmFtZSAuIC4gLiAuIC4gLiAuIC4gLiAuIC4gLiA6IEJQMUxDU0FQMDE5DQo=</rsp:Stream>
      ....
      <rsp:CommandState CommandId="8B1CD50B-F0BA-44B5-9718-1D4387274C58" State="http://schemas.microsoft.com/wbem/wsman/1/windows/shell/CommandState/Done"><rsp:ExitCode>0</rsp:ExitCode></rsp:CommandState>
    </rsp:ReceiveResponse>
    </s:Body>
    </s:Envelope>

    As we know, the rsp:Stream content are BASE64 encoded. Decoding those gives the following bytes:

    1d:00:1d:00:0d:0a:57:69:6e:64:6f:77:73:20:49:50:20:43:6f:6e:66:69:67:75:72:61:74:69:6f:6e:0d:0a:0d:0a
                \r \n  W  i  n  d  o  w  s     I  P     C  o  n  f  i  g  u  r  a  t  i  o  n \r \n \r \n

    According to CompressOrDecompressWin2k3 the first 4 bytes (1d001d00) are a bitmask that indicates to whether the next byte to be processed is data, or if the next byte (or series of bytes) is metadata. From inspection, its clear this buffer is not compressed with this algorithm. Rather it not compressed at all. As it happens the first 2 bytes appear encode the length of the buffer (29 bytes). So if we base64 decode and look at the next rsp:Stream it begins with the following 4 bytes, 33:00:33:00, that stream is 0x33 bytes long and is also uncompressed.

    I suspect the first 4 bytes encode the uncompressed and compressed size of the buffer. So, if the first 2 bytes are the same as the following 2 bytes we know the buffer is not compressed. Lets assume the algorithm only uses compression when the buffer is sufficiently large to justify the overhead.

    To test this, I generated a 64KB file with very low entropy containing the ASCII sequence 'abc' repeated. An ideal candidate for compression. I sent a command to output the contents of the file to stdout in the shell ('cat %temp%\64.bin'). The base64 bytes of the first rsp:Stream begin with:

    52:55:06:01:00:00:00:00...

    So, lets assume the first 2 bytes are the uncompressed size (21842) , followed by 2 bytes to represent the compressed size (262). Since they are different we assume the buffer is now compressed. From inspection the buffer appears to be compressed with the LZ77+Huffman encoding described in [MS-XCA].

    Questions:
    [ms-wsmv] Section 4.5 - Compression Example appears to be missing the example, is there an example?
    [ms-wsmv] refers to COMP_ALG_W2K3 in MS-XCA but there is no mention of COMP_ALG_W2K3 there
    What is the algorithm used to compress the rsp:Stream and how or when does the algorithm decide to use compression?
    (Do the leading 4 bytes encode the uncompressed and compressed sizes)

    Thanks

    Ian

    Thursday, October 16, 2014 11:18 AM

Answers

  • Forum update:

    This issue is now resolved. Here is the algorithm that is employed to perform Remote Shell Compression (http://msdn.microsoft.com/en-us/library/ee878659.aspx)

    1. The first 4 bytes represent the (length of the original message-1) and the (length of the compressed message – 1).
    2. Anything smaller than 266 is not compressed
    3. If the size of the original data is 266 or larger, LZ77+Huffman algorithm described in MS-XCA (http://msdn.microsoft.com/en-us/library/hh554002.aspx) is used to compress data.
    4. If the total compressed size above is more than input data/chunk, the data is written as is. In this case original message length = compressed message length. Likewise, such encoded block should be read as is (instead of processing through de-compressor)

    5. The stream is compressed in 64KB chunks. So, for example, a 100KB input data is compressed in 2 separate blocks.

    A future release of MS-WSMV document will incorporate these changes.


    Regards, Obaid Farooqi

    Friday, October 24, 2014 4:49 PM
    Owner

All replies

  • Hello Ian,
    Thank you for your inquiry about MS-WSMV specification. One of the Open specifications team member will contact you shortly.

     
    Regards,
    Sreekanth Nadendla
    Microsoft Windows Open specifications

    Thursday, October 16, 2014 2:18 PM
    Moderator
  • Hi Ian:

    I'll help you with this issue and will be in touch as soon as I have an answer.


    Regards, Obaid Farooqi

    Thursday, October 16, 2014 5:32 PM
    Owner
  • Hi Ian:

    I have filed bugs against MS-WSMV to correct the issues you observed in the document. To give you an explanation of the compression algorithm used to compress the data, I need some traces from you.

    I'll set up a workspace and send you instructions on how to collect traces and send them to me.


    Regards, Obaid Farooqi

    Monday, October 20, 2014 9:57 PM
    Owner
  • Forum update:

    This issue is now resolved. Here is the algorithm that is employed to perform Remote Shell Compression (http://msdn.microsoft.com/en-us/library/ee878659.aspx)

    1. The first 4 bytes represent the (length of the original message-1) and the (length of the compressed message – 1).
    2. Anything smaller than 266 is not compressed
    3. If the size of the original data is 266 or larger, LZ77+Huffman algorithm described in MS-XCA (http://msdn.microsoft.com/en-us/library/hh554002.aspx) is used to compress data.
    4. If the total compressed size above is more than input data/chunk, the data is written as is. In this case original message length = compressed message length. Likewise, such encoded block should be read as is (instead of processing through de-compressor)

    5. The stream is compressed in 64KB chunks. So, for example, a 100KB input data is compressed in 2 separate blocks.

    A future release of MS-WSMV document will incorporate these changes.


    Regards, Obaid Farooqi

    Friday, October 24, 2014 4:49 PM
    Owner