locked
Queue Service methods that require specifying a Pop Receipt will fail if the Pop Receipt contains a “+” character RRS feed

  • Question

  • With the Azure Queue Service REST API, whenever you perform a Get Message action on a queue, the message(s) you get back have a “pop receipt”. This value is used downstream any time you want to call Update Message or Delete Message. These functions all appear to function normally most of the time. However, we’ve found that whenever the pop receipt contains a “+” character, we get an authentication failure when calling Update Message or Delete Message with that pop receipt. And the pop receipt is a string given to us by the Get Message method, so to our knowledge, we have no control over what characters are present in any given pop receipt.

    We originally thought that maybe there was a bug in our authentication approach, but we have verified that our authentication algorithm generates exactly the same hash as the .NET implementation of HMAC-SHA-256.

    We were able to narrow down the issue to the “+” character in the pop receipt because whenever you get an authentication error from Azure, the error message returns the string that the server thought you were trying to send, based on decoding the hash. So for example, if we are trying to send an Update Message request, this is what the authenticated string looks like:

    PUT

     

     

    59

     

     

     

     

     

     

     

     

    x-ms-date:Mon, 23 Oct 2017 18:10:05 GMT

    x-ms-version:2016-05-31

    /XXXXX/queue-unit-tests-814240410/messages/879948f7-783c-4d91-bcdb-5ac673265ab0

    popreceipt:AgAAAAMAAAAAAAAAZzkPKCpM0wE=

    visibilitytimeout:2

    This string works fine, and note that the popreceipt value is “AgAAAAMAAAAAAAAAZzkPKCpM0wE=”. Now here is the same Update Message request, but this one uses a pop receipt with a “+” character:

    PUT

     

     

    59

     

     

     

     

     

     

     

     

    x-ms-date:Mon, 23 Oct 2017 18:12:19 GMT

    x-ms-version:2016-05-31

    /XXXXX/queue-unit-tests-351957909/messages/adf08eff-f7bd-41a3-98c2-0fef6b8fc152

    popreceipt:AgAAAAMAAAAAAAAAKZS+dypM0wE=

    visibilitytimeout:2

     

    This request fails, with the following error message:

    The MAC signature found in the HTTP request 'QRSg3kyjWheBxIZwu85EyV2Anl5ly7okITjSri0zwzI=' is not the same as any computed signature. Server used following string to sign: 'PUT

     

     

    59

     

     

     

     

     

     

     

     

    x-ms-date:Mon, 23 Oct 2017 18:12:19 GMT

    x-ms-version:2016-05-31

    /XXXXX/queue-unit-tests-351957909/messages/adf08eff-f7bd-41a3-98c2-0fef6b8fc152

    popreceipt:AgAAAAMAAAAAAAAAKZS dypM0wE=

    visibilitytimeout:2'.

    Note that in the ‘popreceipt’ parameter near the bottom, there is a space where the “+” should be. Our assessment is that something is going wrong on the Azure server side that is misinterpreting the signed string we are sending when the pop receipt contains a “+” character. We think this is a parsing error on the server side, but if it's something we're doing wrong on our end, we need help figuring out what that is.


    • Edited by DNatt Monday, October 23, 2017 7:17 PM
    Monday, October 23, 2017 7:04 PM

Answers

  • The “+” character in a URL means ‘space’.  If you’re trying to send something with a “+” (or other special character) in a GET request then you need to URI Encode it (may also apply to POST requests). The ‘+’ character is encoded as %2B; however, you could use one of the standard URI encoding APIs as that will handle all special characters, not just ‘+’.

    Also, as per https://tools.ietf.org/html/rfc2396#section-2, query name and values must be URL encoded if they contain any reserved characters, like ‘+’. The safe thing to do is to always encode before sending.

    Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    • Proposed as answer by Md Shihab Wednesday, October 25, 2017 3:43 AM
    • Marked as answer by DNatt Thursday, October 26, 2017 7:21 PM
    Wednesday, October 25, 2017 3:43 AM

All replies

  • The “+” character in a URL means ‘space’.  If you’re trying to send something with a “+” (or other special character) in a GET request then you need to URI Encode it (may also apply to POST requests). The ‘+’ character is encoded as %2B; however, you could use one of the standard URI encoding APIs as that will handle all special characters, not just ‘+’.

    Also, as per https://tools.ietf.org/html/rfc2396#section-2, query name and values must be URL encoded if they contain any reserved characters, like ‘+’. The safe thing to do is to always encode before sending.

    Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    • Proposed as answer by Md Shihab Wednesday, October 25, 2017 3:43 AM
    • Marked as answer by DNatt Thursday, October 26, 2017 7:21 PM
    Wednesday, October 25, 2017 3:43 AM
  • Thanks, Md Shihab. Your information helped us to figure out the issue. We are now encoding the request parameters that are added to the URL (but not encoding the strings used to generate the signature), and that solved our issue.

    One additional point, though...we find it strange that the pop receipts returned by Get Message not only sometimes have "+" characters (which was the issue I originally reported), but they always have "=" characters. Based on what you said, I would have expected our requests to fail because we were not encoding the "=" character in our URLs. But any Delete Message or Update Message request we sent without a "+" worked just fine, even when the "=" character was not encoded. We are now encoding all characters (including "+", "=", and all the other "special" characters designated in the URI spec), but we're curious why an "=" that was not encoded didn't cause us any problems originally. Any insights on that?

    Wednesday, October 25, 2017 5:40 PM
  • Characters like ‘?’, ‘&’, and ‘=’ are considered reserved (along with characters like + and % which are used for encoding other characters). However, the reality is that most URI parsers are context sensitive. Thus, if given a URI that looks like this:

                    ?name=valuecontaining=&othername=othervalue

    Once the parser finds the first ‘=’, it’s scanning for an ‘&’ or an end of string, so the value containing an ‘=’ is parsed properly.  However, if the ‘=’ was used in the name, then it would be improperly parsed. ('+' is defined as an alternate for space, so it’s always converted/reserved no matter where it is). 

    So, the general principle is that the ‘=’ character is reserved and should be encoded just to be safe (even though it often doesn’t have to be in this case, a different parser might change the rules subtly and cause failures in the future).

    -----------------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Monday, October 30, 2017 4:13 AM