I can't get the StorageClient library to return rows with %6B in the partition key

Question

I've been experiencing a seemingly nonsensical problem running some queries and I've broken it down to this.  If add an item to an Azure table (AZT) and give it a partition key of %6B and an empty row key, when I use run a linq query that brings back the item by partition key and row key, it always brings back null.

I've tried running Fiddler to see if it wasn't been encoded properly, but that all looks OK.

I've tried running the same query in Cerebrata cloud storage studio and that works OK, so it doesn't appear to be an issue with AZT itself.

Does anybody have any ideas about how to get around this?

Answer 1

After some extra testing, if you try to set the partition key to be "%FF" you get a "One of the request inputs is not valid" error.  If I use "%GF" everything works fine.  If I add a row to the table with the partition key of 'k' (6B in ASCII), running the query with %6B will bring back that row.  I can see what's going on here, but it's certainly not the expected behavior.

Answer 2

I believe that using the  % symbol in either the PartitionKey or RowKey can cause problems.

The following is from the excellent Moving Applications to the Cloud book published by Microsoft Patterns and Practices Team:

Windows Azure places some restrictions on the characters that you can use in partition and row keys. Generally speaking, the restricted characters are ones that are meaningful in a URL. For more information, see http://msdn.microsoft.com/en-us/library/dd179338.aspx. In the aExpense application [exmple application], it's possible that these illegal characters could appear in the UserName used as the partition key value for the Expense table.

If there is an illegal character in your partition key, Windows Azure will return a Bad Request (400) message.

To avoid this problem, the aExpense application encodes the UserName value using a base64 encoding scheme before using the UserName value as a row key. Implementing base64 encoding and decoding is very easy.

The team at Adatum first tried to use the UrlEncode method because it would have produced a more human readable encoding, but this approach failed because it does not encode the percent sign (%) character.

According to the documentation, the percent sign character is not an illegal character in a key, but Adatum's  [example company] testing showed that entities with a percent sign character in the key could not be deleted.

Answer 3

This problem has actually come up because of Base64 encoding.  Because \ is a valid Base64 character, but not a valid in a partition key, I had to replace it with something.  I arbitrarily replaced it with % because this is not on the list of invalid partition key characters and it kinda looks the same.  Most of the time this was OK unless it was followed by 2 hex characters (which, wouldn't you know it, never showed up in testing).  My solution is to just not use % and instead use something that is URL safe.  What confuses me the most is that it is possible to write a query that brings back this information (thanks Cerebrata), so it must be something in the way the client library is building the query.

Answer 4

This problem has actually come up because of Base64 encoding.  Because \ is a valid Base64 character, but not a valid in a partition key, I had to replace it with something.

Well, scratch that idea.

Answer 5

Hello, this is not an expected behavior. I don't know how Cerebrata brings back the entity. But probably it is doing a batch query, where the keys do not need to be included in the URL. If you try to update/delete the entity, in which case the key has to present in URL, you won't be able to do it because of the '%'.

I'll forward this issue to the storage team.

---

Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.

Answer 6

Hi Yi-Lun,

We're basically doing a URL Encode of the query string. Here is a part of our code which constructs the $filter element of the query string:

string.Format("$filter={0}&", HttpUtility.UrlEncode(QueryText));

Where QueryText = PartitionKey eq '%6B'

I also created a temporary table and used "%6B" as the partition key. When I looked the request trace, we're encoding "%" sign as well. So the query: "PartitionKey eq '%6B'" goes across the wire as

$filter=PartitionKey+eq+'%256B'

Hope this helps

Thanks

Gaurav Mantri

Cerebrata Software

http://www.cerebrata.com

Answer 7

Thanks for the information. I've verified query works fine. But update/delete doesn't work even if you do a URL encoding. I'll forward these information to the storage team.

---

Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.

Answer 8

Same here. We're also not able to update/delete through Cloud Storage Studio because we're not doing any URLEncoding there. In past we tried by URL encoding the PartitionKey and RowKey values for update/deletes and it did not work either.

Thanks

Gaurav

Answer 9

Thanks for your help on this everyone.  Hopefully this will stop others from wasting time trying to figure out why their updates aren't working.