Table storage REST api merge an entity. Get 400 exception

Friday, May 27, 2011 6:00 AM

Moderator

This is probably the first place I would look.

It should at least narrow you down to a couple of possibilities.

A good thing to note is alot of the REST based api is Case Sensitive, one thing I would do is check to make sure you don't have a case mismatch between the table name in storage and the table name you are sending with your request.

Beyond that, it might be worth reviewing the Windows Azure Storage REST API one more time after a rather large cup of coffee.

You could also mock the expected results up with Fiddler, this will allow you to see how things are going over the wire. Then you can make your API Call with your code to see if the results are the same.

Cory Fowler Windows Azure MVP http://blog.syntaxc4.net

Friday, May 27, 2011 6:38 AM

Answerer

0

I find Fiddler invaluable in this type of situation. I also find something like Cloud Storage Studio really useful since it is surprising how often I had goofed with the casing of names which, as Cory points out, is crucial.

Friday, May 27, 2011 1:56 PM

0

I am assuming this is against the real cloud service. I agree with Cory and Neil - fiddler or wireshark is your friend here.

Some common reasons are:

1> date time property should be initialized - the minimum value can be got using DateTime.FromFileTimeUtc(0). This value is different from .NET's minimum value

2> Unsupported data type being sent. The above link also has the data types supported

3> key values cannot be updated

4> Each entity has a 1 MB size limit

5> byte[] and string can be upto 64KB in size

6> You can have upto 252 custom properties and property names should follow naming rules of c# identifiers

Also, as Cory mentioned, table names are case insensitive but column names are case sensitive. If the case changes for property names, the update will succeed but will add a new property with the specified case.

If it is none of the above, feel free to paste the entity xml sent over the wire and one of us can help spot it.

Thanks,

jai

Friday, May 27, 2011 4:22 PM

0

Thank you every one !

After quite a few large cup of coffee, I guess the problem may be the key name change (property name). The entity doesn't have a property, but when I update I am adding a new property, do you think this is the major issue?

Friday, May 27, 2011 5:03 PM

0

This is the fiddler RAW data

PUT http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473',RowKey='S_0000000000_0000000020') HTTP/1.1
x-ms-date: Fri, 27 May 2011 16:54:28 GMT
x-ms-version: 2009-09-19
Content-Type: application/atom+xml
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Accept-Charset: UTF-8
Authorization: SharedKey eachcloud2:m/XcsXABb0VRHyyj9pm2yOvFfnB3V6Y6UJiz5BxCfuk=
Host: eachcloud2.table.core.windows.net
Content-Length: 1169

<?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-27T16:54:23.3688074Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473',RowKey='S_0000000000_0000000020')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473</d:PartitionKey>
<d:RowKey>S_0000000000_0000000020</d:RowKey>
<d:Timestamp m:type="Edm.DateTime">2011-05-27T16:54:23.3688074Z</d:Timestamp>
<d:DHash>0</d:DHash>
<d:EndPos>20</d:EndPos>
<d:PHash>327879684</d:PHash>
<d:PrevSeg></d:PrevSeg>
<d:SelectCandRk></d:SelectCandRk>
<d:SelectType>1</d:SelectType>
<d:Src>Ok so I didn't sleep.</d:Src>
<d:SrcWithTag>Ok so I didn't sleep.</d:SrcWithTag>
<d:StartPos>0</d:StartPos>
<d:XPath></d:XPath>
<d:zh_CN>好了，所以我没有睡觉。</d:zh_CN>
 </m:properties> </content> </entry>

This is the response I got from fiddler

HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 367389cc-e292-4376-8ee7-4c63cb064a40
x-ms-version: 2009-09-19
Date: Fri, 27 May 2011 16:54:31 GMT

14E
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
 <code>InvalidInput</code>
 <message xml:lang="en-US">One of the request inputs is not valid.
RequestId:367389cc-e292-4376-8ee7-4c63cb064a40
Time:2011-05-27T16:54:31.4569965Z</message>
</error>
0

It is pretty short, but I did not figure out which input value is invalid.

Any thoughts?

Friday, May 27, 2011 5:16 PM

0

It does not look like you have the etag specified in your request. From your original post, If-Match header should be set to "*" right?

We will work on providing more helpful error messages in future but for now setting this header should help.

Thanks,

jai

Friday, May 27, 2011 5:27 PM

0

See the C# source code

       request = CreateRESTRequest("PUT", resource, requestBody, headers, "*");



...................



    public HttpWebRequest CreateRESTRequest(string method, string resource, string requestBody = null, SortedList<string, string> headers = null,
      string ifMatch = "", string md5 = "")
    {
      byte[] byteArray = null;
      DateTime now = DateTime.UtcNow;
      string uri = Endpoint + resource;

      HttpWebRequest request = HttpWebRequest.Create(uri) as HttpWebRequest;
      request.Method = method;
      request.ContentLength = 0;
      request.Headers.Add("x-ms-date", now.ToString("R", System.Globalization.CultureInfo.InvariantCulture));
      request.Headers.Add("x-ms-version", "2009-09-19");

      if (IsTableStorage)
      {
        request.ContentType = "application/atom+xml";

        request.Headers.Add("DataServiceVersion", "1.0;NetFx");
        request.Headers.Add("MaxDataServiceVersion", "1.0;NetFx");
      }

      if (headers != null)
      {
        foreach (KeyValuePair<string, string> header in headers)
        {
          request.Headers.Add(header.Key, header.Value);
        }
      }

      if (!String.IsNullOrEmpty(requestBody))
      {
        request.Headers.Add("Accept-Charset", "UTF-8");

        byteArray = Encoding.UTF8.GetBytes(requestBody);
        request.ContentLength = byteArray.Length;
      }

      request.Headers.Add("Authorization", AuthorizationHeader(method, now, request, ifMatch, md5));

      if (!String.IsNullOrEmpty(requestBody))
      {
        request.GetRequestStream().Write(byteArray, 0, byteArray.Length);
      }

      return request;
    }

................................


    // Generate an authorization header.

    public string AuthorizationHeader(string method, DateTime now, HttpWebRequest request, string ifMatch = "", string md5 = "")
    {
      string MessageSignature;

      if (IsTableStorage)
      {
        MessageSignature = String.Format("{0}\n\n{1}\n{2}\n{3}",
          method,
          "application/atom+xml",
          now.ToString("R", System.Globalization.CultureInfo.InvariantCulture),
          GetCanonicalizedResource(request.RequestUri, StorageAccount)
          );
      }
      else
      {
        MessageSignature = String.Format("{0}\n\n\n{1}\n{5}\n\n\n\n{2}\n\n\n\n{3}{4}",
          method,
          (method == "GET" || method == "HEAD") ? String.Empty : request.ContentLength.ToString(),
          ifMatch,
          GetCanonicalizedHeaders(request),
          GetCanonicalizedResource(request.RequestUri, StorageAccount),
          md5
          );
      }
      byte[] SignatureBytes = System.Text.Encoding.UTF8.GetBytes(MessageSignature);
      System.Security.Cryptography.HMACSHA256 SHA256 = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String(StorageKey));
      String AuthorizationHeader = "SharedKey " + StorageAccount + ":" + Convert.ToBase64String(SHA256.ComputeHash(SignatureBytes));
      return AuthorizationHeader;
    }

From those code above, the IfMatch = "*" is passed through to MessageSingaure, but the problem is here: IsTableStorage is true, the IfMatch header is not used. I think this is the real problem.

The code is got from this forum somewhere. I think it is out of date. I do not know how to fix this by adding the IfMatch to header for table storage. What is the correct format of MessageSignature for table storage? Thanks

Friday, May 27, 2011 5:49 PM

0

Is there a reason for not using the official Storage client library which takes care of this?

You would need to add the header in the table storage too. but the problem is with etag format. You would need to conform to the OData format. The storage client library that comes with SDK handles these nuances for you and is built over WCF data Services .NET (aka OData implementation).

Thanks,

jai

Friday, May 27, 2011 6:07 PM

0

The reason is just lazy. I copied the code from someone else in the forum and did not even carefully read it until just now. I may have to change to use

storageAccount.Credentials.SignRequestLite(hwr);....

to create the header and sign.

Do you think it is a better idea? I have not tried that before, is this what you mean?

Usually lazy is the biggest enemy of myself.

Thanks

Friday, May 27, 2011 7:32 PM

0

I meant you could create the context and use AttachTo and UpdateObject. Here is a sample code which takes care of auth and dispatching the request.

CloudStorageAccount account = CloudStorageAccount.Parse(connectionString);
CloudTableClient tableClient = account.CreateCloudTableClient();
TableServiceContext context = tableClient.GetDataServiceContext();
      
context.AttachTo(tableName, myEntityToUpdate, "*");
context.UpdateObject(myEntityToUpdate);
context.SaveChanges();

connection string is of the form "DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=mykey"

Thanks,

Jai

Friday, May 27, 2011 8:19 PM

0

I see what you meant. Well that's something I cannot do. My table storage is schemaless. Every row has different properties, like other noSQL data.

I do not even have a class for this entity, so the Cloud storage lib won't work in my case. That's why I have to use RESTful API. I do not realy like REST but it work.

Friday, May 27, 2011 9:09 PM

0

What is the correct way to create a MessageSignature for Table entity merge?

Friday, May 27, 2011 11:41 PM

0

I changed the way how to sign the Auth. Now see the fiddler output

MERGE http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683',RowKey='S_0000000000_0000000037') HTTP/1.1
x-ms-version: 2009-09-19
Content-Type: application/atom+xml
If-Match: *
Accept-Charset: UTF-8
x-ms-date: Fri, 27 May 2011 23:25:19 GMT
Authorization: SharedKey eachcloud2:gCs375e68bZQMFaw6nf7uwxCumRfFJ1T7kV3jGOZ2WQ=
Host: eachcloud2.table.core.windows.net
Content-Length: 1194

<?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-27T23:25:19.6624306Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683',RowKey='S_0000000000_0000000037')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683</d:PartitionKey>
<d:RowKey>S_0000000000_0000000037</d:RowKey>
<d:Timestamp m:type="Edm.DateTime">2011-05-27T23:25:19.6624306Z</d:Timestamp>
<d:DHash>0</d:DHash>
<d:EndPos>37</d:EndPos>
<d:PHash>419539206</d:PHash>
<d:PrevSeg></d:PrevSeg>
<d:SelectCandRk></d:SelectCandRk>
<d:SelectType>1</d:SelectType>
<d:Src>I couldn't type that fast enough #fail</d:Src>
<d:SrcWithTag>I couldn't type that fast enough #fail</d:SrcWithTag>
<d:StartPos>0</d:StartPos>
<d:XPath></d:XPath>
<d:zh_CN>我没法打字那么快</d:zh_CN>
 </m:properties> </content> </entry>
 
 
 ======================================
 HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Content-Length: 437
Content-Type: application/xml
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: c66928a5-8c97-4a2b-9a57-0c78058fceec
Date: Fri, 27 May 2011 23:25:23 GMT

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
 <code>AuthenticationFailed</code>
 <message xml:lang="en-US">Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:c66928a5-8c97-4a2b-9a57-0c78058fceec
Time:2011-05-27T23:25:23.3409831Z</message>
</error>

From the error message, the authenticate has some problem.

I changed the code of authenticate, here is the code I am using now

   ((StorageCredentials)GeneralUtil.StorageAccount.Credentials).SignRequest(request);

Is there anything wrong?

Saturday, May 28, 2011 12:42 AM

0

I changed to SignLite, the error changed to "One of the Http headers is not in the correct format".

See the fiddler output

MERGE http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680',RowKey='S_0000000000_0000000014') HTTP/1.1
x-ms-version: 2009-09-19
Content-Type: application/atom+xml
If-Match: *
Accept-Charset: UTF-8
x-ms-date: Sat, 28 May 2011 00:37:55 GMT
Authorization: SharedKeyLite eachcloud2:k5VezSXp1GmlPlW6X/coodH75+iYqzRmezd/KiBrAEI=
Host: eachcloud2.table.core.windows.net
Content-Length: 857

<?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-28T00:37:49.6332347Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680',RowKey='S_0000000000_0000000014')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680</d:PartitionKey>
<d:RowKey>S_0000000000_0000000014</d:RowKey>
<d:Timestamp m:type="Edm.DateTime">2011-05-28T00:37:49.6332347Z</d:Timestamp>
<d:UpdateKey>UpdateValue</d:UpdateKey>
 </m:properties> </content> </entry>

resposne

HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 653d86ce-b6f5-4e84-b7b1-133228263a0f
x-ms-version: 2009-09-19
Date: Sat, 28 May 2011 00:37:59 GMT

170
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
 <code>InvalidHeaderValue</code>
 <message xml:lang="en-US">The value for one of the HTTP headers is not in the correct format.
RequestId:653d86ce-b6f5-4e84-b7b1-133228263a0f
Time:2011-05-28T00:37:59.6329637Z</message>
</error>
0

There are only a few headers, which one is in wrong format?

Saturday, May 28, 2011 2:15 AM

0

Are you misisng the following headers?

DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx

Thanks,

Jai

Marked As Answer by KevinGZ Saturday, May 28, 2011 3:15 AM

Saturday, May 28, 2011 3:15 AM

0

Wow, it finally work this time. Thank you so much.

Where can I get the detailed information about DataServiceVersion: 1.0;NetFx... ? Programming using REST is like a bingo game :(

Saturday, April 07, 2012 12:34 AM

0

I know this posting is very old now, but I will add a piece of information that I believe will result helpful to other people having the same problem.

The values for booleans have to be in lower case. THE FOLLOWING WILL CAUSE A 400 ERROR: <d:myProperty m:type="Edm.Boolean">False</d:myProperty> because False has a capital F. When replaced by <d:myProperty m:type="Edm.Boolean>false</d:myProperty> the response will be accepted and pass. It is very easy to fall into this error if you use myBoolean.ToString(). It will capitalize the first letter. Instead you need to do something like (myBoolean) ? "true" : "false" when you create the contents of the request.

Table storage REST api merge an entity. Get 400 exception

All Replies