locked
Table storage REST api merge an entity. Get 400 exception RRS feed

  • Question

  • I am using REST API to merge an entity into a table storage. 

    I always got  a exception 400 bad request. I checked the request body and header, they all look good. I have If-Match header set to "*". The row existing in the table. I cannot figure out the reason. Can any body give me any advice on how to debug this kind of exception? How I can get more information rather then the 400 bad request?

     

    Thanks

    Friday, May 27, 2011 5:41 AM

Answers

  • Are you misisng the following headers?

    DataServiceVersion: 1.0;NetFx
    MaxDataServiceVersion: 1.0;NetFx

    Thanks,

    Jai

    • Marked as answer by KevinGZ Saturday, May 28, 2011 3:15 AM
    Saturday, May 28, 2011 2:15 AM

All replies

  • This is probably the first place I would look.

    Table Service Error Codes on MSDN

    It should at least narrow you down to a couple of possibilities.

    A good thing to note is alot of the REST based api is Case Sensitive, one thing I would do is check to make sure you don't have a case mismatch between the table name in storage and the table name you are sending with your request.

    Beyond that, it might be worth reviewing the Windows Azure Storage REST API one more time after a rather large cup of coffee.

    You could also mock the expected results up with Fiddler, this will allow you to see how things are going over the wire. Then you can make your API Call with your code to see if the results are the same.


    Cory Fowler Windows Azure MVP http://blog.syntaxc4.net
    Friday, May 27, 2011 6:00 AM
    Answerer
  • I find Fiddler invaluable in this type of situation. I also find something like Cloud Storage Studio really useful since it is surprising how often I had goofed with the casing of names which, as Cory points out, is crucial.
    Friday, May 27, 2011 6:38 AM
    Answerer
  • I am assuming this is against the real cloud service. I agree with Cory and Neil - fiddler or wireshark is your friend here.

    Some common reasons are:

    1> date time property should be initialized - the minimum value can be got using DateTime.FromFileTimeUtc(0). This value is different from .NET's minimum value

    2> Unsupported data type being sent. The above link also has the data types supported

    3> key values cannot be updated

    4> Each entity has a 1 MB size limit

    5> byte[] and string can be upto 64KB in size

    6> You can have upto 252 custom properties and property names should follow naming rules of c# identifiers

    Also, as Cory mentioned, table names are case insensitive but column names are case sensitive. If the case changes for property names, the update will succeed but will add a new property with the specified case.

     

    If it is none of the above, feel free to paste the entity xml sent over the wire and one of us can help spot it.

     

    Thanks,

    jai

    Friday, May 27, 2011 1:56 PM
  • Thank you every one !

    After quite a few large cup of coffee, I guess the problem may be the key name change (property name). The entity doesn't have a property, but when I update I am adding a new property, do you think this is the major issue?

    Friday, May 27, 2011 4:22 PM
  • This is the fiddler RAW data

    PUT http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473',RowKey='S_0000000000_0000000020') HTTP/1.1
    x-ms-date: Fri, 27 May 2011 16:54:28 GMT
    x-ms-version: 2009-09-19
    Content-Type: application/atom+xml
    DataServiceVersion: 1.0;NetFx
    MaxDataServiceVersion: 1.0;NetFx
    Accept-Charset: UTF-8
    Authorization: SharedKey eachcloud2:m/XcsXABb0VRHyyj9pm2yOvFfnB3V6Y6UJiz5BxCfuk=
    Host: eachcloud2.table.core.windows.net
    Content-Length: 1169
    
    <?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-27T16:54:23.3688074Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473',RowKey='S_0000000000_0000000020')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611319505275795473</d:PartitionKey>
    <d:RowKey>S_0000000000_0000000020</d:RowKey>
    <d:Timestamp m:type="Edm.DateTime">2011-05-27T16:54:23.3688074Z</d:Timestamp>
    <d:DHash>0</d:DHash>
    <d:EndPos>20</d:EndPos>
    <d:PHash>327879684</d:PHash>
    <d:PrevSeg></d:PrevSeg>
    <d:SelectCandRk></d:SelectCandRk>
    <d:SelectType>1</d:SelectType>
    <d:Src>Ok so I didn't sleep.</d:Src>
    <d:SrcWithTag>Ok so I didn't sleep.</d:SrcWithTag>
    <d:StartPos>0</d:StartPos>
    <d:XPath></d:XPath>
    <d:zh_CN>好了,所以我没有睡觉。</d:zh_CN>
     </m:properties> </content> </entry>

    This is the response I got from fiddler

     

    HTTP/1.1 400 Bad Request
    Transfer-Encoding: chunked
    Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
    x-ms-request-id: 367389cc-e292-4376-8ee7-4c63cb064a40
    x-ms-version: 2009-09-19
    Date: Fri, 27 May 2011 16:54:31 GMT
    
    14E
    <?xml version="1.0" encoding="utf-8" standalone="yes"?>
    <error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
     <code>InvalidInput</code>
     <message xml:lang="en-US">One of the request inputs is not valid.
    RequestId:367389cc-e292-4376-8ee7-4c63cb064a40
    Time:2011-05-27T16:54:31.4569965Z</message>
    </error>
    0
    

    It is pretty short, but I did not figure out which input value is invalid. 

    Any thoughts?

     

    Friday, May 27, 2011 5:03 PM
  • It does not look like you have the etag specified in your request. From your original post, If-Match header should be set to "*" right? 

    We will work on providing more helpful error messages in future but for now setting this header should help.

     

    Thanks,

    jai

    Friday, May 27, 2011 5:16 PM
  • See the C# source code 

           request = CreateRESTRequest("PUT", resource, requestBody, headers, "*");
    
    
    
    ...................
    
    
    
        public HttpWebRequest CreateRESTRequest(string method, string resource, string requestBody = null, SortedList<string, string> headers = null,
          string ifMatch = "", string md5 = "")
        {
          byte[] byteArray = null;
          DateTime now = DateTime.UtcNow;
          string uri = Endpoint + resource;
    
          HttpWebRequest request = HttpWebRequest.Create(uri) as HttpWebRequest;
          request.Method = method;
          request.ContentLength = 0;
          request.Headers.Add("x-ms-date", now.ToString("R", System.Globalization.CultureInfo.InvariantCulture));
          request.Headers.Add("x-ms-version", "2009-09-19");
    
          if (IsTableStorage)
          {
            request.ContentType = "application/atom+xml";
    
            request.Headers.Add("DataServiceVersion", "1.0;NetFx");
            request.Headers.Add("MaxDataServiceVersion", "1.0;NetFx");
          }
    
          if (headers != null)
          {
            foreach (KeyValuePair<string, string> header in headers)
            {
              request.Headers.Add(header.Key, header.Value);
            }
          }
    
          if (!String.IsNullOrEmpty(requestBody))
          {
            request.Headers.Add("Accept-Charset", "UTF-8");
    
            byteArray = Encoding.UTF8.GetBytes(requestBody);
            request.ContentLength = byteArray.Length;
          }
    
          request.Headers.Add("Authorization", AuthorizationHeader(method, now, request, ifMatch, md5));
    
          if (!String.IsNullOrEmpty(requestBody))
          {
            request.GetRequestStream().Write(byteArray, 0, byteArray.Length);
          }
    
          return request;
        }
    
    ................................
    
    
        // Generate an authorization header.
    
        public string AuthorizationHeader(string method, DateTime now, HttpWebRequest request, string ifMatch = "", string md5 = "")
        {
          string MessageSignature;
    
          if (IsTableStorage)
          {
            MessageSignature = String.Format("{0}\n\n{1}\n{2}\n{3}",
              method,
              "application/atom+xml",
              now.ToString("R", System.Globalization.CultureInfo.InvariantCulture),
              GetCanonicalizedResource(request.RequestUri, StorageAccount)
              );
          }
          else
          {
            MessageSignature = String.Format("{0}\n\n\n{1}\n{5}\n\n\n\n{2}\n\n\n\n{3}{4}",
              method,
              (method == "GET" || method == "HEAD") ? String.Empty : request.ContentLength.ToString(),
              ifMatch,
              GetCanonicalizedHeaders(request),
              GetCanonicalizedResource(request.RequestUri, StorageAccount),
              md5
              );
          }
          byte[] SignatureBytes = System.Text.Encoding.UTF8.GetBytes(MessageSignature);
          System.Security.Cryptography.HMACSHA256 SHA256 = new System.Security.Cryptography.HMACSHA256(Convert.FromBase64String(StorageKey));
          String AuthorizationHeader = "SharedKey " + StorageAccount + ":" + Convert.ToBase64String(SHA256.ComputeHash(SignatureBytes));
          return AuthorizationHeader;
        }
    
    
    
    

    From those code above, the IfMatch = "*" is passed through to MessageSingaure, but the problem is here: IsTableStorage is true, the IfMatch header is not used. I think this is the real problem.

    The code is got from this forum somewhere. I think it is out of date. I do not know how to fix this by adding the IfMatch to header for table storage. What is the correct format of MessageSignature for table storage?  Thanks

     

     

    Friday, May 27, 2011 5:27 PM
  • Is there a reason for not using the official Storage client library which takes care of this?

    You would need to add the header in the table storage too. but the problem is with etag format. You would need to conform to the OData format. The storage client library that comes with SDK handles these nuances for you and is built over WCF data Services .NET (aka OData implementation).

     

    Thanks,

    jai

     

     

    Friday, May 27, 2011 5:49 PM
  • The reason is just lazy. I copied the code from someone else in the forum and did not even carefully read it until just now. I may have to change to use 

    storageAccount.Credentials.SignRequestLite(hwr);....

     

    to create the header and sign.

    Do you think it is a better idea? I have not tried that before, is this what you mean?

     

     

    Usually lazy is the biggest enemy of myself.

    Thanks

    Friday, May 27, 2011 6:07 PM
  • I meant you could create the context and use AttachTo and UpdateObject. Here is a sample code which takes care of auth and dispatching the request.
    CloudStorageAccount account = CloudStorageAccount.Parse(connectionString);
    CloudTableClient tableClient = account.CreateCloudTableClient();
    TableServiceContext context = tableClient.GetDataServiceContext();
          
    context.AttachTo(tableName, myEntityToUpdate, "*");
    context.UpdateObject(myEntityToUpdate);
    context.SaveChanges();
    
    connection string is of the form "DefaultEndpointsProtocol=https;AccountName=myaccountname;AccountKey=mykey"
    Thanks,
    Jai
    Friday, May 27, 2011 7:32 PM
  • I see what you meant. Well that's something I cannot do. My table storage is schemaless. Every row has different properties, like other noSQL data. 

    I do not even have a class for this entity, so the Cloud storage lib won't work in my case. That's why I have to use RESTful API. I do not realy like REST but it work.

    Friday, May 27, 2011 8:19 PM
  • What is the correct way to create a MessageSignature for Table entity merge?
    Friday, May 27, 2011 9:09 PM
  • I changed the way how to sign the Auth. Now see the fiddler output

    MERGE http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683',RowKey='S_0000000000_0000000037') HTTP/1.1
    x-ms-version: 2009-09-19
    Content-Type: application/atom+xml
    If-Match: *
    Accept-Charset: UTF-8
    x-ms-date: Fri, 27 May 2011 23:25:19 GMT
    Authorization: SharedKey eachcloud2:gCs375e68bZQMFaw6nf7uwxCumRfFJ1T7kV3jGOZ2WQ=
    Host: eachcloud2.table.core.windows.net
    Content-Length: 1194
    
    <?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-27T23:25:19.6624306Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683',RowKey='S_0000000000_0000000037')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611229341039076683</d:PartitionKey>
    <d:RowKey>S_0000000000_0000000037</d:RowKey>
    <d:Timestamp m:type="Edm.DateTime">2011-05-27T23:25:19.6624306Z</d:Timestamp>
    <d:DHash>0</d:DHash>
    <d:EndPos>37</d:EndPos>
    <d:PHash>419539206</d:PHash>
    <d:PrevSeg></d:PrevSeg>
    <d:SelectCandRk></d:SelectCandRk>
    <d:SelectType>1</d:SelectType>
    <d:Src>I couldn't type that fast enough #fail</d:Src>
    <d:SrcWithTag>I couldn't type that fast enough #fail</d:SrcWithTag>
    <d:StartPos>0</d:StartPos>
    <d:XPath></d:XPath>
    <d:zh_CN>我没法打字那么快</d:zh_CN>
     </m:properties> </content> </entry>
     
     
     ======================================
     HTTP/1.1 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
    Content-Length: 437
    Content-Type: application/xml
    Server: Microsoft-HTTPAPI/2.0
    x-ms-request-id: c66928a5-8c97-4a2b-9a57-0c78058fceec
    Date: Fri, 27 May 2011 23:25:23 GMT
    
    <?xml version="1.0" encoding="utf-8" standalone="yes"?>
    <error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
     <code>AuthenticationFailed</code>
     <message xml:lang="en-US">Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
    RequestId:c66928a5-8c97-4a2b-9a57-0c78058fceec
    Time:2011-05-27T23:25:23.3409831Z</message>
    </error>

    From the error message, the authenticate has some problem.

    I changed the code of authenticate, here is the code I am using now

       ((StorageCredentials)GeneralUtil.StorageAccount.Credentials).SignRequest(request);

    Is there anything wrong?

    Friday, May 27, 2011 11:41 PM
  • I changed to SignLite, the error changed to "One of the Http headers is not in the correct format".

     

    See the fiddler output

    MERGE http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680',RowKey='S_0000000000_0000000014') HTTP/1.1
    x-ms-version: 2009-09-19
    Content-Type: application/atom+xml
    If-Match: *
    Accept-Charset: UTF-8
    x-ms-date: Sat, 28 May 2011 00:37:55 GMT
    Authorization: SharedKeyLite eachcloud2:k5VezSXp1GmlPlW6X/coodH75+iYqzRmezd/KiBrAEI=
    Host: eachcloud2.table.core.windows.net
    Content-Length: 857
    
    <?xml version="1.0" encoding="utf-8" standalone="yes"?><entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices"    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"    xmlns="http://www.w3.org/2005/Atom">  <title />  <updated>2011-05-28T00:37:49.6332347Z</updated>  <author>  <name/>  </author>  <id>http://eachcloud2.table.core.windows.net/DocSegEntities(PartitionKey='a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680',RowKey='S_0000000000_0000000014')</id>  <content type="application/xml"> <m:properties><d:PartitionKey>a3a26ceb-f30e-4663-9856-54b906367adc_5611229332605114680</d:PartitionKey>
    <d:RowKey>S_0000000000_0000000014</d:RowKey>
    <d:Timestamp m:type="Edm.DateTime">2011-05-28T00:37:49.6332347Z</d:Timestamp>
    <d:UpdateKey>UpdateValue</d:UpdateKey>
     </m:properties> </content> </entry>

    resposne

    HTTP/1.1 400 Bad Request
    Transfer-Encoding: chunked
    Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
    x-ms-request-id: 653d86ce-b6f5-4e84-b7b1-133228263a0f
    x-ms-version: 2009-09-19
    Date: Sat, 28 May 2011 00:37:59 GMT
    
    170
    <?xml version="1.0" encoding="utf-8" standalone="yes"?>
    <error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
     <code>InvalidHeaderValue</code>
     <message xml:lang="en-US">The value for one of the HTTP headers is not in the correct format.
    RequestId:653d86ce-b6f5-4e84-b7b1-133228263a0f
    Time:2011-05-28T00:37:59.6329637Z</message>
    </error>
    0
    
    There are only a few headers, which one is in wrong format?

    Saturday, May 28, 2011 12:42 AM
  • Are you misisng the following headers?

    DataServiceVersion: 1.0;NetFx
    MaxDataServiceVersion: 1.0;NetFx

    Thanks,

    Jai

    • Marked as answer by KevinGZ Saturday, May 28, 2011 3:15 AM
    Saturday, May 28, 2011 2:15 AM
  • Wow, it finally work this time. Thank you so much.

     

    Where can I get the detailed information about DataServiceVersion: 1.0;NetFx... ? Programming using REST is like a bingo game :(

    Saturday, May 28, 2011 3:15 AM
  • I know this posting is very old now, but I will add a piece of information that I believe will result helpful to other people having the same problem.

    • The values for booleans have to be in lower case.  THE FOLLOWING WILL CAUSE A 400 ERROR: <d:myProperty m:type="Edm.Boolean">False</d:myProperty> because False has a capital F.  When replaced by <d:myProperty m:type="Edm.Boolean>false</d:myProperty> the response will be accepted and pass.  It is very easy to fall into this error if you use myBoolean.ToString().   It will capitalize the first letter.   Instead you need to do something like  (myBoolean) ? "true" : "false"   when you create the contents of the request.

    Saturday, April 7, 2012 12:34 AM