none
DateTimeOffset handling... very serious bug RRS feed

  • Question

  • I am storing objects in a collection which contain several properties which are DateTimeOffsets. One of my documents has a value which has been serialized to:

    2016-12-15T15:43:30+00:00

    I am seeing this by looking at the document in the azure portal. However, when I retrieve the document with the following code...

                Uri documentUri = UriFactory.CreateDocumentUri("admn", "draw", $"{drawRefId}");
    
                var response = await client.ReadDocumentAsync(documentUri);
    
                if (response.StatusCode == System.Net.HttpStatusCode.OK)
                {
                    return JsonConvert.DeserializeObject<Draw>(response.Resource.ToString());
                }

    if i look at response.Resource.ToString() the value has been changed to:

    2016-12-15T07:43:30-08:00

    Now this would seem to indicate that DocumentDb has recognized the DateTimeOffset (which is awesome), but that it is deserializing the value to local time which is VERY VERY VERY bad.

    I only just noticed this today which leads me to believe that something has changed, otherwise I would have seen it earlier. Can someone else confirm this? If so, the DocumentDB team has a VERY MAJOR BUG!!!

    Thursday, December 15, 2016 4:06 PM

Answers

  • Hi Aravind,

    Thanks for the response. It looks like my best option at this point is to set JsonConvert.DefaultSettings. I can't provide custom settings to the DeserializeObject call because the only DocumentClient call that lets you access the underlying JSON is ReadDocumentAsync which returns a ResourceResponse (which provides access to the ResponseStream.)

    We use DateTimeOffset everywhere in our code, so changing the DefaultSettings should not be a problem. Some shops may not be so lucky. Thanks again.

    Friday, April 14, 2017 7:35 PM

All replies

  • Hi Jack,

          There is no date format in JSON, there's only strings as serialized date format in Json.

         When you deserialize to DateTime, you can then decide to deserialize to local time or UTC time. For example,  can you try setting JsonSerializerSettings in your  JsonConvert.DeserializeObject?  Then you should get UTC time back. 

    new JsonSerializerSettings
        {
          DateTimeZoneHandling = DateTimeZoneHandling.Utc
        }

       Thanks!

       Ming

    Thursday, December 15, 2016 5:26 PM
  • I appreciate your response, but let me emphasize something rather emphatically...

    The date is being changed BEFORE any JSON deserialization. If I look at the document in the Azure portal, the value is stored correctly as:

    2016-12-15T15:43:30+00:00

    On the other hand, if I look at the value in the debugger of response.Resource.ToString, it is:

    2016-12-15T07:43:30-08:00

    And to further clarify, if I change the value in the portal to:

    2016-12-15T15:43:30-05:00

    the value contained in response.Resource.ToString() is:

    2016-12-15T12:43:30-08:00

    This is a MAJOR BUG. There is NO WAY that the DocumentDb SDK should be performing this conversion.

    Thursday, December 15, 2016 5:38 PM
  • Nearly four months, and this issue has still not been resolved.
    Thursday, April 6, 2017 5:50 PM
  • As discussed under my GitHub post, this is indeed still an issue. It looks like DocumentDb uses JsonConvert (JSON.NET) under the hood to deserialize documents, but Scott Allens article about using JSONConvert defaults to control how dates are deserialized is not working.

    http://odetocode.com/blogs/scott/archive/2015/04/28/serialization-options-with-azure-documentdb.aspx

    I can only assume they have explicit deserialisation options being provided to JSONConvert which is assuming UTC, stripping off the Offset in the string data. 

    Come on guys, offsets are there for a reason (ISO 8601 anyone).

    Wake up and respond DocumentDb team!


    VDI for the win!

    Monday, April 10, 2017 7:42 PM
  • @jackhanbond

    I'm from the DocumentDB team. Sincere apologies for the delay in responding and any gaps in communication. Let me explain what's happening here, our plans for addressing this, and how you can workaround this issue.

    DocumentDB uses JSON.net for serialization and deserialization. Normally, JSON.net preserves the TimeZone offset when you do this across the conversion. However, when you are working with dynamic objects, JSON.net decides on whether a string is DateTime, DateTimeOffset, or string based on your serializer settings. By default, when a string parses to a DateTime or a DateTimeOffset, it chooses DateTime - which means that the offset information is dropped. See snippet below with just JSON.net:

    public class Event
    {
        [JsonProperty("id")]
        public String Id { get; set; }
    
        [JsonProperty("ts")]
        public DateTimeOffset TS { get; set; }
    }
    
    Event myEvent = new Event { Id = "Zero", TS = new DateTimeOffset(2017, 1, 1, 0, 0, 0, TimeSpan.Zero) };
    string json = JsonConvert.SerializeObject(myEvent);
    
    
    // You have the value of TS in local time
    Event newEvent2 = JsonConvert.DeserializeObject<Event>(json);
    
    // You have the correct value of TS with the UTC offset
    Event newEvent = JsonConvert.DeserializeObject<Event>(json, new JsonSerializerSettings { DateParseHandling = DateParseHandling.DateTimeOffset });

    Since DocumentDB works with any object type, internally all Documents are parsed as dynamic with the default Json serializer settings, which means that it is treated as DateTime. Note this is only on deserialization, the data as stored in DocumentDB will have the original timestamp with Offset. This is not mutated or changed by DocumentDB (you can verify via Fiddler). The change you see is only on read.

    We are actively working on a change that will allow users to plug in custom Json serializer settings to the DocumentDB client. Since this is a fairly large change since DocumentDB uses JSON conversion in a number of places including metadata for collections and databases, unfortunately we cannot provide a stop-gap hotfix for DateTimeOffset parsing.

    However, there are a couple of workarounds we had suggested to your questions - 1. storing the DateTimeOffset as a string and 2. implementing a custom JsonConverter that ensures the offsets are deserialized as you want it.

    Thanks again for using DocumentDB. And once again, sorry about the delay. Please email us at askdocdb@microsoft.com, and we can discuss in more detail over a call and help you with a solution.


    Thursday, April 13, 2017 9:11 PM
  • Hi Aravind,

    Thanks for the response. It looks like my best option at this point is to set JsonConvert.DefaultSettings. I can't provide custom settings to the DeserializeObject call because the only DocumentClient call that lets you access the underlying JSON is ReadDocumentAsync which returns a ResourceResponse (which provides access to the ResponseStream.)

    We use DateTimeOffset everywhere in our code, so changing the DefaultSettings should not be a problem. Some shops may not be so lucky. Thanks again.

    Friday, April 14, 2017 7:35 PM
  • Did changing the DefaultSettings end up working for you? I tried this and it doesn't have any effect on the DateTimeOffsets getting deserialized - same problem. I also read a couple other places this solution didn't work, so I was just curious if you had success with it.
    Saturday, June 17, 2017 1:11 PM
  • I just ran in to this very issue, and the changes suggested by Aravind worked for me.  Reading up a little on the documentation for JsonSerializerSettings, I would actually prefer the following code, which I find to be just a little more clear on what we are trying to achieve:

    var serializerSettings = new JsonSerializerSettings {
        DateTimeZoneHandling = DateTimeZoneHandling.RoundtripKind,
        DateParseHandling = DateParseHandling.DateTimeOffset
    };
    
    documentClient = new DocumentClient(uri, key, serializerSettings);
    

    And this is what we ended up with, and it works a charm.

    Wednesday, October 4, 2017 11:16 AM