none
Can we maintain multiple versions of a service fabric stateful service dictionary collection when schema changes RRS feed

  • Question

  • Can we maintain multiple versions of a service fabric stateful service dictionary collection when schema changes
    Monday, April 15, 2019 1:48 PM

All replies

  • I am checking with our Service Fabric Product team on this and will update you once I hear back. 
    Monday, April 15, 2019 5:57 PM
    Moderator
  • I don’t know quite what you mean by “multiple versions”. 

    That being said, if you have var d1 =  Dictionary<T1,T2> and then in an upgrade add var d2 = Dictionary <T1,T2’>, that will work. They are separate collections with separate names and types. SF doesn’t know that T2’ is a schema change from the data in T2 in d1.

    Monday, April 15, 2019 8:28 PM
    Moderator
  • Reliable Collections serialize your objects using .NET’s DataContractSerializer. The serialized objects are persisted to the primary replica’s local disk and are also transmitted to the secondary replicas. As your service matures, it’s likely you’ll want to change the kind of data (schema) your service requires. You must approach versioning of your data with great care. First and foremost, you must always be able to deserialize old data. Specifically, this means your deserialization code must be infinitely backward compatible: Version 333 of your service code must be able to operate on data placed in a reliable collection by version 1 of your service code 5 years ago.

    Furthermore, service code is upgraded one upgrade domain at a time. So, during an upgrade, you have two different versions of your service code running simultaneously. You must avoid having the new version of your service code use the new schema as old versions of your service code might not be able to handle the new schema. When possible, you should design each version of your service to be forward compatible by 1 version. Specifically, this means that V1 of your service code should be able to simply ignore any schema elements it does not explicitly handle. However, it must be able to save any data it doesn’t explicitly know about and simply write it back out when updating a dictionary key or value.

    Alternatively, you can perform what is typically referred to as a 2-phase upgrade. With a 2-phase upgrade, you upgrade your service from V1 to V2: V2 contains the code that knows how to deal with the new schema change but this code doesn’t execute. When the V2 code reads V1 data, it operates on it and writes V1 data. Then, after the upgrade is complete across all upgrade domains, you can somehow signal to the running V2 instances that the upgrade is complete. (One way to signal this is to roll out a configuration upgrade; this is what makes this a 2-phase upgrade.) Now, the V2 instances can read V1 data, convert it to V2 data, operate on it, and write it out as V2 data. When other instances read V2 data, they do not need to convert it, they just operate on it, and write out V2 data.


    Links:
     

    Creating forward compatible data contracts, see Forward-Compatible Data Contracts.

    Best practices on versioning data contracts, see Data Contract Versioning.

    Implement version tolerant data contracts, see Version-Tolerant Serialization Callbacks.

    Provide a data structure that can operate across multiple versions, see IExtensibleDataObject.


    Wednesday, April 17, 2019 12:14 AM
  • Any update on this issue?

    If the proposed answer was useful please remember to mark it as the answer so others can easily find it. 

    Friday, April 19, 2019 5:58 PM
    Moderator
  • Any update on this issue? 

    If a suggestion was useful, remember to upvote and mark as answer so others in the community can easily find it. 

    Friday, May 3, 2019 7:00 PM
    Moderator