none
Azure Storage retention policy RRS feed

  • Question

  • Is there a way to implement a Azure table storage retention policy such that anything older than n number of days can be deleted from table storage? How would one go about implementing something like this?
    Tuesday, February 12, 2013 9:33 PM

Answers

  • Hi janzero,

    This is currently on our feature request list but we do not have a timeline to share.

    To accomplish this, you could create a new table every N days and issue a delete table request for the previous one. This would be the most efficient approach.

    Alternatively, consider batching deletes in an EntityGroupTransaction which can support up to 100 delete commands. However, this would require these entities to have same partitionKey.

    Thanks,

    Jean

    Wednesday, February 13, 2013 6:40 AM
  • Currently this is not something available via platform. You would need to implement it yourself.

    Regarding the implementation, it would be much easier and efficient if your PartitionKey somehow represented a date/time value. Do take a look at how Windows Azure Diagnostics data is configured. If you look closely at the PartitionKey value, it actually represents a date/time value (Ticks in UTC). If that's the case, then you could query the table for the date/time range using PartitionKey values and delete those records. For smaller tables, you could go with querying on "Timstamp" attribute but this would result in full table scan and thus not really suitable for very large tables.

    Hope this helps.

    Wednesday, February 13, 2013 2:39 AM

All replies

  • Currently this is not something available via platform. You would need to implement it yourself.

    Regarding the implementation, it would be much easier and efficient if your PartitionKey somehow represented a date/time value. Do take a look at how Windows Azure Diagnostics data is configured. If you look closely at the PartitionKey value, it actually represents a date/time value (Ticks in UTC). If that's the case, then you could query the table for the date/time range using PartitionKey values and delete those records. For smaller tables, you could go with querying on "Timstamp" attribute but this would result in full table scan and thus not really suitable for very large tables.

    Hope this helps.

    Wednesday, February 13, 2013 2:39 AM
  • Hi janzero,

    This is currently on our feature request list but we do not have a timeline to share.

    To accomplish this, you could create a new table every N days and issue a delete table request for the previous one. This would be the most efficient approach.

    Alternatively, consider batching deletes in an EntityGroupTransaction which can support up to 100 delete commands. However, this would require these entities to have same partitionKey.

    Thanks,

    Jean

    Wednesday, February 13, 2013 6:40 AM
  • you need to have a kind of scheduled task in a worker which checks for stale data and remove it.

    Mark Answered, if it solves your question and Vote if you found it helpful.
    Rohit Arora

    Wednesday, February 13, 2013 9:15 AM
  • Please check this article

    https://msdn.microsoft.com/en-us/library/azure/hh343263.aspx 

    Hope this helps.

    Wednesday, April 15, 2015 8:01 AM
  • Is this feature for retention policy of actual data in table storage implemented at all? Its been in the feature request list since last 3 years?
    Wednesday, August 31, 2016 8:53 PM