Store an array as an element in Azure Table

Question

Hi

I wanted to know if I can store an array of type double as an element of an entity(a row)?

I have an object that has 4 properties:

1) Location

2) Date

3) double[] Amplitude

4) double[] Phase

My data context stores location and date successfully, but I assignmet values to Amplitude and Phase too and it did not save those to the table.

Anthing of how I should proceed? Convert the array to byte and then store it as an element, isn't what I have on mind. I would really like if someone can just look at the table and still be able to make sense what the data is!

Regards,

Rakesh.

---

RK

Answer 1

Hi,

As far as i know, please do not add array in TabelServiceEntity as a property of table storage directly, because Table Storage is REST service, all data need be serialized. So please try to serialize your array to a Stream or Xml or other types, then you can use Table Storage API to upload them normallly.

Hope it can help you.

---

Please mark the replies as answers if they help or unmark if not. If you have any feedback about my replies, please contact [e-mail] Microsoft One Code Framework

Answer 2

Hi, thanks for the question. Arwind is correct, and the behavoir you are seeing is "normal" - arrays will not be stored in table entities, as they are not one of the types supported by .NET Data Services (bool, DateTime, Byte[], byte[], double, Guid, int, Int32, Int64, and string). However, you can store a non-supported type (including your double[]) by converting to and from one of the supported types, likely either string or one of the byte arrays. Here's some sample code that shows storing an array of doubles as a semicolon delimited string, but you could obviously change it to a different format or use this approach to deal with storing other types as well. Please make sure you test it for your uses, as this is just "proof-of-concept" type code, not tested for production uses (though I've at least tried to include suggestions for null and empty cases). One note to keep in mind with this code and others, is that the maximum legal size for a table entity is 1MB, and the maximum size for a batch transaction is 4MB. So if your serialization format is large (like XML), then you may run into those limits earlier.

Some brief explanation of this code: your application would do all access through the get/set for MyDoubles. When you set MyDoubles, this will set myDoubles and also set myDoubleString. Operations to and from table storage will then use the get/set for MyDoubleString. The “get” reads myDoubleString, which is always updated by the set for MyDoubles, and the “set” (used when reading entities from table storage), initializes myDoubles appropriately.

    // A simple enitity which has a double array
    public class DoublesArrayEntity : TableServiceEntity
    {
        private double[] myDoubles;
        private string myDoubleString;

        // This is for access by your application, which will work with the array.
        public double[] MyDoubles
        {
            get
            {
                // Just return the private array, which is updated by both setters
                return myDoubles;
            }
            set
            {
                // Simple assignment to the private array
                myDoubles = value;
                
                // This check is necessary to make sure we preserve "null" as different than "empty"
                if (myDoubles != null)
                {
                    // Build a string representing the doubles in the list, like "1.5;2.2;3.4;4.8"
                    StringBuilder sb = new StringBuilder();
                    foreach (double d in value)
                    {
                        sb.Append(d);
                        sb.Append(';');
                    }
                    // Trim the last semi-colon, so that we have "1;2;3" instead of "1;2;3;"
                    myDoubleString = sb.ToString().TrimEnd(';');
                }
                else
                {
                    // Assign null to the string to preserve the fact that the array was null
                    myDoubleString = null;
                }
            }
        }

        // This is for storing in table storage
        public string MyDoubleString
        {
            get
            {
                // Just return the private string, which is updated by both setters
                return myDoubleString;
            }
            set
            {
                // Simple assignment of the string read from table storage
                myDoubleString = value;

                // Initialize the array by converting the string.  Make sure that null and empty are treated correctly.
                if (!String.IsNullOrEmpty(myDoubleString))
                {
                    // Split on ';' to get the individual doubles
                    string[] dStrings = value.Split(';');
                    int len = dStrings.Length;
                    // Create an array the same length
                    myDoubles = new double[len];
                    for (int i = 0; i < len; i++)
                    {
                        // Store each in the same order they were retured.
                        myDoubles[i] = Convert.ToDouble(dStrings[i]);
                    }
                }
                // Empty string would split to one string if we didn't handle it separately, resulting in the wrong array.
                else if (myDoubleString == String.Empty)
                {
                    // So we just create a new empty array here.
                    myDoubles = new double[0];
                }
                else
                {
                    // If it's null, just assign that.
                    myDoubles = null;
                }
            }
        }

        public DoublesArrayEntity() { }
    }

---

-Jeff

Answer 3

Thanks guys!

Yeah, this is probably a good idea.

The only problem is that my worker role sends a request to the WCF service, hosted somewhere else, to work on this data(array of 40 + 40, for 365 days) and then perform some very complex operation on it, which takes a lot of time, during this time the underlying connection(HTTP binding b/w client(worker role) and server(WCF service)) gets killed by the load balancer ( This exception, in detail here ). So, converting to one type and then re-converting would add up more time for the operation.

Still trying to find a solution for this.

---

RK

Answer 4

Hi - glad this helped somewhat.  If performance is your primary concern over human-readability, you could consider other serialization options, such as BinarySerializer, which may be faster.  Binary serialization will probably also result in a smaller entity size.  However, unfortunately the double[] type is not supported at this time, so a type conversion is necessary to store this data in a table entity.

Let us know if we can help any further!

---

-Jeff

Answer 5

Hello,

I have recently released an Azure Storage Table client, Lucifure Stash, which supports arrays and list intrinsically. This is done by storing each element of the array as a separate property in the table after applying an index suffix to the array name.

This is a shameless plug, but Lucifure Stash, is available absolutely free for personal use, no strings attached. It simplifies array storage and supports many other features like enumerations, string and byte array larger than 64K, in-built and user-defined serialization etc. It comes with a tutorial written as C# test cases to demonstrate all the features.

You can get it at www.lucifure.com.

Edit: Now open sourced at CodePlex