Fast load Reliable Dictionaries

Question

I am considering loading a ReliableDictionary with approx. 250k items. Each item is quite small (2 strings, 2  ints, 1 date)

I have tried loading via a file deployed in the data package. This is giving me 5000 items loading in about 21 seconds so 250k would be around 17-18 minutes.

Is there any rule of thumb guidance on the size of a ReliableDictionary? Is 250k considered big/small/ok?

What would be the fastest method to load items in to a ReliableDictionary?

Answer 1

Hi,

I have reached out to the internal teams. I will post the answer here once i have some information.

Answer 2

Hi,

Are you committing to the  Reliable collection after uploading each item?

If yes,  can you try loading all the items and commit at the end.

Answer 3

Yes committing for each item. I'll try your suggestion and update here. 

Answer 4

Adds 5000 in less than 2 seconds - good enough. Thanks

Answer 5

Hi,

Good to know that the time reduced.

When we commit to a reliable collection, It writes  that collection in the primary and all secondary replicas.

Once it gets the confirmation from the secondary replicas, Then we can do another write.

Because of this process it took more time when we committed for each entry.

Its explained in the document here. 

When we are uploading in bulk, We can load all items and commit at the end.

For the benefit of the community, Please click on "mark as answer" for the replies which helped you to solve the problem.