Answered by:
Fast load Reliable Dictionaries

Question
-
I am considering loading a ReliableDictionary with approx. 250k items. Each item is quite small (2 strings, 2 ints, 1 date)
I have tried loading via a file deployed in the data package. This is giving me 5000 items loading in about 21 seconds so 250k would be around 17-18 minutes.
Is there any rule of thumb guidance on the size of a ReliableDictionary? Is 250k considered big/small/ok?
What would be the fastest method to load items in to a ReliableDictionary?
Friday, October 18, 2019 8:57 AM
Answers
-
Hi,
Are you committing to the Reliable collection after uploading each item?
If yes, can you try loading all the items and commit at the end.
- Proposed as answer by jakaruna-MSFTMicrosoft employee, Owner Tuesday, October 22, 2019 4:55 AM
- Marked as answer by dob_xml Tuesday, October 22, 2019 1:24 PM
Friday, October 18, 2019 12:29 PMOwner
All replies
-
Hi,
I have reached out to the internal teams. I will post the answer here once i have some information.
Friday, October 18, 2019 12:01 PMOwner -
Hi,
Are you committing to the Reliable collection after uploading each item?
If yes, can you try loading all the items and commit at the end.
- Proposed as answer by jakaruna-MSFTMicrosoft employee, Owner Tuesday, October 22, 2019 4:55 AM
- Marked as answer by dob_xml Tuesday, October 22, 2019 1:24 PM
Friday, October 18, 2019 12:29 PMOwner -
Yes committing for each item. I'll try your suggestion and update here.Friday, October 18, 2019 3:14 PM
-
Adds 5000 in less than 2 seconds - good enough. ThanksMonday, October 21, 2019 9:23 AM
-
Hi,
Good to know that the time reduced.
When we commit to a reliable collection, It writes that collection in the primary and all secondary replicas.
Once it gets the confirmation from the secondary replicas, Then we can do another write.
Because of this process it took more time when we committed for each entry.
Its explained in the document here.
When we are uploading in bulk, We can load all items and commit at the end.
For the benefit of the community, Please click on "mark as answer" for the replies which helped you to solve the problem.
Tuesday, October 22, 2019 5:02 AMOwner