none
Fast load Reliable Dictionaries RRS feed

  • Question

  • I am considering loading a ReliableDictionary with approx. 250k items. Each item is quite small (2 strings, 2  ints, 1 date)

    I have tried loading via a file deployed in the data package. This is giving me 5000 items loading in about 21 seconds so 250k would be around 17-18 minutes.

    Is there any rule of thumb guidance on the size of a ReliableDictionary? Is 250k considered big/small/ok?

    What would be the fastest method to load items in to a ReliableDictionary?

    Friday, October 18, 2019 8:57 AM

Answers

All replies

  • Hi,

    I have reached out to the internal teams. I will post the answer here once i have some information.


    Friday, October 18, 2019 12:01 PM
    Moderator
  • Hi,

    Are you committing to the  Reliable collection after uploading each item?

    If yes,  can you try loading all the items and commit at the end.

    Friday, October 18, 2019 12:29 PM
    Moderator
  • Yes committing for each item. I'll try your suggestion and update here. 
    Friday, October 18, 2019 3:14 PM
  • Adds 5000 in less than 2 seconds - good enough. Thanks
    Monday, October 21, 2019 9:23 AM
  • Hi,

    Good to know that the time reduced.

    When we commit to a reliable collection, It writes  that collection in the primary and all secondary replicas.

    Once it gets the confirmation from the secondary replicas, Then we can do another write.

    Because of this process it took more time when we committed for each entry.

    Its explained in the document here

    When we are uploading in bulk, We can load all items and commit at the end.

    For the benefit of the community, Please click on "mark as answer" for the replies which helped you to solve the problem.

    Tuesday, October 22, 2019 5:02 AM
    Moderator