Avoid getting duplicate file from Azure Blob Storage


  • Hi,

    I am writing a program to download files from Azure blob storage to on premises file system using Azcopy or Azure Cli commands.  My process is running continues to pull files from Storage and pushing to file system. Is there any way to avoid same file download again from Azure storage.  When my process run again i dont want to pull the files which i have already pulled in previous run.

    Please suggest way to avoid duplicate files

    Scenario :

    Time   Azure Storage            Process                       File system

    A         F1,F2,F3                    P - running      pull files  F1,F2,F3

    After some time 2 more files added to storage

    B        F1,F2,F3,F4,F5         P -running    pull files  F4,F5        (should not pull F1,F2)        

    Also i dont want to maintains files at file system


    Pawan Chhajer

    miercuri, 16 mai 2018 20:23

Toate mesajele

  • See the link which might helps you .


    If this answer was helpful, click “Mark as Answer” or Up-Vote. To provide additional feedback on your forum experience, click here

    miercuri, 16 mai 2018 20:38
  • Hey Pawan ,

    Just checking in to see if the above suggestions helped or you need further assistance on this issue.

    vineri, 18 mai 2018 17:47