none
Iterate through blob files in Azure Storage

    Question

  • Hi Guys,

    I have a PowerShell script that gives me Azure Blobs metadata.

    However, I don't think the script is working in an efficient way.

    You see I have over a million files in azure storage and this script is iterating through all the files (millions of files) before giving me the specific metadata that I'm requesting based on my variable value

    Is there a way to point my single object $_. to go straight to a specific path within the azure blob storage?

    #Define Context Variable
    $Ctx = New-AzureStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey 
    
    #List all blobs in a container. 
    $blobs = Get-AzureStorageBlob -Container $ContainerName -Context $Ctx  
    
    $FullPath = "PMS/OnQ/Hilton/OARevenueExtract/$BrandYear"
    $blobs | Where-Object {$_.Name.Contains($FullPath)} | Select-Object Name,LastModified,Length | Format-Table -AutoSize 

    Thank you


    Codernater

    Tuesday, February 28, 2017 11:30 PM

Answers

All replies

    1. Are you using -Prefix argument for the Get-AzureStorageBlob call?

    Specifies a prefix for the blob names that you want to get. This parameter does not support using regular expressions or wildcard characters to search. This means that if the container has only blobs named "My", "MyBlob1", and "MyBlob2" and you specify "-Prefix My*", the cmdlet returns no blobs. However, if you specify "-Prefix My", the cmdlet returns "My", "MyBlob1", and "MyBlob2".

    Reference: https://docs.microsoft.com/en-us/powershell/storage/azure.storage/v2.3.0/get-azurestorageblob#parameters  

    • Marked as answer by Codernater Wednesday, March 1, 2017 9:33 PM
    Wednesday, March 1, 2017 12:55 PM
    Moderator
  • Thank you

    for the community here is my final Script

            $FullPath = "PMS/Advantage/Choice/BackOffice/$BrandYear"
            #List all blobs related to brand selection in variable. 
            $Blobs = Get-AzureStorageBlob -Container $ContainerName -Context $Ctx -Prefix $FullPath
            $Blobs | Select-Object Name,LastModified,Length | Format-Table -AutoSize


    Codernater

    Wednesday, March 1, 2017 9:34 PM