locked
How to get the Access Log for the Storage Account? RRS feed

  • Question

  • Hi,

    I want to know how public users download files from my Azure storage account. I am reading the article https://docs.microsoft.com/en-us/rest/api/storageservices/enabling-storage-logging-and-accessing-log-data , but still have some problems:

    1. In "How to enable Storage Logging using the Azure portal", it said to use "Diagnostic" blade. However, in my account, in "Monitor" section, there is no "Diagnostic" blade. Only in "Moniter(classic") section, there is "Diagnostic Settings" blade. Does that mean the new(non-classic) monitor does not support to get access log?

    2. In the Monitor(classic) -> Diagnostic Settings, I see "Hours Metrics" are selected already, does that mean the log is enabled by default?

    3. In "Finding your Storage Logging log data", it said one should use "storage-browsing tool" to see the hidden $log container. But where is the "storage-browsing tool"?

    Thanks

    Thursday, March 14, 2019 12:54 AM

Answers

  • Apologies for the delay!
    1. Storage Logging using portal you can enable Metrics.

    You can monitor metrics over time in the Azure portal.  Storage account->Monitoring-> Metrics options Access metrics in the Azure portal

    2. Hour Metrics 

    The Azure portal does not currently enable you to configure minute metrics in your storage account; you must enable minute metrics using PowerShell or programmatically.

    The cmdlets that control Storage Metrics use the following parameters:

    • MetricsType: possible values are Hour and Minute.
    • ServiceType: possible values are Blob, Queue, and Table.
    • MetricsLevel: possible values are None, Service, and ServiceAndApi.
    $storagecontext = New-AzStorageContext -StorageAccountName <storageaccountname> -StorageAccountKey <storageaccountkey>
    
    Set-AzStorageServiceMetricsProperty -MetricsType Minute -ServiceType Blob -MetricsLevel ServiceAndApi  -RetentionDays 5`  -Context $storagecontext.context
    
    Get-AzStorageServiceMetricsProperty -MetricsType Hour -ServiceType Blob -Context $storagecontext.Context

    For more information, you may refer to the suggestions mentioned in this article.

     3. Storage - browsing tool is Microsoft Azure Storage Explorer

    Azure Storage Explorer: Easily manage Storage anywhere from Windows, macOS and Linux, Access multiple accounts and subscriptions across Azure, Azure Stack, and the sovereign Cloud, Create, delete, view, and edit storage resources, View and edit Blob, Queue, Table, File, Cosmos DB storage and Data Lake Storage.

    Viewing $logs blob containers for Storage Accounts with enabled metrics


    Additional information: Third-Party Azure Storage Client Tools

    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.


    Friday, March 15, 2019 10:19 AM
  • As I understand you need the information on download count for a given blob stored in azure blob storage.

     Blob storage doesn't support getting the download count directly now

    1.On the client side.

    You could try to count the download number by creating an application.

    If user click the download button or something else, get the number of downloading count (you could store it into blob file matadata) and calculate.

    Notice: the azure blob has public and private permission. If the blob is public, we could directly download the blob from the URL. So, I would suggest you could try to set the blob permission to private. By doing this the number of downloading count t is right.

    2.On the server side.

    If you enable the azure storage account's diagnostics' blob logs, it will log the blob's each read/write/delete operations.

    By reading these operations, I think you could get the download (getblob operation) count for a given blob stored in azure blob storage.

    These logs are storing in the $log container.

    Notice: The storage log has size limit(20TB) and date limit(MAX 365). So I suggest you could try to run a timertrigger webjob to search the logs. In this webjobs, you could use azure storage SDK's CloudAnalyticsClient to get the logs and store the number and scanned date in the file metadata. Each time when the timertrigger webjob runs, you just need to search the context from last scanned date.

    Additional information: You may also refer to the suggestions mentioned in this TechNet Link.

    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Monday, March 25, 2019 9:57 AM

All replies

  • Apologies for the delay!
    1. Storage Logging using portal you can enable Metrics.

    You can monitor metrics over time in the Azure portal.  Storage account->Monitoring-> Metrics options Access metrics in the Azure portal

    2. Hour Metrics 

    The Azure portal does not currently enable you to configure minute metrics in your storage account; you must enable minute metrics using PowerShell or programmatically.

    The cmdlets that control Storage Metrics use the following parameters:

    • MetricsType: possible values are Hour and Minute.
    • ServiceType: possible values are Blob, Queue, and Table.
    • MetricsLevel: possible values are None, Service, and ServiceAndApi.
    $storagecontext = New-AzStorageContext -StorageAccountName <storageaccountname> -StorageAccountKey <storageaccountkey>
    
    Set-AzStorageServiceMetricsProperty -MetricsType Minute -ServiceType Blob -MetricsLevel ServiceAndApi  -RetentionDays 5`  -Context $storagecontext.context
    
    Get-AzStorageServiceMetricsProperty -MetricsType Hour -ServiceType Blob -Context $storagecontext.Context

    For more information, you may refer to the suggestions mentioned in this article.

     3. Storage - browsing tool is Microsoft Azure Storage Explorer

    Azure Storage Explorer: Easily manage Storage anywhere from Windows, macOS and Linux, Access multiple accounts and subscriptions across Azure, Azure Stack, and the sovereign Cloud, Create, delete, view, and edit storage resources, View and edit Blob, Queue, Table, File, Cosmos DB storage and Data Lake Storage.

    Viewing $logs blob containers for Storage Accounts with enabled metrics


    Additional information: Third-Party Azure Storage Client Tools

    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.


    Friday, March 15, 2019 10:19 AM
  • Hi,

    Just more specific, if I need to see who download my Blob binary files, I use use "xxxx, Egress, Sum", where xxxx is the storage account name. Is that correct? 

    Saturday, March 16, 2019 2:48 AM
  • As I understand you need the information on download count for a given blob stored in azure blob storage.

     Blob storage doesn't support getting the download count directly now

    1.On the client side.

    You could try to count the download number by creating an application.

    If user click the download button or something else, get the number of downloading count (you could store it into blob file matadata) and calculate.

    Notice: the azure blob has public and private permission. If the blob is public, we could directly download the blob from the URL. So, I would suggest you could try to set the blob permission to private. By doing this the number of downloading count t is right.

    2.On the server side.

    If you enable the azure storage account's diagnostics' blob logs, it will log the blob's each read/write/delete operations.

    By reading these operations, I think you could get the download (getblob operation) count for a given blob stored in azure blob storage.

    These logs are storing in the $log container.

    Notice: The storage log has size limit(20TB) and date limit(MAX 365). So I suggest you could try to run a timertrigger webjob to search the logs. In this webjobs, you could use azure storage SDK's CloudAnalyticsClient to get the logs and store the number and scanned date in the file metadata. Each time when the timertrigger webjob runs, you just need to search the context from last scanned date.

    Additional information: You may also refer to the suggestions mentioned in this TechNet Link.

    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Monday, March 25, 2019 9:57 AM
  • Thank you very much.
    Tuesday, March 26, 2019 1:55 AM
  • Sorry but I have downloaded Microsoft Azure Storage Explorer, then login my account, but still cannot find the $log container, why?
    Tuesday, March 26, 2019 3:05 AM
  • All logs are stored in block blobs in a container named $logs, which is automatically created when Storage Analytics is enabled for a storage account. The $logs container is located in the blob namespace of the storage account,

    Please enable the Storage account logging using the below mentioned PS cmdlets:

    $storagecontext = New-AzStorageContext -StorageAccountName <Storage account name> -StorageAccountKey <storageaccountkey>
    
    
    Set-AzStorageServiceMetricsProperty -MetricsType Minute -ServiceType Blob -MetricsLevel ServiceAndApi  -RetentionDays 5`  -Context $storagecontext.context

    You may also refer to the suggestions mentioned in the SO thread. 

    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Up-vote on the post that helps you, this can be beneficial to other community members.

    Tuesday, March 26, 2019 12:07 PM
  •  @tempc Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.
    Monday, April 15, 2019 6:04 PM