locked
Monitoring storage transactions RRS feed

  • Question

  • Hello,

    I got  Azure free trial and I deployed my first application to azure. In my Azure bill, storage transaction amount is quite large and I don´t know why is so large. I would like to monitor storage transaction amount in development environment. Is there any  tool to analyze storage transactions before being billed? Should I use Azure Diagnostics?

     

    Thanks.

    Thursday, September 8, 2011 1:48 PM

Answers

All replies

  • You can do this in code if you are using table storage you can place a counter in the ReadingEntity and WritingEntity event and increase the value on each request.  Of course you will need to persist this somewhere if you want a running total you could event persist it for the month

    With BLOB and Queues though it isn't that simple.

    There may also be somthing in the Management API to query from the Azure point of view.  Unfortunatly I havn't used those API's yet.

     

    Thursday, September 8, 2011 1:59 PM
  • You can analyze your storage usage (transaction and capacity) using Storage Analytics. The blog post covers in detail information on how to set it up and where the data is stored. You can start with metrics that gives you an aggregated hourly view and if you want to dive into details of each transaction, you can use logs to analyze them. Steve has posted some extensions too @ http://blog.smarx.com/posts/analytics-leasing-and-more-extensions-to-the-net-windows-azure-storage-library.

    Thanks,

    jai

     

    Thursday, September 8, 2011 2:16 PM
  • Here's a thread from last week where BrentDaCodeMonkey and I discussed the various options to analyze Azure usage with the local development emulator. http://social.msdn.microsoft.com/Forums/en-US/windowsazuretroubleshooting/thread/9c1f83c5-791f-45a7-bff0-3c5f5cbf2475

    Eric


    Eric D. Boyd - Director, Chicago + Cloud at Centare 
    Blog | twitter
    Thursday, September 8, 2011 5:57 PM
  • You can analyze your storage usage (transaction and capacity) using Storage Analytics. The blog post covers in detail information on how to set it up and where the data is stored. You can start with metrics that gives you an aggregated hourly view and if you want to dive into details of each transaction, you can use logs to analyze them. Steve has posted some extensions too @ http://blog.smarx.com/posts/analytics-leasing-and-more-extensions-to-the-net-windows-azure-storage-library.

    Thanks,

    jai

     

    In addition to looking into Storage Analytics, the tip for developers who are new to Azure is to look into their pattern for querying of Queue information.  I'm often seeing new Azure developers who inadvertently implement a no-delay loop that keeps checking Azure Queues non-stop in a WorkerRole-WebRole pattern - thus generating hundreds of thousands of transactions against storage in short amounts of time.

    HTH


    Auto-scaling & monitoring service for Windows Azure applications at http://www.paraleap.com
    Tuesday, September 13, 2011 6:06 PM
  • Thanks for your answers,

    I am now using Storage Analytics to monitor storage usage.  I turned Storage Analytics Logging on and logging data is stored in $logs container. Then, I analyze blobs in this container. For example:

     

    Blob: https://myaccount.blob.core.windows.net/$logs/table/2011/09/12/1300/000000.log

    Log version

     Transaction Start Time

     REST Operation Type

     Transaction Status

     HTTP Status

    1.0

    2011-09-12T13:44:39.1400232Z

    QueryTables

    Success

    200

    1.0

    2011-09-12T13:45:11.0620232Z

    InsertEntity

    Success

    201

    1.0

    2011-09-12T13:45:11.2580232Z

    QueryEntities

    Success

    200

    1.0

    2011-09-12T13:45:29.0270232Z

    InsertEntity

    Success

    201

    1.0

    2011-09-12T13:45:29.2160232Z

    QueryEntities

    Success

    200

    Blob: https://myaccount.blob.core.windows.net/$logs/table/2011/09/12/1300/000001.log

    1.0

    2011-09-12T13:54:35.8571189Z

    InsertEntity

    Success

    201

    1.0

    2011-09-12T13:54:36.0660980Z

    QueryEntities

    Success

    200

    1.0

    2011-09-12T13:55:10.7373105Z

    QueryTables

    Success

    200

    1.0

    2011-09-12T13:55:15.4903105Z

    QueryEntities

    Success

    200

     

     

    I can see each blob stores several log entries, I would like to know if each of these log entries means a storage transaction. What is considered a storage transaction? If a request to the storage service is considered a storage transaction, can I say each log entry is a storage transaction?  

     

    Finally, thanks Igor for your answer too, I will review my code to avoid querying queue information too often.

    Wednesday, September 14, 2011 10:43 AM