locked
Manage IOPS of the Azure Storage Account RRS feed

  • Question

  • Hi all,

    I have Azure Storage Account and it's crossing 20K IOPS limit. How can I reduce the IOPS of the storage account? We are doing copy operation and also delete operation on the file share. Can we do this using batch operation so that it could be counted as one transaction.

    Please advise.

    Wednesday, February 27, 2019 5:13 AM

Answers

  • If all the IOs are on File Share then, you are also constrained by the SMB behavior and the nature of the Fileshare contents and if are you deleting many small files? Here basically you must be deleting + creating 20,000 files every second?

    Batch operations can be used only when you can delete the whole share

    e.g. if you are uploading a blob, then using put blob will help reduce the IOs, instead of multiple put blocks/pages etc

    You can use the following steps to measure the share IOPS from Performance Monitor. This can also be used to isolate which share has a significant amount of activity:

    1. Open Performance Monitor (PerfMon.msc).
    2. Go to Performance | Monitoring Tools | Performance Monitor.
    3. Delete all default counters at the bottom of the display.
    4. Click on the green + sign at the top of the display.
    5. Under "Available counters," expand "SMB Client Shares."
    6. Select "Data Requests/sec" which is the total number of IOPS.

    a.  You can also select "Data Bytes/sec" if you want to see throughput and whether you are getting to the 60 MBps limit.

    1. Under "Instances of selected object," choose the share(s) that you suspect are hitting the 1000 IOPS limit.

    a.  Use Ctrl to select multiple, or choose "<All Instances>" to select all.

    1. Click on "Add >>" and then click on OK.

    The graph will display the IOPS over time. The "Last" field displays the number of IOs in the previous second.


    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Wednesday, February 27, 2019 9:35 AM

All replies

  • If all the IOs are on File Share then, you are also constrained by the SMB behavior and the nature of the Fileshare contents and if are you deleting many small files? Here basically you must be deleting + creating 20,000 files every second?

    Batch operations can be used only when you can delete the whole share

    e.g. if you are uploading a blob, then using put blob will help reduce the IOs, instead of multiple put blocks/pages etc

    You can use the following steps to measure the share IOPS from Performance Monitor. This can also be used to isolate which share has a significant amount of activity:

    1. Open Performance Monitor (PerfMon.msc).
    2. Go to Performance | Monitoring Tools | Performance Monitor.
    3. Delete all default counters at the bottom of the display.
    4. Click on the green + sign at the top of the display.
    5. Under "Available counters," expand "SMB Client Shares."
    6. Select "Data Requests/sec" which is the total number of IOPS.

    a.  You can also select "Data Bytes/sec" if you want to see throughput and whether you are getting to the 60 MBps limit.

    1. Under "Instances of selected object," choose the share(s) that you suspect are hitting the 1000 IOPS limit.

    a.  Use Ctrl to select multiple, or choose "<All Instances>" to select all.

    1. Click on "Add >>" and then click on OK.

    The graph will display the IOPS over time. The "Last" field displays the number of IOs in the previous second.


    Kindly let us know if the above helps or you need further assistance on this issue.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

    Wednesday, February 27, 2019 9:35 AM
  • Thanks Sumanth for your time & suggestion. Yes, you are right we are deleting & creating more than 20000 files per second so hitting IOPS limit. 

    Please advise where I should run PerfMon.msc, on vm where I have mounted the file share ? Please confirm.

    Thanks.

    Wednesday, February 27, 2019 10:36 AM
  • @VishwajeetALMCloud 

     You can run on the mounted VM. The graph will display the IOPS over time. The "Last" field displays the number of IOs in the previous second.

    For more information on this scenario, refer to this article.

    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.



     
    Wednesday, February 27, 2019 11:50 AM
  • Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same, which might be beneficial to other community members reading this thread. And, if you have any further query do let us know.
    Thursday, February 28, 2019 5:25 AM