locked
Azure Blob IOPS limit question RRS feed

  • Question

  • Hi, I'm looking over the service limits for blob storage, and I came across the following limit:

    Total Request Rate (assuming 1KB object size) per storage account: 20,000

    Does this mean that if all of my objects were 2KB in size, would my request rate be 10,000? what about 3KB? 4KB? Basically I want to know how does that request rate scale. I searched and couldn't find any info about this.

    Thanks for any info.

    Thursday, May 7, 2015 9:10 PM

Answers

  • Hi icarpenter,

    The input/output (I/O) unit size is 8KB.

    If the data being transferred is less than 8KB, it is considered a single I/O unit. The larger I/O sizes are counted as multiple I/Os of size 8KB.

    Hope this helps.

    Regards,

    Mekh.

    • Marked as answer by Manu Rekhar Monday, May 11, 2015 6:30 PM
    Friday, May 8, 2015 9:49 AM

All replies

  • Hi icarpenter,

    The input/output (I/O) unit size is 8KB.

    If the data being transferred is less than 8KB, it is considered a single I/O unit. The larger I/O sizes are counted as multiple I/Os of size 8KB.

    Hope this helps.

    Regards,

    Mekh.

    • Marked as answer by Manu Rekhar Monday, May 11, 2015 6:30 PM
    Friday, May 8, 2015 9:49 AM
  • Hi -

    Can you provide a clearer idea of what you are trying to do?

    The max 8 KB IOPS applies to reads/writes to a persistent disk in an Azure VM (as shown here). 

    However, if you are not using an Azure VM, then the figure you quoted above (Total Request Rate (assuming 1KB object size) per storage account: 20,000) applies only to messages in queues or entities in tables, not blobs. 

    The target throughput for a single blob is up to 60 MB per second, or up to 500 requests per second. 

    Tuesday, May 12, 2015 9:46 PM
  • Hi All,
    I think Your answer is not really clear - same as Azure documentation is not clear in this topic. I spent all day on testing VM data (persistent) disks performance and comparing it with the documentation.

    Facts:

    1) There is a transfer performance limit ~60MB/s - in my opinion 62.5MB/s (will answer later why) but it is not a very hard limit - sometimes it can be a bit bigger (like 70MB/s) but it can depend on way we measure, so let say 62.5MB/s.

    2) There is IOPS limit, globally for all standard blobs (including VM data disks): 500IOPS

    3) In premium storage with 256kB I/O unit everything seems to be mathematically clear (500x 256kB = 125MB/s, with throttling  we have 100MB/s per P10 disk). So limits are clear - 500IOPS or 100MB/s and so on with P20 and P30.

    4) I/O unit for standard storage is not clear. With 8kB unit and 500IOPS limit we have 500x 8kB = 3.9MB/s. Transfer limit is ~60MB/s so mathematically (in my opinion - I can be wrong) it should be not possible to reach 60MB/s but... there is no problem with it. Disks are running with no problem at ~60MB/s and in RAID0 configuration we have transfers equal to "performance = ~60MB/s x # of disks" so in RAID0 with 8 disks I have ~480MB/s.

    From those facts we can calculate, that IOPS unit for standard storage is equal to 128kB. 500x 128kB = 62.5MB/s.

    Can You clarify the mathematics of 8kB I/O unit size and prove that I'm wrong or check if maybe I'm right and change the documentation? It is very confusing in example during RAID configuration and optimization.

    Best regards,

    Michal

    Friday, July 10, 2015 6:16 PM
  • Sorry for the late reply. My use case involves writing millions of blobs, specifically block blobs, of sizes ranging from under 1KB to 80KB in size, to Azure storage. I'm trying to figure out what the theoretical maximum limit I can write these blobs is before getting throttled. I know I'm not reaching the 10Gbps ingress limit, however, I'm unsure if I'm reaching the message limit because the documentation is very unclear on how it applies to block blobs.

    The throughput for an individual blob is irrelevant for this use case as I am only writing these blobs once.

    Tuesday, July 14, 2015 1:35 AM