none
How to trace Azure File Storage - File Read/Write Operations?

    Întrebare

  • Recently Microsoft began billing us for Azure File Storage transaction. 

    I reviewed the usage data and it appears we're doing a lot of transactions on our central file storage. Below is the data for 1 day, coming from the usage CSV file.

    "Standard IO - File Protocol Operation Units (in 10,000s)" 700.873500
    "Standard IO - File Delete Operation Units (in 10,000s)" 0.274100
    "Standard IO - File List Operation Units (in 10,000s)" 0.220200
    "Standard IO - File Read Operation Units (in 10,000s)" 825.954000
    "Standard IO - File Write Operation Units (in 10,000s)" 798.692400

    On a montly basis, this resulted in over 217 million File Write operations and 229 million reads. Wow.

    How do I figure out which processes are doing all these operations? 

    joi, 27 octombrie 2016 13:51

Toate mesajele

  • Hello

    You can refer this link

    https://blogs.msdn.microsoft.com/windowsazurestorage/2010/07/08/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity/


    Regards, Regin Ravi

    joi, 27 octombrie 2016 14:18
  • While that link is somewhat useful, it doesn't cover Azure File storage (https://azure.microsoft.com/en-us/services/storage/files/)

    Since my previous post I did some testing and with the help of the Azure Usage API I can now get hourly insights into the number of transactions. Unfortunately the API lags a few hours behind, so after every test I have to wait about 6 hours to see the results. Atleast now I can disable parts of our system for an hour and see the effect of that change a few hours later.

    One of the tests I did was on a new and unused File storage. I simply copied 1 file from a virtual machine to the shared storage. The filesize was 450,751 KB (~451MB). This resulted in:

    "meterName": "Standard IO - File Write Operation Units (in 10,000s)",
    "quantity": 0.0451

    So apparently write transactions are divided into 1MB chunks or something, some documentation to confirm this would be helpful.

    I also installed Process Monitor (https://technet.microsoft.com/en-us/sysinternals/processmonitor.aspx) on several VM's to get more insight into what's actually writing to the shared storage. With a lot of filtering and trial and error I found some operations that result in File Write operations:

    Operation: WriteFile
    Operation: CreateFile with Disposition : Create
    Operation: CreateFile with Disposition: OverwriteIf

    There are probably more, because counting these events dont amount for our monthly total. 

    I'll investigate this further, but wanted to update the thread with my findings sofar. Any feedback is appreciated.




    • Editat de RikD_ miercuri, 2 noiembrie 2016 15:10
    miercuri, 2 noiembrie 2016 15:07
  • Hi,

    Thank you for posting here!

    You could try to enable Analytics Log to monitor the IP Address of the requester.
    More detail on this can be found in the link given below.
    https://msdn.microsoft.com/library/hh343259.aspx?f=255&MSPPError=-2147217396


    Regards.

    Md. Shihab

     

    Please remember to click "Mark as Answer" on the post that helps you as this can be beneficial to other community members reading the thread. And vote as helpful.


    miercuri, 2 noiembrie 2016 18:19
  • Hi! Have you solved your issue? We currently have a similar issue - suddenly file read/write operation metrics became 10 times larger and we have no clue what have caused it and how to find that out. Did you have any success with that?
    vineri, 16 decembrie 2016 12:31
  • Hi! Have you solved your issue? We currently have a similar issue - suddenly file read/write operation metrics became 10 times larger and we have no clue what have caused it and how to find that out. Did you have any success with that?
    I also have this problem. I see peaks of file operations but i dont even use file storage for my function that is running? Any solution?
    joi, 12 iulie 2018 09:18
  • @Ponziworld:

    Suggest you to create a new forum thread along with more details on your environment .

    vineri, 13 iulie 2018 22:04
    Moderator