How to trace Azure File Storage - File Read/Write Operations?

Question

Recently Microsoft began billing us for Azure File Storage transaction. 

I reviewed the usage data and it appears we're doing a lot of transactions on our central file storage. Below is the data for 1 day, coming from the usage CSV file.

"Standard IO - File Protocol Operation Units (in 10,000s)" 700.873500
"Standard IO - File Delete Operation Units (in 10,000s)" 0.274100
"Standard IO - File List Operation Units (in 10,000s)" 0.220200
"Standard IO - File Read Operation Units (in 10,000s)" 825.954000
"Standard IO - File Write Operation Units (in 10,000s)" 798.692400

On a montly basis, this resulted in over 217 million File Write operations and 229 million reads. Wow.

How do I figure out which processes are doing all these operations? 

Answer 1

Hello

You can refer this link

https://blogs.msdn.microsoft.com/windowsazurestorage/2010/07/08/understanding-windows-azure-storage-billing-bandwidth-transactions-and-capacity/

---

Regards, Regin Ravi

Answer 2

While that link is somewhat useful, it doesn't cover Azure File storage (https://azure.microsoft.com/en-us/services/storage/files/)

Since my previous post I did some testing and with the help of the Azure Usage API I can now get hourly insights into the number of transactions. Unfortunately the API lags a few hours behind, so after every test I have to wait about 6 hours to see the results. Atleast now I can disable parts of our system for an hour and see the effect of that change a few hours later.

One of the tests I did was on a new and unused File storage. I simply copied 1 file from a virtual machine to the shared storage. The filesize was 450,751 KB (~451MB). This resulted in:

"meterName": "Standard IO - File Write Operation Units (in 10,000s)",
"quantity": 0.0451

So apparently write transactions are divided into 1MB chunks or something, some documentation to confirm this would be helpful.

I also installed Process Monitor (https://technet.microsoft.com/en-us/sysinternals/processmonitor.aspx) on several VM's to get more insight into what's actually writing to the shared storage. With a lot of filtering and trial and error I found some operations that result in File Write operations:

Operation: WriteFile
Operation: CreateFile with Disposition : Create
Operation: CreateFile with Disposition: OverwriteIf

There are probably more, because counting these events dont amount for our monthly total. 

I'll investigate this further, but wanted to update the thread with my findings sofar. Any feedback is appreciated.

Answer 3

Hi,

Thank you for posting here!

You could try to enable Analytics Log to monitor the IP Address of the requester.
More detail on this can be found in the link given below.
https://msdn.microsoft.com/library/hh343259.aspx?f=255&MSPPError=-2147217396

Regards.

Md. Shihab

 

Please remember to click "Mark as Answer" on the post that helps you as this can be beneficial to other community members reading the thread. And vote as helpful.

Answer 4

Hi! Have you solved your issue? We currently have a similar issue - suddenly file read/write operation metrics became 10 times larger and we have no clue what have caused it and how to find that out. Did you have any success with that?

Answer 5

Hi! Have you solved your issue? We currently have a similar issue - suddenly file read/write operation metrics became 10 times larger and we have no clue what have caused it and how to find that out. Did you have any success with that?

I also have this problem. I see peaks of file operations but i dont even use file storage for my function that is running? Any solution?

Answer 6

@Ponziworld:

Suggest you to create a new forum thread along with more details on your environment .