We were just trying out the Azure Storage Analytics Service, and something very unusual caught our attention.
The transaction count for the diagnostics storage account ( the account to which the Diagnostics Service writes it's data) was extremely high. We are talking about
600~ transaction per hour, all of which are GetBlob() operations, and all of them ended with error ( ClientOtherError is equal to the total number of operations ).
Further investigation revealed that each instance running which has
Diagnostics turned on, produces 300~ transactions per hour
( we has 2 instances, thus the 600). Continuing the investigation, looking at the $logs that the Analytics Service is producing revealed what really going on :
The log is filled with lots of calls to an xml file that's not exists. The log file itself is very cluttered but it's very clear that most of the calls are searching for
https://*****.blob.core.windows.net/mam/MACommand.xml and also /mam/MACommandb.xml
all those calls have an error of 404.
This issue is a real problem for us, and we have no idea what causing it.
Has anyone encountered this issue ?
(edit: Forgot to mention, the Diagnostics Service is not logging anything - scheduledTransferPeriod is zero for all the categories)
(edit: here is some part of the log file, sensitive information is replaced with ****
Thank you, but this is not exactly the same problem. Although his log shows something about the enigmatic MACommand.xml, his problem is related
to the emulator. My problem is, as I mentioned, when the WebRole is live and running in staging.
After a week trying to figure this out, with the help of
Farida Bharmal from the azure support team, we managed solve the issue. Those calls are an expected behavior,
and happens from SDK 1.6.
Taken from a response Farida got from the product team :
"I also got a response from
the product team. There is a change in the SDK 1.6. These calls are expected. There are about 20 calls per 5 minutes for every role instance. Which means that there will be 6000 calls a day. This will cost around a penny every day for every role instance.
(Storage Transactions = $0.01 per 10,000 transactions)"
The product team also said that they looking into optimizing this behavior.
Marked As Answer byYaron LeviTuesday, January 10, 2012 9:57 PM