WADLogsTable not generated - anything wrong if package is created using msbuild outside of Visual Studio

Question

Logging has suddenly stopped working in one of our environments. I have tried all the recommendations on various threads including changes after 1.4 of full IIS. My invesitation is suggesting that if package is generated using MSBuild from TFS, no matter what logs are not generated (transfered???). But if the same code is compiled and package is created using Visual Studio 2010 IDE, it works. Have someone saw this issue before?

Thanks

---

Arvind Maurya

Answer 1

Hi,

  >> But if the same code is compiled and package is created using Visual Studio 2010 IDE, it works.

Can you make sure the output packages are exactly the same? In addition, the latest SDK (November 2011) has better support for TFS and MSBuild. I would like to suggest you to see whether it helps.

 

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 2

I must say that this sounds very similar to what I'm experiencing with the 1.5 SDK.  I have the diagnostic trace listener transferring data as it should, but performance counters are not (despite having them work when I deployed though VS).  I'm not able to upgrade to the 1.6 SDK until early next week to test to see if this makes a difference. 

Answer 3

Thanks Ming. Is there anyway we can compare the packages other than opening them as zip files and manually looking at the content. I tried doing manual verification and have not found any major difference.

Thanks

Arvind

---

Arvind Maurya

Answer 4

Interesting. I have also seen (performance counters not logged) this but had not posted in the forum as was trying to figure out myself whats wrong.

 

Thanks

Arvind

---

Arvind Maurya

Answer 5

Thanks Ming. Is there anyway we can compare the packages other than opening them as zip files and manually looking at the content. I tried doing manual verification and have not found any major difference.

Thanks

Arvind

---

Arvind Maurya

No better approach as far as I know. May I get your working/failing packages to compare? Maybe I can find something. My email is allenc at microsoft.com

---

Allen Chen [MSFT]
MSDN Community Support | Feedback to us
Get or Request Code Sample from Microsoft
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 6

I am experiencing the same issue - the WadLogsTable table is created and I can see the trace logs without problems if I publish using the Visual Studio IDE.  

But if we create the azure package using MSBuild and deploy through TFS, the trace logs no longer work.  It did not make a difference which SDK I was using - the problem happened with both 1.5 and 1.6.

I was hoping the announced support for new MSBuild features with Azure SDK 1.6 would help - but I see no documentation on how to use them.  Does anyone know of any examples or such documentation?

 

Here's the MSBuild call we make today -

'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MsBuild.exe' '..\Source\OURSOLUTIONNAME.sln' /p:Configuration='OURTARGETSOLUTIONCONFIGURATION' /p:Platform=x64 /t:Publish /v:q

What can we do differently with the Azure 1.6 SDK?

 

Answer 7

Hi Emmanuel

I had exactly the same problem as you, and it was thanks to your post that finally clued me in on how to solve it.  First of all, the answer does not lie with MSBuild, it lies with CSpack.  CSPack is what Visual Studio calls to package the solution ready for deployment to Azure.  It's a command line util, and you can script it so it's part of your build process.

I use NAnt rather than TFS for our Continuous Integration, so hopefully you can use the steps I outline, in the blog post I wrote on the subject.  You need to ensure CSPack includes the new (as of SDK 1.5 entrypoint param).  Once I did that, diagnostics where flowing into Azure Storage again :-)

Auto Packaging using CSPack and Azure SDK 1.6

Hope it solves your problem.

Cheers

Iain

Answer 8

Hi Iain,

Thanks for your response and tweet - I checked your solution and saw how you made sure cspack had the correct entrypoint.

In our case, we are using msbuild to create the azure package, with the "/t:Publish" parameter - so I'm not sure if I have to or how to specify the entrypoint to msbuild.

Can anyone from Microsoft help?

Answer 9

Hi Iain,

Thanks for your response and tweet - I checked your solution and saw how you made sure cspack had the correct entrypoint.

In our case, we are using msbuild to create the azure package, with the "/t:Publish" parameter - so I'm not sure if I have to or how to specify the entrypoint to msbuild.

Can anyone from Microsoft help?

No one from Microsoft can help?

I thought the forums were moderated - this is a pretty HUGE issue for anyone deploying through TFS and using MSBUILD to create the azure package.

 

Answer 10

Hi Emmanuel,

Sorry for late reply. Usually we track reply from original poster only so I missed your reply. I've escalated this issue to an engineer who is more familiar with this issue. Please wait for reply.

---

Allen Chen [MSFT]
MSDN Community Support | Feedback to us
Get or Request Code Sample from Microsoft
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 11

I've got the same problem except I'm using VS2010 to package the solution and then I'm manually uploading it to the cloud using the Azure silverlight interface.

Windows Event Logs (WADWindowsEventLogsTable) and Azure logs (WADLogsTable) are being collected locally but not when I go onto Azure. My storage connection string is definately correct as Perf counters are working fine.

I'm on SDK 1.5 but am currently upgrading to 1.6... hope this works

Answer 12

Hi Emmanual

Yeah I don't think the /t:Publish is ever going to work

Check this Stackoverflow post which outlines how to create a CSPack task in MSBuild

http://stackoverflow.com/questions/4731665/using-msbuild-and-cspack-task-to-package-azure-roles

You should then be able to configure cspack, as I outline in my blog post. Hope it helps

Cheers

Iain

Answer 13

I've got the same problem except I'm using VS2010 to package the solution and then I'm manually uploading it to the cloud using the Azure silverlight interface.

Windows Event Logs (WADWindowsEventLogsTable) and Azure logs (WADLogsTable) are being collected locally but not when I go onto Azure. My storage connection string is definately correct as Perf counters are working fine.

I'm on SDK 1.5 but am currently upgrading to 1.6... hope this works

nope... still don't work :-(

Answer 14

There is known issue for Azure SDK 1.4 tools Pack/Unpack during the MSBuild Taking out Pack/UnPack or upgrade to SDK 1.6.

If the problem is persist, please RDP to the VM and check the details. RDP onto the VM and make sure DiagnosticsAgent.exe and MonAgentHost.exe process is  running. If it doesn't run, please give me the RDP file and crendential to check it.

Then you can use ProcMon.exe http://technet.microsoft.com/en-us/sysinternals/bb896645 to check permission issue, if the logs file already collect in the local but dont transfer to Storage, please check below:

 
1. Make sure the storage account is not set to "UseDevelopmentStorage=true".  Validate this by checking the role config file on the Azure VM under C:\Config or your service definition.
 
2. Check the blobs in the wad-control-container blob container in the diagnostics storage account.  You are looking for the blob named with the <deploymentID>/<RoleName>/<RoleInstance>
 
Open the XML file and make sure it is configured the way you expect.  If you see <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes> for a given data source, it means that the diagnostics are not configured to transfer any data.
 
3. If Diagnostics.Trace information is not being collected, make sure the DiagnosticMonitorTraceListener is configured in the web.config or app.config.  This is configured by default in cloud projects, but some  comment it out which will cause the trace statements to not be collected by diagnostics. 

 
4. Do you  query the diagnostics storage correctly? Sometime we will return no results and assuming that diagnostics are not being captured. 
 
 

Answer 15

Hi,

After two days trying to make diagnostics work in my devenv, with SDK 1.6, no luck whatsoever trying all the solutions posted so far.

Now i download and started the Diagnostics Azure sample, started it from VS2010 sp1, and it does not work either (tried in two computers).

The only thing I see in storage is a blob wad-control-container, the xml says:

   <Logs>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </Logs>

How i am suppose to check the logs??

Pls help, we have to put stuff in production and this is delaying everything!

 

The emulator UI displays:

[MonAgentHost] Error: MA EVENT: 2011-12-15T15:00:40.677Z
[MonAgentHost] Error:     2
[MonAgentHost] Error:     7416
[MonAgentHost] Error:     7464
[MonAgentHost] Error:     NetTransport
[MonAgentHost] Error:     0
[MonAgentHost] Error:     x:\btsdx\215\services\monitoring\shared\nettransport\src\netutils.cpp
[MonAgentHost] Error:     OpenHttpSession
[MonAgentHost] Error:     749
[MonAgentHost] Error:     0
[MonAgentHost] Error:     2f94
[MonAgentHost] Error:    
[MonAgentHost] Error:     WinHttpGetProxyForUrl(http://127.0.0.1) failed ERROR_WINHTTP_AUTODETECTION_FAILED (12180)

 

df

 

PD:sooo frustrating all of this, it was working before... there is something called regression testing...

Answer 16

Hi Druida,

 

The statment below mean you don't want to tranfer any data to blob storage.

   <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>

Ycan change it by Storage utility or from code.

Quickly check the file from container WAD-Control-Container under your diagnostic storage account,

My working XML like below:

You can manual change it and time to 1 mins and wait some time, it shall begin to transfer the data.

 

Thanks

leo lin

 

 

<?xml version="1.0"?>
<ConfigRequest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <OnDemandTransfers />
  <DataSources>
    <OverallQuotaInMB>4080</OverallQuotaInMB>
    <Logs>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>1</ScheduledTransferPeriodInMinutes>
      <ScheduledTransferLogLevelFilter>Verbose</ScheduledTransferLogLevelFilter>
    </Logs>
    <DiagnosticInfrastructureLogs>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </DiagnosticInfrastructureLogs>
    <PerformanceCounters>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <Subscriptions />
    </PerformanceCounters>
    <WindowsEventLog>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <Subscriptions />
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </WindowsEventLog>
    <Directories>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <Subscriptions>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\06b22e330e8b430f9b74313ab64fe289.LoggingWebRole.DiagnosticStore\FailedReqLogFiles</Path>
          <Container>wad-iis-failedreqlogfiles</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\06b22e330e8b430f9b74313ab64fe289.LoggingWebRole.DiagnosticStore\LogFiles</Path>
          <Container>wad-iis-logfiles</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\06b22e330e8b430f9b74313ab64fe289.LoggingWebRole.DiagnosticStore\CrashDumps</Path>
          <Container>wad-crash-dumps</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
      </Subscriptions>
    </Directories>
  </DataSources>
  <IsDefault>false</IsDefault>
</ConfigRequest>

Answer 17

Iain,

We used to deploy our Azure web and worker roles through TFS using cspack - but using msbuild is much more elegant and you are less likely to have to make changes across Azure SDK versions.

We have been deploying through MSBuild for months without problems - and the actual creation of the azure package works without problems.

My guess is that in version 1.5 or 1.6 Microsoft introduced a bug or a new setting that is needed when creating the package through msbuild.  Since this thread has gone all over the place, I'm starting a new thread specifically on this issue.