locked
Batch App Service: An error occurred setting up for task 1 for job RRS feed

  • Question

  • we're running into an issue with a job that's in out batch app service.

    When we submit the job, we see it split, then we get this message 3 times in the job logs before the job stops:

    "An error occurred setting up for task 1 for job f46a81fe-7fa0-4ffa-9174-c2c05c07d7c5"

    I've remoted into the vm we provision for the job and found this in the stdout.txt for the task we produced for the job:

    TaskProcessorHost.Main Starting...
    Failed to load GreenButton.Cloud.SDK into secondary app domain.  Assemblies loaded into secondary app domain:
    mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
    GreenButton.Proxies, Version=2.0.25450.72, Culture=neutral, PublicKeyToken=null
    System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

       at GreenButton.Proxies.TaskProcessProxyServerFactory.EnsureSdkLoaded() in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Proxies\TaskProcessProxyServerFactory.cs:line 125
       at GreenButton.Proxies.TaskProcessProxyServerFactory.CopySdkGlobals(Object globalLogger, Object globalTaskLogger) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Proxies\TaskProcessProxyServerFactory.cs:line 76
       at GreenButton.Proxies.TaskProcessProxyServerFactory.Initialise(CloudAssemblyPackage assemblies, Object globalLogger, Object globalTaskLogger, IConstructorArgumentsProvider constructorArgumentProvider)
       at GreenButton.Cloud.Framework.Proxies.AssemblyProxy..ctor(String name, CloudAssemblyPackage assemblies, IConfigurationSerialiser configurationSerialiser, IConstructorArgumentsProvider constructorArgumentsProvider) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Cloud.Framework\Proxies\AssemblyProxy.cs:line 44
       at GreenButton.Cloud.Framework.Loader.LoadedCloudAssemblyPackage.LoadNonSpecial(CloudAssemblyPackage package, IContainer autofacContainer, String user) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Cloud.Framework\Loader\LoadedCloudAssemblyPackage.cs:line 23
       at GreenButton.Cloud.Framework.Loader.Sdk1Registrar.RegisterTaskProcessors(CloudAssemblyPackage package, IContainer container, ContainerBuilder builder, TimeSpan cancellationResolution, SpecialAssemblies specialAssemblyHandling, Logs logs) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Cloud.Framework\Loader\Sdk1Registrar.cs:line 32
       at GreenButton.Cloud.Framework.Loader.Sdk1CloudAssembly.RegisterTaskProcessors(IContainer container, ContainerBuilder builder, TimeSpan cancellationResolution) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Cloud.Framework\Loader\Sdk1CloudAssembly.cs:line 31
       at GreenButton.Cloud.Framework.Loader.TaskProcessorDeploymentModule.Load(ContainerBuilder builder) in e:\BuildAgent3\work\DeployRegionAsSthEast\GreenButton.Cloud.Framework\Loader\TaskProcessorDeploymentModule.cs:line 60
       at Autofac.Module.Configure(IComponentRegistry componentRegistry)
       at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
       at GreenButton.Framework.TaskProcessorHost.CommandLine.TaskProcessorHostRole.LoadTaskProcessorsFromStorage() in e:\BuildAgent3\work\DeployRegionAsSthEast\TaskProcessorHost\TaskProcessorHost.CommandLine\TaskProcessorHostRole.cs:line 168
       at GreenButton.Framework.TaskProcessorHost.CommandLine.TaskProcessorHostRole.RunForTask(String user, Guid jobId, Int32 taskId) in e:\BuildAgent3\work\DeployRegionAsSthEast\TaskProcessorHost\TaskProcessorHost.CommandLine\TaskProcessorHostRole.cs:line 114
       at GreenButton.Framework.TaskProcessorHost.CommandLine.Program.Execute(String user, Guid jobId, Int32 taskId, Module deploymentModule, Logs logs) in e:\BuildAgent3\work\DeployRegionAsSthEast\TaskProcessorHost\TaskProcessorHost.CommandLine\Program.cs:line 125



    Usage: TaskProcessorHost.exe <ServiceName> <User> <JobId> <TaskId> <LocalStoragePath>

    12/1/2015 10:23:25 PM Shutting down.  Exit code 2
    12/1/2015 10:25:55 PM Shutdown Complete
    TaskProcessorHost.exe.exe completed with exit code 2

     At this point, we have no idea what's wrong. Has anyone seen this before?

    Thanks,

    Matt

    Tuesday, December 1, 2015 10:39 PM

Answers

  • The error turned out to be caused by an ApplicationImage being uploaded directly to the batchApps storage account, and erroneously placed into the container used for cloudAssemblies (greenbutton-cloud-assemblies) instead of the container used for ApplicationImages (clouddrives).

    The BatchApps service then attempted to load the ApplicationImage zip as a cloudAssembly, and failed as the ApplicationImage zip did not contain a version of Microsoft.Azure.Batch.Apps.Cloud.dll which matched a compatible version of the GreenButton.Cloud.SDK.dll on the service. 

    We are looking into clarifying the logging in this case.

    Cheers,

    David

    • Marked as answer by blueawning Friday, December 4, 2015 4:19 PM
    Friday, December 4, 2015 2:05 AM

All replies

  • Some of the details for the latest error in our batch apps service

    Service:

    pcdsbatchbidoptimizedev

    JobId: 

    f46a81fe-7fa0-4ffa-9174-c2c05c07d7c5

    Tuesday, December 1, 2015 10:43 PM
  • The error turned out to be caused by an ApplicationImage being uploaded directly to the batchApps storage account, and erroneously placed into the container used for cloudAssemblies (greenbutton-cloud-assemblies) instead of the container used for ApplicationImages (clouddrives).

    The BatchApps service then attempted to load the ApplicationImage zip as a cloudAssembly, and failed as the ApplicationImage zip did not contain a version of Microsoft.Azure.Batch.Apps.Cloud.dll which matched a compatible version of the GreenButton.Cloud.SDK.dll on the service. 

    We are looking into clarifying the logging in this case.

    Cheers,

    David

    • Marked as answer by blueawning Friday, December 4, 2015 4:19 PM
    Friday, December 4, 2015 2:05 AM