locked
Method not found: Microsoft.WindowsAzure.Storage.Table.TableOperation RRS feed

  • Question

  • Hi there guys,

    I am currently using the azure data factory with a custom c# pipeline for the azure blob, which uses an azure batch service to run the c# code in a scheduled manner.

    My C# project uses the azure table store to retrieve an entity from the table store for use within the C# script, as such it uses the Microsoft.WindowsAzure.Storage.Table class. Upon attempting to use the retrieve method I get an error, as seen below:

    Error in Activity: Unknown error in module: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.

    ---> System.MissingMethodException: Method not found: 'Microsoft.WindowsAzure.Storage.Table.TableOperation Microsoft.WindowsAzure.Storage.Table.TableOperation.Retrieve(System.String, System.String, System.Collections.Generic.List`1<System.String>)'.

    at TableValidationAlphaTest.GetDataFromTable.Execute(IEnumerable`1 linkedServices, IEnumerable`1 datasets, Activity activity, IActivityLogger logger)

    at Microsoft.Azure.Management.DataFactories.Runtime.ActivityExecutor.Execute(Object job, String configuration, Action`1 logAction)

    --- End of inner exception stack trace ---

    I have used both the most recent version of WindowsAzure.storage and also downgraded back to version 7 where prior issues similar to this were resolved, all to no avail.

    I have also attempted to use alternative methods in the same tableoperation class, and these have been deployed and ran successfully. Using an insert entity into my table store on my batch account works perfectly fine, even though it's a method that also within the same class as the retrieve function, but when I attempt to use the retrieve method from the tableoperation class I run into the error above.

    Any help would be much appreciated!

    Tuesday, October 24, 2017 9:24 AM

All replies

  • In general, Batch does not preinstall SDKs.   You will probably get a more ADF-aware response if you asked your question in the ADF forum.

    Batch provides several features for getting files (dlls/assemblies) or installs done.  Please note that getting an assembly on the VM does not mean it is on the path.  Indeed, this could be your problem now.

    Getting files onto a VM:

        ResourceFiles:  Specified on each task.

        ApplicationPackages: Specified on the pool or the task.

    "Installing":

        StartTask:  set on the pool 

        JobPreperationTask:  set on the job

    It might be good to ask your question on the ADF forum and discover which assemblies need to be on the VM and if they must be installed or "copied"... then return here and we can give you more specific recommendations for your exact use case.

    d

    Tuesday, October 24, 2017 7:47 PM
  • Hi Matt,

    It looks to me like for some reason the wrong version of the Azure Storage dll is being put on the nodes you're running on. Then when you try to invoke "Retrieve" it doesn't work because that method doesn't exist in the version on the node (even though it does exist when you compile).

    According to this: https://github.com/Azure/azure-storage-net/blob/master/changelog.txt it looks like some overloads of retrieve were removed in version 7.  You might try version 6 and see if that works (as a workaround).

    This issue likely has nothing to do with Azure Batch -- I am not sure how ADF is getting your dependencies onto the node, but my first suggestion would be to either RDP into the node or write a code snippet that confirms what version of the Azure Storage assembly is on the node.

    If the above confirms that the version of the storage assembly isn't what you expect, I suggest posting this question on the ADF forums here: https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=AzureDataFactory

    Hope this helps,

    -Matt

    Tuesday, October 24, 2017 7:55 PM