locked
Python Azure Batch - Permission Denied Linux VM RRS feed

  • Question

  • Hi,

    I am running a python script on several Linux nodes (after the creation of a pool) using Azure Batch. Each node uses 14.04.5-LTS version of Ubuntu.

    In the script, I am uploading several files on each node and then I run several tasks on each one of these node. But, I get a "Permission Denied" error when I try to execute the first task. Actually, the task is an unzip of few files (fyi, the uploading of these zip files went well). 

    This script was running well until last weeks. I suspect an update of Ubuntu version but maybe it's something else. 

    Do you have any insights to help me on this issue? Thank you very much.

    Robin

    Friday, November 24, 2017 11:04 AM

All replies

  • Hiya Robin,

    Is there any specific reason that you are using shared_dir for this,  I have added some suggesstion to use this feature for your scenario.

    With regards to the scenario and if it was working before using (Shared Dir) I am unsure what changed in the Ubuntu image i.e. Ubuntu 14.04.5-LTS (since this scenario working previously). (Someone will pick this up and will defintley comment on this.)

    I think you can also log into the node and find out what exactly happened with in Shared dir level.

    Porting my recommendation further here:

    looking at the error, i.e. 

        error: cannot open zipfile [ /mnt/batch/tasks/shared/01-AXAIS_HPC.zip ]
        Permission denied unzip: cannot find or open /mnt/batch/tasks/shared/01-AXAIS_HPC.zip,

    This seems like that the file is not present at the current shared directory location or it is is not in correct permission. The former is more likely.

    • Is there any particular reason you are using the shared directory way? also, How are you uploading the file? (i.e. hope that the use of async and await is correctly done, i.e. there is not greedy process which is running your task before the shared_dir stuff is available to the node.)
    • side note: you own the node so you can RDP / SSH into the node and find it out that the shared_dir are actually present.
    Possible other ways / recommendation:

    There are few other ways you can use zip files in the azure node, like via resourcefile or via application package. (The applicaiton package way might suite it better to deal with *.zip file) I have added few documetns and places you can have a look at the sample implementation and guidance for this.

    I think a good place to start are: hope material and sample below will help you. `:)`

    Also I would recommend to recreate your pool if it is old which will ensure you have the node running at the latest version.

      [1]: https://azure.microsoft.com/en-us/documentation/learning-paths/batch/
      [2]: https://github.com/Azure/azure-batch-samples
      [3]: https://github.com/Azure/azure-batch-samples/blob/master/CSharp/ArticleProjects/ParallelTasks/Program.cs
      [4]: https://docs.microsoft.com/en-us/azure/batch/batch-application-packages
      [5]: https://docs.microsoft.com/en-us/azure/batch/batch-api-basics

    Thanks,

    ^Tats







    Monday, November 27, 2017 1:07 AM