none
Run tensorflow serving in an ACI with the docker run command RRS feed

  • Question

  • normally run my tensorflow serving image with docker using this command: docker run -p 8500:8500 \ --mount type=bind,source=/mnt/docker/models,target=/models \ --mount type=bind,source=/mnt/docker/configs/models.config,target=/models/models.config \ -t tensorflow/serving \ --model_config_file=/models/models.config &sleep 2m

    and want to deploy the same tensorflow/serving image from docker hub on an Azure Container Instance using az create and run it with the same command line argument as provided above.

    I've given it multiple tries and been getting several errors e.g. the following: run: 1: run: docker: not found

    What is the correct way of doing this?

    Thursday, August 29, 2019 12:50 PM

All replies

  • Hi,

    Lets take  the arguments to the "docker run" command one by one and convert it into azure cli arguments.

    Ports:

    In ACI, port mapping is not there. We need to open the ports from the container.  Since your source and destinations ports are same, you dont have any issues there.

    In azure cli ports flag will look like this  "--ports 8500". If you have multiple ports then give a comma separated list like this "80,8500" 

    Volume mounts:

    ACI supports multiple volume options. Out of them 2 options fits our scenario. They are azure file share and gitrepo.

    I am seeing that you are attaching the config files from your local machine to the container. 

    Github:

    If those config files are stored in github, you can attached the whole github repo as volume to the container.

    Below flags will help you to achieve that.

    Git Repo Volume Arguments
        --gitrepo-dir                    : The target directory path in the git repository. Must not
                                           contain '..'.  Default: ..
        --gitrepo-mount-path             : The path within the container where the git repo volume
                                           should be mounted. Must not contain colon ':'.
        --gitrepo-revision               : The commit hash for the specified revision.
        --gitrepo-url                    : The URL of a git repository to be mounted as a volume.

    Azure file share:

    Here you need to create a storage account and then a file share in it. Then you can load your config files into the file share(from portal or using azcopy or using storage explorer).

    Then you can use the below flags to mount that file share  to the container

    Azure File Volume Arguments
        --azure-file-volume-account-key  : The storage account access key used to access the Azure File
                                           share.
        --azure-file-volume-account-name : The name of the storage account that contains the Azure File
                                           share.
        --azure-file-volume-mount-path   : The path within the container where the azure file volume
                                           should be mounted. Must not contain colon ':'.
        --azure-file-volume-share-name   : The name of the Azure File share to be mounted as a volume

    From cli you can attach only one file share to the container. You can use the yaml file to create a ACI if you want to attach multiple file shares.

    image: 

    for image use --image flag.

    Starting command:

    For starting command use --command-line flag. You can also use the yaml way to create,

    Try this out and let me know if you face issues.

    Also provide the full azure cli command which you tried .

    Note:

    You can export the running ACI into yaml and then make changes in the yaml and then you can update it.

    Friday, August 30, 2019 8:32 AM
    Moderator
  • Hi,

    Did you got a chance to look at the above comment

    Tuesday, September 3, 2019 12:31 PM
    Moderator
  • Hi Jakurna, I somehow missed the alert to your first comment, will be looking at it tomorrow.
    Tuesday, September 3, 2019 5:01 PM
  • Sure.  
    Feel free to post here if you face any issues
    Thursday, September 5, 2019 4:27 AM
    Moderator
  • Hello,

    Do you have any update on the issue?

    Just checking in to see if you got a chance to try the suggestion provided.

    If the suggested answer has solved your query, do click on "Mark as Answer" and “Vote as Helpful” on the post that helps you, this can be beneficial to other community members.

    Thanks.

    Wednesday, September 11, 2019 9:47 AM
    Moderator