none
Using Azure CLI to access Storage Account from a cron job

    Question

  • On Ubuntu Linux, I have a bash script that can leverage the Azure CLI to upload archive files to storage account blob containers:

    root@az01pptappt02:~# more /var/tmp/backup.sh 
    #!/bin/bash
    # THIS FILE IS MANAGED BY PUPPET
    # Backup of Puppet Enterprise unique items to designated Azure storage account container
    # Should be run once daily; built-in backup retention of one week
    # Reference: https://docs.puppet.com/pe/latest/maintain_backup_restore.html
    # Reference: https://blog.sleeplessbeastie.eu/2013/03/12/simple-shell-script-to-backup-selected-directories/
    PROXY=http://proxy.mycompany.com:8080
    export HTTP_PROXY=$PROXY
    export HTTPS_PROXY=$PROXY
    export http_proxy=$PROXY
    export https_proxy=$PROXY
    export AZURE_STORAGE_ACCOUNT=myazurestorageaccount
    export AZURE_STORAGE_ACCESS_KEY=myReallyLongAzureStorageAccountAccessKey==
    DEPLOYMENT=puppettest
    DAY=`date +%a |tr A-Z a-z`
    BACKUP_DIR=/opt/pe_backup
    azure storage container delete $DEPLOYMENT-$DAY -q
    # create new container for this deployment and day; create will fail until old container is deleted
    until azure storage container create $DEPLOYMENT-$DAY
    do
      sleep 5s
    done
    if [ ! -d $BACKUP_DIR ]; then
      mkdir -p $BACKUP_DIR
    fi
    ARCHIVE_NAME=`echo /etc/puppetlabs | sed s/^\\\/// | sed s/\\\//_/g`
    tar pcfzP ${BACKUP_DIR}/${ARCHIVE_NAME}.tgz /etc/puppetlabs 2>&1 | tee > ${BACKUP_DIR}/${ARCHIVE_NAME}.log
    azure storage blob upload -f $BACKUP_DIR/$ARCHIVE_NAME.tgz --container $DEPLOYMENT-$DAY -b $ARCHIVE_NAME.tgz -q
    ARCHIVE_NAME=`echo /etc/puppetlabs/puppet/ssl | sed s/^\\\/// | sed s/\\\//_/g`
    tar pcfzP ${BACKUP_DIR}/${ARCHIVE_NAME}.tgz /etc/puppetlabs/puppet/ssl 2>&1 | tee > ${BACKUP_DIR}/${ARCHIVE_NAME}.log
    azure storage blob upload -f $BACKUP_DIR/$ARCHIVE_NAME.tgz --container $DEPLOYMENT-$DAY -b $ARCHIVE_NAME.tgz -q
    ARCHIVE_NAME=`echo /opt/puppetlabs/server/data/console-services/certs | sed s/^\\\/// | sed s/\\\//_/g`
    tar pcfzP ${BACKUP_DIR}/${ARCHIVE_NAME}.tgz /opt/puppetlabs/server/data/console-services/certs 2>&1 | tee > ${BACKUP_DIR}/${ARCHIVE_NAME}.log
    azure storage blob upload -f $BACKUP_DIR/$ARCHIVE_NAME.tgz --container $DEPLOYMENT-$DAY -b $ARCHIVE_NAME.tgz -q
    ARCHIVE_NAME=`echo /opt/puppetlabs/server/data/postgresql/9.4/data/certs | sed s/^\\\/// | sed s/\\\//_/g`
    tar pcfzP ${BACKUP_DIR}/${ARCHIVE_NAME}.tgz /opt/puppetlabs/server/data/postgresql/9.4/data/certs 2>&1 | tee > ${BACKUP_DIR}/${ARCHIVE_NAME}.log
    azure storage blob upload -f $BACKUP_DIR/$ARCHIVE_NAME.tgz --container $DEPLOYMENT-$DAY -b $ARCHIVE_NAME.tgz -q
    azure storage blob upload -f /opt/db_backup/pe-activity.bin --container $DEPLOYMENT-$DAY -b pe-activity.bin -q
    azure storage blob upload -f /opt/db_backup/pe-classifier.bin --container $DEPLOYMENT-$DAY -b pe-classifier.bin -q
    azure storage blob upload -f /opt/db_backup/pe-puppetdb.bin --container $DEPLOYMENT-$DAY -b pe-puppetdb.bin -q
    azure storage blob upload -f /opt/db_backup/pe-rbac.bin --container $DEPLOYMENT-$DAY -b pe-rbac.bin -q
    azure storage blob upload -f /opt/db_backup/pe-orchestrator.bin --container $DEPLOYMENT-$DAY -b pe-orchestrator.bin -q

    This works great from an interactive shell as root.  However, when trying to run it in a root cron job, I get:

    /var/tmp/backup.sh: line 19: azure: command not found

    I'm currently trying to figure out how to present the node.js environment to the cron job.  The cron entry for this job is something like:

    30 3 * * * . /root/.bashrc; /var/tmp/backup.sh > /var/tmp/backup.log 2>&1

    I'm thinking I need to create a supplemental Azure CLI-specific .bashrc that includes the variables needed to path in the node.js environment, like what's documented here: http://stackoverflow.com/questions/22478277/executing-azure-cli-from-shell-script-as-cron-job.

    This seems like a common need - has Microsoft or anyone else documented this?

    Thanks.

    <style type="text/css">p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 15.0px Consolas; color: #f4f4f4; background-color: #000000} span.s1 {font-variant-ligatures: no-common-ligatures} </style><style type="text/css">p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 15.0px Consolas; color: #f4f4f4; background-color: #000000} span.s1 {font-variant-ligatures: no-common-ligatures} </style><style type="text/css">p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 15.0px Consolas; color: #f4f4f4; background-color: #000000} span.s1 {font-variant-ligatures: no-common-ligatures} </style>

    Friday, January 13, 2017 3:31 PM

All replies

  • Hi Alex,

    Thank you for contacting Microsoft forums. We are pleased to answer your query….

    This would happen when the PATH is missing /usr/bin where node and azure apps are installed. I am not sure why the cronjob results in failure of PATH variable, however you may simply include the following line in the script and check it. No need of a bashrc.

    1. The best way to ensure your script can find the azure command is to hard-code the full path to the place where it was installed. I don’t know if that’s /usr/local or /var/azure or whatever; simply use the full path.
    2. The cron program, by design, starts all cron jobs with a minimal PATH which just points to standard system directories. You can change the default PATH by setting it in the crontab itself:
      PATH=/bin:/usr/sbin:/usr/bin:/usr/local/bin:/var/omi/bin:/whatever/you/need
      30 3 * * * . /root/.bashrc; /var/tmp/backup.sh > /var/tmp/backup.log 2>&1
    3. As Sercan points out, you can change PATH at the top of your backup.sh script.
      export PATH=$PATH:/additional/path/you/need:/another/path/your/script/needs
      will just add the additional directories to the end of the search path.

    I hope that the reply will assist you in getting your query addressed. In case you require further assistance, please do reply to the thread as we are always available to your queries.


    Regards,
    Sumanth BM
    ---------------------------------------------------------------------------------------------------------
    Kindly click "Mark as Answer" on the post that helps you, this can be beneficial to other community members reading the thread. And Vote as Helpful.

    Saturday, January 14, 2017 12:01 PM
    Moderator