none
Backup failing with 'Internal error' RRS feed

  • Question

  • This is the message I'm getting when trying to backup (on the step: taking snapshot):

    ERROR DETAILS
    Backup failed with an internal error.

    Please retry the operation in a few minutes. If the problem persists, contact Microsoft Support.

    When I check the link I see two possible causes:

    You can get this error for 2 reasons:

    1. There is too much data to be copied.
    2. The original VM has been deleted and therefore backup cannot be taken. In order to keep the backup data for a deleted VM but stop the backup errors, Unprotect the VM and choose the option to keep the data. This will stop the backup schedule and also the recurring error messages.

    First reason: What is 'too much data'?  This is an Ubuntu server and it's size isn't out of this world.  How can I backup this machine?  Is Microsoft actually saying that it isn't capable of taking a backup of certain virtual machines?  This is a Standard DS2 machine, no additional storage added.

    Second reason: although I'm pretty sure that's not the reason since the machine is live whole the time and backups where going fine till the 26th when they suddenly stopped working.  I have completely unprotected and unregistered the server, deleted all backups and started over but I'm getting the exact same message.

    /edit: backup and VM are both located in the same region.

    /edit2: version is: WALinuxAgent-2.0.14 running on Ubuntu

    Thursday, October 29, 2015 8:54 AM

All replies

  • Hi,

    Have you upgraded to the latest version of Azure Linux Agent (waagent) ?

    Please check this link that might help : 
    https://acom-swtest-2.azurewebsites.net/en-us/documentation/articles/virtual-machines-linux-update-agent/

    Also, a similar issue  here that talks about a software firewall which was preventing the waagent from starting up correctly, only needed to disable the software firewall for the initial startup for everything to function correctly.

    https://social.msdn.microsoft.com/Forums/en-US/3f5d1f86-3e16-4e19-af37-63d2341156af/linux-backup-fails-using-recovery-services?forum=windowsazureonlinebackup

    Regards,
    Azam khan

    Thursday, October 29, 2015 1:49 PM
  • I'm running the latest version available for Ubuntu (2.0.14).

    Firewall isn't active.  It was functioning before and stopped all of a sudden.

    When I look in the logs I see that it's getting requests to backup when I ask it to backup through the interface.

    Here is an abstract of the log /var/log/azure/Microsoft.Azure.RecoveryServices.VMSnapshotLinux/1.0.3.0/extension.log

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]Config decoded correctly.

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0] current identity {21aff6cf-fa2a-4744-d7be-ebef810ebd04}

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]set most recent sequence number to 3

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]2015-10-29 13:17:49.858136   Info   starting to enable

    2015/10/29 13:17:49 

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]2015-10-29 13:17:49.858642   Info   commandToExecute is Snapshot

    2015/10/29 13:17:49 

    2015/10/29 13:17:49 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]2015-10-29 13:17:49.859218   Info   doing freeze now...

    2015/10/29 13:17:49

    2015/10/29 13:17:50 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]2015-10-29 13:17:49.837177   Info   out_lsblk_output:

    2015/10/29 13:17:50 sda  disk

    2015/10/29 13:17:50 sda1 part ext4   /

    2015/10/29 13:17:50 sdb  disk

    2015/10/29 13:17:50 sdb1 part ext4   /mnt

    2015/10/29 13:17:50 

    2015/10/29 13:17:50 2015-10-29 13:17:49.858136   Info   starting to enable

    2015/10/29 13:17:50 2015-10-29 13:17:49.858642   Info   commandToExecute is Snapshot

    2015/10/29 13:17:50 2015-10-29 13:17:49.859218   Info   doing freeze now...

    2015/10/29 13:17:50 2015-10-29 13:17:49.859622   Info   freeze.../mnt type part

    2015/10/29 13:17:50 2015-10-29 13:17:49.866320   Info   freeze_result...0

    2015/10/29 13:17:50 2015-10-29 13:17:49.866408   Info   freeze.../ type part

    2015/10/29 13:17:50 2015-10-29 13:17:50.474439   Info   freeze_result...0

    2015/10/29 13:17:50 2015-10-29 13:17:50.474570   Info   freeze result

    2015/10/29 13:17:50 2015-10-29 13:17:50.474593   Info   doing snapshot now...

    2015/10/29 13:17:50 2015-10-29 13:17:50.474622   Info   doing snapshotall now...

    2015/10/29 13:17:50 2015-10-29 13:17:50.474884   Info   {'Content-Length': 0, u'x-ms-meta-CurrentTicks': u'635817214933256176', u'x-ms-meta-BackUpOwner': u'IAASVmBackUp', u'x-ms-meta-BackUpRetryCount

    ': u'0', u'x-ms-meta-BackUpOperationCurrentId': u'd5dbb1ee-e52b-4327-9047-3d5a170262cf-2015-10-29 13:18:03Z', u'x-ms-meta-SnapShotFailureRetryCount': u'0', u'x-ms-meta-BackUpOperationJobId': u'ce71c3

    ee-91f6-4799-ba49-f33e4a4d2831'}

    2015/10/29 13:17:50 2015-10-29 13:17:50.530961   Info   [('x-ms-version', '2012-02-12'), ('x-ms-snapshot', '2015-10-29T13:19:32.3135858Z'), ('transfer-encoding', 'chunked'), ('x-ms-request-id', '45f4

    413b-0001-0039-7d4c-12520c000000'), ('server', 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'), ('last-modified', 'Thu, 29 Oct 2015 13:19:32 GMT'), ('etag', '"0x8D2E06395416372"'), ('date', 'Thu, 29

    Oct 2015 13:19:31 GMT')]

    2015/10/29 13:17:50 2015-10-29 13:17:50.531184   Info   snapshotall ends...

    2015/10/29 13:17:50 2015-10-29 13:17:50.531213   Info   Enable Succeeded

    2015/10/29 13:17:50 2015-10-29 13:17:50.531225   Info   doing unfreeze now...

    2015/10/29 13:17:50 2015-10-29 13:17:50.531252   Info   unfreeze.../mnt type part

    2015/10/29 13:17:50 2015-10-29 13:17:50.534165   Info   unfreeze_result...0

    2015/10/29 13:17:50 2015-10-29 13:17:50.534316   Info   unfreeze.../ type part

    2015/10/29 13:17:50 2015-10-29 13:17:50.536110   Info   unfreeze_result...0

    2015/10/29 13:17:50 2015-10-29 13:17:50.536168   Info   unfreeze result

    2015/10/29 13:17:50 2015-10-29 13:17:50.536193   Info   unfreeze ends...

    2015/10/29 13:17:50 2015-10-29 13:17:50.536215   Info   committing the log

    2015/10/29 13:17:50

    2015/10/29 13:17:50 [Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0]Enable,success,1,Enable SucceededEnable Succeeded with error: []


    Thursday, October 29, 2015 2:34 PM
  • Hi,

    I would suggest you to open a support ticket, here is the link:

    https://azure.microsoft.com/en-in/support/options/

    Regards,
    Azam Khan

    Monday, November 2, 2015 6:01 PM
  • I opened a support ticket, this is the reply I got.  It is a known issue which means currently I have no way of getting backups of the Linux box.

    I went through the logs and the Azure Database and found this Error:

     

    ErrorCode: GuestAgentSnapshotTaskStatusError

     

    When a VM is backed up, the VMAgent inside the guest invokes the backup snapshot extension which in turn performs a snapshot inside the guest.  A status file is created with the results of the snapshot and that is then copied to a blob in azure where a backup service reads it.  There is an API that the agent uses to copy that file to the blob, and the API seems to have problems copying it sometimes.

    The product group is re-architecting the backup service in azure so we no longer rely on the API to copy the status file to the blob and will instead read the log file on the VM to get the status of the snapshot.  This will be a new fix inside an Azure service and there is no customer action required. 

     

    It is a known Issue and the concerned team is working on it. However, there is no ETA on this.

    I will keep you updated with the progress of the case.


    Monday, November 2, 2015 7:37 PM