none
Backup to Azure via DPM fails with internal service error, please retry the operation RRS feed

  • Question

  • All online backups are failing with this event: 

    “Creation of online recovery points for *DB* on *SERVER* have failed. The last online recovery point creation failed for the following reason: (ID: 3188)

    The current operation failed due to an internal service error [0x1FBD2]. Please retry the operation after sometime. (ID: 100066)”

    Backups were previously successful; subsequent backups are failing.

    Failure doesn't happen until all the data is transferred/uploaded.

    Any ideas?

    Tuesday, June 30, 2015 11:05 PM

All replies

  • Hello Mike,

    Thanks for posting here!

    There are two things you need to check.

    1) In case you have multiple servers protected by Azure Backup then you must use the same Certificate on all the servers.

    2) Not sure if you have updated this roll up package - https://support.microsoft.com/en-us/kb/2779569

    Let us know the results.


    Best Regards

    Sadiqh Ahmed

    ________________________________________________________________________________________________________________

    If a post answers your question, please click Mark As Answer on that post and Vote as Helpful.

    Wednesday, July 1, 2015 5:07 AM
    Moderator
  • Update Rollup 6 is installed for R2 Data Protection Manager (on Windows Server 2012 R2 Standard).

    I can certainly try regenerating the vault credentials, but how do I apply the updated vault credentials to DPM? Will this process invalidate existing cloud backups?



    Wednesday, July 1, 2015 4:30 PM
  • I pulled the last few warnings/fatal entries from the log after the last failure (CBEngineCurr.errlog):

    WARNING Mapping Invisible ErrorCode 130002 to CBPGenericOperationFailure
    WARNING Mapping Invisible ErrorCode 130002 to CBPGenericOperationFailure

    ...

    FATAL Failed to make web service call | Params: {Exception:  = System.ServiceModel.FaultException`1[Microsoft.Internal.CloudBackup.Common.FailureModeling.CloudServiceFault]: Internal Service Error (Fault Detail is equal to ErrorCode = CloudInvalidInputError, DetailedErrorCode = 0, DetailedErrorSource = None/Client, Message = 
    FATAL ).}
    WARNING FMBlock: FM exception caught | Params: {OperationCode = WcfProxyCall}{OperationStartTime = 7/1/2015 7:31:49 PM}{FMExceptionDetail = FMException: [ErrorCode:CloudInvalidInputError, DetailedCode:0, Source:Service/None/Client, Message:]
    WARNING {InitialTargetEndpoint = https://*SNIP*.svc}{Name = BasicHttpBinding_IROCatalogExternal}{WorkitemId = *SNIP*}{CurrentTargetEndpoint = https://*SNIP*.svc}{RequestId = *SNIP*}{IdentityType = Container}{Identity = *SNIP*}
    WARNING }
    WARNING FMBlock: FM exception caught | Params: {OperationCode = WcfProxyAsyncCall}{OperationStartTime = 7/1/2015 7:31:49 PM}{FMExceptionDetail = FMException: [ErrorCode:CloudInvalidInputError, DetailedCode:0, Source:Service/None/Client, Message:]
    WARNING {InitialTargetEndpoint = https://*SNIP*.svc}{Name = BasicHttpBinding_IROCatalogExternal}{WorkitemId = *SNIP*}{CurrentTargetEndpoint = *SNIP*.svc}{RequestId = *SNIP*}{IdentityType = Container}{Identity = *SNIP*}
    WARNING }
    WARNING Error occured, DlsErrorCode - 537000914, Exception - FMException: [ErrorCode:CloudInvalidInputError, DetailedCode:0, Source:Service/None/Client, Message:]
    WARNING {InitialTargetEndpoint = https://*SNIP*.svc}{Name = BasicHttpBinding_IROCatalogExternal}{WorkitemId = *SNIP*}{CurrentTargetEndpoint = https://*SNIP*.svc}{RequestId = *SNIP*}{IdentityType = Container}{Identity = *SNIP*}
    WARNING Stack trace:   at Microsoft.Internal.Common.FailureModeling.FMComponent`3.HandleUnknownException(FMBlockArgs args, Int32 currentAttempt, DateTime startTime, OperationProperties`3 opProperties, Exception ex)
    WARNING   at Microsoft.Internal.Common.FailureModeling.FMComponent`3.HandleException(FMBlockArgs args, Int32 currentAttempt, DateTime startTime, OperationProperties`3 opProperties, Exception ex)
    WARNING   at Microsoft.Internal.Common.FailureModeling.FMComponent`3._FMBlock(String fileName, Int32 lineNumber, FMBlockArgs args, Action fmPrecheckBlock, Action fmUserBlock)
    WARNING   at Microsoft.Internal.CloudBackup.Common.WcfClient.WcfClient`1.CallWcfServiceWrapper(Guid workitemId, Action internalServiceCall, String webServiceDescription)
    WARNING   at Microsoft.Internal.CloudBackup.Common.WcfClient.WcfClient`1.<>c__DisplayClassd.<CallWcfService>b__c()
    WARNING   at Microsoft.Internal.Common.FailureModeling.FMComponent`3._FMBlock(String fileName, Int32 lineNumber, FMBlockArgs args, Action fmPrecheckBlock, Action fmUserBlock)
    WARNING   at Microsoft.Internal.CloudBackup.Common.WcfClient.WcfClient`1.CallWcfService(Action webServiceCall, String webServiceDescription)
    WARNING   at Microsoft.Internal.CloudBackup.Client.Proxy.CAL.CalExternalServiceProxy.AddRPVolumeReplicaMapInfo(AddRPVolumeReplicaMap[] rpVolumeReplicaInfo)
    WARNING   at Microsoft.Internal.CloudBackup.Client.Proxy.CAL.CalExternalUnmanagedProxy.AddRPVolumeReplicaMapInfo(CalExternalUnmanagedProxy* , HResult* , CAL_ADD_RP_VOLUME_REPLICA_MAP_V2014_09* rgVolumeReplicaMapInfo, UInt32 cVolumeReplicaMapInfo)
    WARNING --->System.ServiceModel.FaultException`1[Microsoft.Internal.CloudBackup.Common.FailureModeling.CloudServiceFault]: Internal Service Error (Fault Detail is equal to ErrorCode = CloudInvalidInputError, DetailedErrorCode = 0, DetailedErrorSource = None/Client, Message = 
    WARNING ).

    I can zip and send the whole log if needed.

    Wednesday, July 1, 2015 10:53 PM
  • Hello Mike,

    Can you please email us the backup job and the agent logs under   (<Product Install Dir>\Windows Azure Online Backup Agent\Temp\CBEngineCurr.errlog and CBUICurr.errlog) to v-saahme @ Microsoft dot com.

    Regards,

    Sadiqh

    Monday, July 6, 2015 5:15 AM
    Moderator