ASA5516 9.8(2) IKEv2 negotiation aborted due unsupported failover version

Question

I have a site to site connection from the ASA to an Azure subscription. The site to site session starts up fine, but after a few minutes (from 3 to 25) the connection fails. If on ASDM I open Monitoring > VPN > VPN Statistics > Sessions, the session is still there, but no communication (e.g. ICMP, RDP, ..) can be performed. If I logout the session, the communication is reestablished, until the next failure a few minutes later.

Every time the connection fails, I observe this warning on the syslog:

4 Sep 18 2018 17:40:58 750003 Local:80.x.y.z:500 Remote:51.a.b.c:500 Username:51.a.b.c IKEv2 Negotiation aborted due to ERROR: Detected unsupported failover version

This is the configuration I have used to setup the site to site connection on the router:

object network HQ-LAN
 subnet 10.0.0.0 255.0.0.0
 description The HQ local network address space on premise
object network Azure-UKSouth-LAN
 subnet 172.16.0.0 255.255.0.0
 description Azure virtual network address space on UKSouth Azure
object-group network AzureLabNet-network
 description Azure AzureLabNet Virtual Network
 network-object object AzureLabNet-LAN
object-group network HQ-network
 description HQ on-premises Network
 network-object object HQ-LAN
object-group network HQ-UKSouth-network
 description The HQ Azure UK South network
 network-object object HQ-UKSouth-LAN
access-list uksouth-s2s-acl extended permit ip object-group HQ-network object-group HQ-UKSouth-network 
nat (LAN,INTERNET) source static HQ-network HQ-network destination static HQ-UKSouth-network HQ-UKSouth-network no-proxy-arp route-lookup
!
crypto ipsec ikev2 ipsec-proposal UKSouth
 protocol esp encryption aes-gcm
 protocol esp integrity null

crypto ipsec security-association lifetime seconds 3600
crypto ipsec security-association lifetime kilobytes 102400000
crypto ipsec security-association pmtu-aging infinite
crypto ipsec inner-routing-lookup

crypto map CRYPTO-MAP 2 match address uksouth-s2s-acl
crypto map CRYPTO-MAP 2 set peer 51.a.b.c 
crypto map CRYPTO-MAP 2 set ikev2 ipsec-proposal UKSouth
crypto map CRYPTO-MAP 2 set ikev2 pre-shared-key *****
crypto map CRYPTO-MAP 2 set security-association lifetime seconds 3600
crypto map CRYPTO-MAP 2 set nat-t-disable
crypto map CRYPTO-MAP interface INTERNET

crypto ca trustpool policy

crypto ikev2 policy 2
 encryption aes
 integrity sha256
 group 19
 prf sha256
 lifetime seconds 28800
crypto ikev2 enable INTERNET
group-policy AzureGroupPolicy-UKSouth internal
group-policy AzureGroupPolicy-UKSouth attributes
 vpn-tunnel-protocol ikev2 

dynamic-access-policy-record DfltAccessPolicy
tunnel-group 51.a.b.c type ipsec-l2l
tunnel-group 51.a.b.c general-attributes
 default-group-policy AzureGroupPolicy-UKSouth
tunnel-group 51.a.b.c ipsec-attributes
 ikev2 remote-authentication pre-shared-key *****
 ikev2 local-authentication pre-shared-key *****
no tunnel-group-map enable peer-ip
tunnel-group-map default-group 51.a.b.c

sysopt connection tcpmss 1350
sysopt connection preserve-vpn-flows

From the Azure side, the site to site connection uses custom policies:

$RG = "RG"
$ConnectionName = "UKSouth-HQ-S2S"

$connection = Get-AzureRmVirtualNetworkGatewayConnection -Name $ConnectionName -ResourceGroupName $RG
$ipsecpolicy = New-AzureRmIpsecPolicy -IkeEncryption AES128 -IkeIntegrity SHA256 -DhGroup ECP256 -IpsecEncryption GCMAES128 -IpsecIntegrity GCMAES128 -PfsGroup ECP256 -SALifeTimeSeconds 3600 -SADataSizeKilobytes 2048

Set-AzureRmVirtualNetworkGatewayConnection -VirtualNetworkGatewayConnection $connection -UsePolicyBasedTrafficSelectors $True -IpsecPolicies $ipsecpolicy

Any suggestion on how to prevent this communication failure?

UPDATE

I am not sure if this is meaningful, but after the connection fails, but the session is still up, "pkts decaps" doesn't increase anymore, but "pkts encaps" keeps increasing:

gateway# show crypto ipsec sa
interface: INTERNET
    Crypto map tag: CRYPTO-MAP, seq num: 1, local addr: 80.x.y.z

      access-list uksouth-s2s-acl extended permit ip 10.0.0.0 255.0.0.0 172.16.0.0 255.255.0.0
      local ident (addr/mask/prot/port): (10.0.0.0/255.0.0.0/0/0)
      remote ident (addr/mask/prot/port): (172.16.0.0/255.255.0.0/0/0)
      current_peer: 51.a.b.c


      #pkts encaps: 4549, #pkts encrypt: 4549, #pkts digest: 4549
      #pkts decaps: 4953, #pkts decrypt: 4953, #pkts verify: 4953
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 4549, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #TFC rcvd: 0, #TFC sent: 0
      #Valid ICMP Errors rcvd: 0, #Invalid ICMP Errors rcvd: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: 80.x.y.z/500, remote crypto endpt.: 51.a.b.c/500
      path mtu 1494, ipsec overhead 55(36), media mtu 1500
      PMTU time remaining (sec): 0, DF policy: copy-df
      ICMP error validation: disabled, TFC packets: disabled
      current outbound spi: 73EF6FC2
      current inbound spi : 766AF315

    inbound esp sas:
      spi: 0x766AF315 (1986720533)
         SA State: active
         transform: esp-aes-gcm esp-null-hmac no compression
         in use settings ={L2L, Tunnel, IKEv2, }
         slot: 0, conn_id: 29892608, crypto-map: CRYPTO-MAP
         sa timing: remaining key lifetime (kB/sec): (92159057/2555)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0xFFFFFFFF 0xFFFFFFFF
    outbound esp sas:
      spi: 0x73EF6FC2 (1945071554)
         SA State: active
         transform: esp-aes-gcm esp-null-hmac no compression
         in use settings ={L2L, Tunnel, IKEv2, }
         slot: 0, conn_id: 29892608, crypto-map: CRYPTO-MAP
         sa timing: remaining key lifetime (kB/sec): (93181492/2555)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

UPDATE #2

While debugging, I have noticed that once the first IKE negotiations completes successfully, the last line on the debug is referring to a peer message ID: 0x1:

debug crypto ipsec 255
debug crypto isakmp 255
debug crypto ikev2 protocol 255
debug crypto ikev2 platform 255
..
IKEv2-PROTO-5: (59): Deleting negotiation context for peer message ID: 0x1

The debug output goes silent afterwards, until the connection fails.
At that point, I observe a number of sequential peer message IDs (0x2, 0x3, 0x4, ..) and their deletion until I don't force the session to logout.

I am not sure if those peer message IDs are the cause (perhaps Azure or the ASA only support a single peer message IDs per security association?) or an effect of the issue.

The peer message ID 0x2 looks like this:

Sep 19 17:35:29 [IKEv1]IKE Receiver: Packet received on 80.x.y.z:500 from 51.a.b.c:500

IKEv2 Recv RAW packet dump
..
IKEv2-PLAT-3: RECV PKT [CREATE_CHILD_SA] [51.a.b.c]:500->[80.x.y.z]:500 InitSPI=0x145521242b6265d7 RespSPI=0x0c25d68e92bfcdc2 MID=00000002
IKEv2-PROTO-5: (59): Request has mess_id 2; expected 2 through 2

(59):
IKEv2-PROTO-2: (59): Received Packet [From 51.a.b.c:500/To 80.x.y.z:500/VRF i0:f0]
(59): Initiator SPI : 145521242B6265D7 - Responder SPI : 0C25D68E92BFCDC2 Message id: 2
(59): IKEv2 CREATE_CHILD_SA Exchange REQUESTIKEv2-PROTO-3: (59): Next payload: ENCR, version: 2.0 (59): Exchange type: CREATE_CHILD_SA, flags: INITIATOR (59): Message id: 2, length: 304(59):
Payload contents:
IKEv2-PROTO-1: decrypt queued(59):
(59): Decrypted packet:(59): Data: 304 bytes
IKEv2-PROTO-1: Asynchronous request queued
IKEv2-PROTO-1:
IKEv2-PLAT-2: (59): Decrypt success status returned via ipc 1
(59): REAL Decrypted packet:(59): Data: 228 bytes
IKEv2-PROTO-5: Parse Notify Payload: REKEY_SA NOTIFY(REKEY_SA)  Next payload: SA, reserved: 0x0, length: 12
    Security protocol id: ESP, spi size: 4, type: REKEY_SA
 SA  Next payload: N, reserved: 0x0, length: 44
  last proposal: 0x0, reserved: 0x0, length: 40
  Proposal: 1, Protocol id: ESP, SPI size: 4, #trans: 3    last transform: 0x3, reserved: 0x0: length: 12
    type: 1, reserved: 0x0, id: AES-GCM
    last transform: 0x3, reserved: 0x0: length: 8
    type: 4, reserved: 0x0, id: DH_GROUP_256_ECP/Group 19
    last transform: 0x0, reserved: 0x0: length: 8
    type: 5, reserved: 0x0, id: Don't use ESN
 N  Next payload: TSi, reserved: 0x0, length: 52

     f6 b6 68 e9 13 e6 a5 8e 04 c4 73 d1 6e 74 c8 35
     2c e9 fd 92 a0 b2 07 6a 33 ce 35 d0 88 b8 4f 42
     85 1b 4a 69 cd 1c 0f cd 9d 52 1c c3 77 f2 a0 57
 TSi  Next payload: TSr, reserved: 0x0, length: 24
    Num of TSs: 1, reserved 0x0, reserved 0x0
    TS type: TS_IPV4_ADDR_RANGE, proto id: 0, length: 16
    start port: 0, end port: 65535
    start addr: 172.16.0.0, end addr: 172.16.255.255
 TSr  Next payload: KE, reserved: 0x0, length: 24
    Num of TSs: 1, reserved 0x0, reserved 0x0
    TS type: TS_IPV4_ADDR_RANGE, proto id: 0, length: 16
    start port: 0, end port: 65535
    start addr: 10.0.0.0, end addr: 10.255.255.255
 KE  Next payload: NONE, reserved: 0x0, length: 72
    DH group: 19, Reserved: 0x0

     f9 63 b6 54 8e bd d7 92 b5 76 7e 5a 7d 3d ab 7c
     22 14 05 76 da 72 a1 ef 9e be b0 30 06 1e dc cb
     9e bc a4 c5 49 c4 05 0d db 0a 09 3e 95 6e c8 b8
     ee 3e 22 25 12 13 72 0f d9 13 b4 8d 65 06 a0 70

IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: READY Event: EV_RECV_CREATE_CHILD
IKEv2-PROTO-5: (59): Action: Action_Null
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_INIT Event: EV_RECV_CREATE_CHILD
IKEv2-PROTO-5: (59): Action: Action_Null
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_INIT Event: EV_VERIFY_MSG
IKEv2-PROTO-2: (59): Validating create child message
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_INIT Event: EV_CHK_CC_TYPE
IKEv2-PROTO-2: (59): Check for create child response message type
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_REKEY_IPSECSA
IKEv2-PROTO-2: (59): Beginning IPSec Rekey as Responder
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_PROC_MSG
IKEv2-PROTO-2: (59): Processing CREATE_CHILD_SA exchange
IKEv2-PLAT-2: Selector received from peer is accepted
IKEv2-PLAT-2: (59): PROXY MATCH on crypto map CRYPTO-MAP seq 1
IKEv2-PROTO-2: (59):
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_NO_EVENT
IPSEC: Received a PFKey message from IKE
IPSEC: Parsing PFKey GETSPI message
IPSEC: Creating IPsec SA
IPSEC: Getting the inbound SPI
IPSEC DEBUG: Inbound SA (SPI 0x00000000) state change from inactive to embryonic
IPSEC: New embryonic SA created @ 0x00007f5357431640,
    SCB: 0x55A268C0,
    Direction: inbound
    SPI      : 0xE3E2B0FD
    Session ID: 0x09DAD000
    VPIF num  : 0x00000007
    Tunnel type: l2l
    Protocol   : esp
    Lifetime   : 240 seconds
IKEv2-PLAT-2: Received PFKEY SPI callback for SPI 0xE3E2B0FD, error FALSE
IKEv2-PLAT-2:
IKEv2 received all requested SPIs from CTM to respond to a tunnel request.
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_OK_RECD_IPSEC_RESP
IKEv2-PROTO-5: (59): Action: Action_Null
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_PROC_MSG
IKEv2-PROTO-2: (59): Processing CREATE_CHILD_SA exchange
IKEv2-PROTO-1: (59): The peer's KE payload contained the wrong DH group
IKEv2-PROTO-1: (59):
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_INV_KE
IKEv2-PROTO-2: (59): Sending invalid ke notification, peer sent group 19, local policy prefers group 0
IKEv2-PROTO-5: Construct Notify Payload: INVALID_KE_PAYLOADIKEv2-PROTO-2: (59): Building packet for encryption.
(59):
Payload contents:
(59):  NOTIFY(INVALID_KE_PAYLOAD)(59):   Next payload: NONE, reserved: 0x0, length: 10
(59):     Security protocol id: ESP, spi size: 0, type: INVALID_KE_PAYLOAD
(59):
(59):      00 00
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_ENCRYPT_MSG
IKEv2-PROTO-2: (59):
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_NO_EVENT
IKEv2-PLAT-2: (59): Encrypt success status returned via ipc 1
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_OK_ENCRYPT_RESP
IKEv2-PROTO-5: (59): Action: Action_Null
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_IPSEC Event: EV_TRYSEND
(59):
IKEv2-PROTO-2: (59): Sending Packet [To 51.a.b.c:500/From 80.x.y.z:500/VRF i0:f0]
(59): Initiator SPI : 145521242B6265D7 - Responder SPI : 0C25D68E92BFCDC2 Message id: 2
(59): IKEv2 CREATE_CHILD_SA Exchange RESPONSEIKEv2-PROTO-3: (59): Next payload: ENCR, version: 2.0 (59): Exchange type: CREATE_CHILD_SA, flags: RESPONDER MSG-RESPONSE (59): Message id: 2, length: 80(59):
Payload contents:
(59):  ENCR(59):   Next payload: NOTIFY, reserved: 0x0, length: 52
(59): Encrypted data: 48 bytes
(59):
IKEv2-PLAT-3: (59): SENT PKT [CREATE_CHILD_SA] [80.x.y.z]:500->[51.a.b.c]:500 InitSPI=0x145521242b6265d7 RespSPI=0x0c25d68e92bfcdc2 MID=00000002
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: CHILD_R_DONE Event: EV_FAIL
IKEv2-PROTO-1: (59): Create child exchange failed
IKEv2-PROTO-1: (59):
IKEv2-PROTO-2: (59): IPSec SA create failed
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: EXIT Event: EV_ABORT
IKEv2-PROTO-5: (59): Sent response with message id 2, Requests can be accepted from range 3 to 3
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: EXIT Event: EV_CHK_PENDING_ABORT
IKEv2-PROTO-5: (59): SM Trace-> SA: I_SPI=145521242B6265D7 R_SPI=0C25D68E92BFCDC2 (R) MsgID = 00000002 CurState: EXIT Event: EV_UPDATE_CAC_STATS
IKEv2-PROTO-2: (59): Abort exchange
IKEv2-PLAT-1: (59): Child SA rekey failure reported for SPI 0x918F75A6.
IKEv2-PROTO-5: (59): Deleting negotiation context for peer message ID: 0x2
IPSEC: Received a PFKey message from IKE
IPSEC DEBUG: Received a DELETE PFKey message from IKE for an inbound SA (SPI 0xE3E2B0FD)
IKEv2-PLAT-1: Failed to remove peer correlation entry from cikePeerCorrTable.  Local Type = 0.  Local Address = 0.0.0.0.  Remote Type = 0.  Remote Address = 0.0.0.0.  Correlation Peer Index = 0. IPSEC Tunnel Index = 0.
IPSEC DEBUG: Migrated SA is deleted, Deleting the Backup SPI entry 0xE3E2B0FD
IPSEC DEBUG: Inbound SA (SPI 0xE3E2B0FD) destroy started, state embryonic
IPSEC: Destroy current inbound SPI: 0xE3E2B0FD
IPSEC DEBUG: Inbound SA (SPI 0xE3E2B0FD) free started, state embryonic
IPSEC DEBUG: Inbound SA (SPI 0xE3E2B0FD) state change from embryonic to dead
IPSEC DEBUG: Inbound SA (SPI 0xE3E2B0FD) free completed
IPSEC DEBUG: Inbound SA (SPI 0xE3E2B0FD) destroy completed
IKEv2-PLAT-2: Received PFKEY delete SA for SPI 0xE3E2B0FD error FALSE

This is followed by seemingly another peer message ID 0x2:

Sep 19 17:42:26 [IKEv1]IKE Receiver: Packet received on 80.x.y.z:500 from 51.a.b.c:500

IKEv2 Recv RAW packet dump
..
IKEv2-PLAT-3: RECV PKT [INFORMATIONAL] [51.a.b.c]:500->[80.x.y.z]:500 InitSPI=0x21ff0438c0410657 RespSPI=0xec3fe5f28836754c MID=00000003
IKEv2-PROTO-5: (99): Request has mess_id 3; expected 3 through 3

(99):
IKEv2-PROTO-2: (99): Received Packet [From 51.a.b.c:500/To 80.x.y.z:500/VRF i0:f0]
(99): Initiator SPI : 21FF0438C0410657 - Responder SPI : EC3FE5F28836754C Message id: 3
(99): IKEv2 INFORMATIONAL Exchange REQUESTIKEv2-PROTO-3: (99): Next payload: ENCR, version: 2.0 (99): Exchange type: INFORMATIONAL, flags: INITIATOR (99): Message id: 3, length: 80(99):
Payload contents:
IKEv2-PROTO-1: decrypt queued(99):
(99): Decrypted packet:(99): Data: 80 bytes
IKEv2-PROTO-1: Asynchronous request queued
IKEv2-PROTO-1:
IKEv2-PLAT-2: (99): Decrypt success status returned via ipc 1
(99): REAL Decrypted packet:(99): Data: 0 bytes

IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: READY Event: EV_RECV_INFO_REQ
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_RECV_INFO_REQ
IKEv2-PROTO-2: (99): Received DPD/liveness query
IKEv2-PROTO-2: (99): Building packet for encryption.
IKEv2-PROTO-2: (99): Sending ACK to informational exchange
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_ENCRYPT_MSG
IKEv2-PROTO-2: (99):
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_NO_EVENT
IKEv2-PROTO-5: (99): Locked SA.Event EV_FREE_NEG queued in the state EXIT
IKEv2-PLAT-2: (99): Encrypt success status returned via ipc 1
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_OK_ENCRYPT_RESP
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_TRYSEND
(99):
IKEv2-PROTO-2: (99): Sending Packet [To 51.a.b.c:500/From 80.x.y.z:500/VRF i0:f0]
(99): Initiator SPI : 21FF0438C0410657 - Responder SPI : EC3FE5F28836754C Message id: 3
(99): IKEv2 INFORMATIONAL Exchange RESPONSEIKEv2-PROTO-3: (99): Next payload: ENCR, version: 2.0 (99): Exchange type: INFORMATIONAL, flags: RESPONDER MSG-RESPONSE (99): Message id: 3, length: 80(99):
Payload contents:
(99):  ENCR(99):   Next payload: NONE, reserved: 0x0, length: 52
(99): Encrypted data: 48 bytes
(99):
IKEv2-PLAT-3: (99): SENT PKT [INFORMATIONAL] [80.x.y.z]:500->[51.a.b.c]:500 InitSPI=0x21ff0438c0410657 RespSPI=0xec3fe5f28836754c MID=00000003
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_CHK_INFO_TYPE
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: INFO_R Event: EV_START_DEL_NEG_TMR
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: EXIT Event: EV_CHK_PENDING
IKEv2-PROTO-5: (99): Sent response with message id 3, Requests can be accepted from range 4 to 4
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: EXIT Event: EV_NO_EVENT
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000002 CurState: EXIT Event: EV_FREE_NEG
IKEv2-PROTO-5: (99): Deleting negotiation context for peer message ID: 0x2

Afterwards, the following peer message IDs (0x3, 0x4, ..) are all similar:

Sep 19 17:42:28 [IKEv1]IKE Receiver: Packet received on 80.x.y.z:500 from 51.a.b.c:500

IKEv2 Recv RAW packet dump
21 ff 04 38 c0 41 06 57 ec 3f e5 f2 88 36 75 4c    |  !..8.A.W.?...6uL
2e 20 25 08 00 00 00 04 00 00 00 50 00 00 00 34    |  . %........P...4
09 7a 1a df 16 98 ed 80 68 8c eb 6f d3 13 a4 2c    |  .z......h..o...,
16 60 0d 69 0d c6 56 a8 2a 0c 53 a0 eb 7a 86 d8    |  .`.i..V.*.S..z..
38 ea 11 7a 3c 57 4c d9 98 8a 96 e2 be b0 c5 db    |  8..z<WL.........
IKEv2-PLAT-3: RECV PKT [INFORMATIONAL] [51.a.b.c]:500->[80.x.y.z]:500 InitSPI=0x21ff0438c0410657 RespSPI=0xec3fe5f28836754c MID=00000004
IKEv2-PROTO-5: (99): Request has mess_id 4; expected 4 through 4

(99):
IKEv2-PROTO-2: (99): Received Packet [From 51.a.b.c:500/To 80.x.y.z:500/VRF i0:f0]
(99): Initiator SPI : 21FF0438C0410657 - Responder SPI : EC3FE5F28836754C Message id: 4
(99): IKEv2 INFORMATIONAL Exchange REQUESTIKEv2-PROTO-3: (99): Next payload: ENCR, version: 2.0 (99): Exchange type: INFORMATIONAL, flags: INITIATOR (99): Message id: 4, length: 80(99):
Payload contents:
IKEv2-PROTO-1: decrypt queued(99):
(99): Decrypted packet:(99): Data: 80 bytes
IKEv2-PROTO-1: Asynchronous request queued
IKEv2-PROTO-1:
IKEv2-PLAT-2: (99): Decrypt success status returned via ipc 1
(99): REAL Decrypted packet:(99): Data: 0 bytes

IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: READY Event: EV_RECV_INFO_REQ
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_RECV_INFO_REQ
IKEv2-PROTO-2: (99): Received DPD/liveness query
IKEv2-PROTO-2: (99): Building packet for encryption.
IKEv2-PROTO-2: (99): Sending ACK to informational exchange
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_ENCRYPT_MSG
IKEv2-PROTO-2: (99):
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_NO_EVENT
IKEv2-PROTO-5: (99): Locked SA.Event EV_FREE_NEG queued in the state EXIT
IKEv2-PLAT-2: (99): Encrypt success status returned via ipc 1
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_OK_ENCRYPT_RESP
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_TRYSEND
(99):
IKEv2-PROTO-2: (99): Sending Packet [To 51.a.b.c:500/From 80.x.y.z:500/VRF i0:f0]
(99): Initiator SPI : 21FF0438C0410657 - Responder SPI : EC3FE5F28836754C Message id: 4
(99): IKEv2 INFORMATIONAL Exchange RESPONSEIKEv2-PROTO-3: (99): Next payload: ENCR, version: 2.0 (99): Exchange type: INFORMATIONAL, flags: RESPONDER MSG-RESPONSE (99): Message id: 4, length: 80(99):
Payload contents:
(99):  ENCR(99):   Next payload: NONE, reserved: 0x0, length: 52
(99): Encrypted data: 48 bytes
(99):
IKEv2-PLAT-3: (99): SENT PKT [INFORMATIONAL] [80.x.y.z]:500->[51.a.b.c]:500 InitSPI=0x21ff0438c0410657 RespSPI=0xec3fe5f28836754c MID=00000004
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_CHK_INFO_TYPE
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: INFO_R Event: EV_START_DEL_NEG_TMR
IKEv2-PROTO-5: (99): Action: Action_Null
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: EXIT Event: EV_CHK_PENDING
IKEv2-PROTO-5: (99): Sent response with message id 4, Requests can be accepted from range 5 to 5
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000004 CurState: EXIT Event: EV_NO_EVENT
IKEv2-PROTO-5: (99): SM Trace-> SA: I_SPI=21FF0438C0410657 R_SPI=EC3FE5F28836754C (R) MsgID = 00000003 CurState: EXIT Event: EV_FREE_NEG
IKEv2-PROTO-5: (99): Deleting negotiation context for peer message ID: 0x3

Answer 1

I will suggest you check this link for more details and sample configurations for connecting Cisco Adaptive Security Appliance (ASA) devices to Azure VPN gateways, make sure that your configuration parameters are correct, in few ASA models you need to Consult your VPN device specifications to verify the algorithms that are supported for your VPN device models and firmware versions.

-----------------------------------------------------------------------------------------------

If this answer was helpful, click “Mark as Answer” or “Up-Vote”. To provide additional feedback on your forum experience, click here.

Answer 2

Dear Zahid, thank you. I am familiar with that page. 
I have configured the IPSec policies on both the ASA and Azure (using custom policies) in the same way (see the table below), the two ends do actually agree on that, the session does start, and I can ping, rdp, http, .. across the two networks, the problem is that after a few minutes, and in a few occasion up to a couple of hours, the connection fails, but not the session. Actually, if I let the security association reach its lifetime, after 1 hour, the session is re-keyed successfully, and the connection reestablished, until the next failure.

Location	Connection		Observations
	Azure	ASA
IKEv2 Encryption	AES128	aes	aes means aes-128
IKEv2 Integrity	SHA256	sha256	-
DH Group	ECP256	19	ECP256 is equivalent to DH Group 19
IPsec Encryption	GCMAES128	aes-gcm	aes-gcm means aes-gcm-128
IPsec Integrity	GCMAES128	null	Azure requested both IPSec encryption and security to be set to the same value, but I understand this is an implementation detail, it means that the integrity is performed at the same time by the same protocol performing the encryption.
PFS Group	ECP256	-	ECP256 is equivalent to DH Group 19. Microsoft currently recommends to disable perfect forward secrecy because it causes disconnections problems [1] [2]
QM SA Lifetime	Seconds: 3600 Kbytes: 2048	Seconds: 3600	-
Policy Based Traffic Selector	True	Configured	this is a hard requirements for Cisco ASA 5516-x routers.

Answer 3

I did open a ticket with Microsoft, and while troubleshooting on the Azure side, the support engineer spotted that I had not configured the pfs group on the router side. I was actually aware of that, I had configured the router so as I understood that was recommended by Microsoft (e.g. see step 7 on the "Troubleshooting: Azure Site-to-Site VPN disconnects intermittently page). When I tried to configure PFSGroup to None on the Azure custom policy I received an error, which I worked around only setting the PfsGroup like the DHGroup.

Anyway, I have now enabled pfs on the crypto map, and this appears to have fixed the issue (or at last it did for the last 15 hours):

crypto map CRYPTO-MAP 2 set pfs group19

I have also asked the Microsoft support engineer if we should remove the pfs from both the ASA and the Azure custom policy, and they answered the more security the better, so they suggested to keep pfs enabled (I reckon under the hypothesis that it was not causing disconnections).