Troubleshooting VPN

© 2006 Cisco Systems, Inc. All rights reserved. Course acronym vx.x—#-1

India TAC Training

© 2006-7 Cisco Systems, Inc. All rights reserved. ASA/PIX 8.0—2-2

Troubleshooting IPSec VPN


Troubleshooting

Show commands

Debug commands

Common Issues/Errors


Troubleshooting - Show Commands IPSEC depends on successful policy negotiation. While

IPSEC peers are negotiating IKE and IPSEC parameters, if the policies do not match the negotiations will result in failure. We can troubleshoot IKE & IPSEC by the following show commands: show crypto isakmp sa (PIX / ASA and IOS routers) show crypto ipsec sa (PIX / ASA and IOS routers) From the show commands we can determine if the SA’s are in the right state, and if ISAKMP went through fine and now the IPSec traffic is being Encrypted/Decrypted between the two IPSec endpoints.


Troubleshooting - Debug Commands IPSEC depends on successful policy negotiation. While

IPSEC peers are negotiating IKE and IPSEC parameters, if the policies do not match the negotiations will result in failure. We can troubleshoot IPSEC by the following commands: debug crypto ipsec debug crypto isakmp From the debug error messages we can determine what part of the negotiation is failing and correct the appropriate parameter.


IPSEC Common Issues


IPSEC Common Issues

NAT with IPSec

Firewalling and IPSec

MTU Issues

Loss of Connectivity of IPSec Peers

Routing

Interoperability Troubleshooting


access-list no_nat permit ip 10.1.1.0 255.255.255.0 10.1.2.0 255.255.255.0

nat (inside) 0 access-list no_nat

interface GigabitEthernet0/0 nameif outside security-level 0 ip address 209.165.202.129 255.255.255.0

interface GigabitEthernet0/1 nameif inside security-level 100 ip address 10.1.1.1 255.255.255.0

Bypassing NAT Entries in ASAAccess-List “bypassnat” Defines Interesting Traffic to bypass NAT for VPNNAT 0 Command Bypasses NAT for the Pkts Destined over the IPSec Tunnel


NAT in the Middle of an IPSec Tunnel

IPSec pass-through: ISAKMP cookie and ESP SPI are used to build translation table ASA(config)# fixup protocol esp-ike

IPSec Nat Transparency (NAT-T): UDP 500 UDP 4500 ASA(config)# isakmp nat-traversal <natkeepalive> IPSec over TCP: TCP 10000 ASA(config) isakmp ipsec-over-tcp port 10000

VPN Client

VPN Client

NATNAT Internet

VPN Gateway


IPSEC Common Issues

NAT with IPSec


MTU Issues


Routing



Firewall in the Middle

One way block UDP port 500 (ISAKMP)

show crypto isakmp sa: MM_NO_STATE

ping from R A, R B has debug, ping from R B , R A has no debug

One way block ESP (IP protocol type 50) show crypto isakmp sa: QM_IDLE

R A has encryption no decryption, R B has decryption and encryption

UDP port 4500 (NAT-T)

VPN client tunnel is up, VPN client statistics “transparent tunnel inactive”

Private

Internet

Private PublicRouter A Router B


IPSEC Common Issues

NAT with IPSec


MTU Issues


Routing



IPSec MTU Issue

InternetInternet

TCP hdr Data

TCP hdr

TCP hdrESP hdr

a. Original Packetb. IPSec Transport Mode 36 bytes c. IPSec Tunnel Mode 20+36=56 bytes

a

b

c

IPSec

IP Hdr 1

Data

Data

IP Hdr 1

IP hdr 2

ESP hdrIP hdr 1


IPSec and Path MTU Discovery

1500 DF=1

ICMP Type3 Code 4

(1444)

1444 DF=1 1500 DF copied

Path 1500Media 1500

IPSec Tunnel

MTU 1500 MTU 1500

MTU1500

MTU1400

MTU1500

Path 1500Media 1500

10.1.1.2 10.1.2.2

e1/1 e1/0

172.16.172.20/28172.16.172.10/28

ICMP (1400)

IPSec SPI copied

ICMP Type3 Code 4

(1344)

1400 1344 14001344 DF=1

ICMP: dst (172.16.172.20) frag. needed and DF set unreachable rcv from 172.16.172.11

Adjust path MTU on corresponding IPSec SApath mtu 1400, media mtu 1500

current outbound spi: EB84DC85

ICMP: dst (10.1.2.2) frag. needed and DF set unreachable sent to

10.1.1.2 (“debug ip icmp” output)


How to manually determine Path MTU Ping from client PC:

ping www.cisco.com -l 1400 -f Pinging www.cisco.com [198.133.219.25] with 1400 bytes of data: Reply from 198.133.219.25: bytes=1400 time=168ms TTL=120

ping www.cisco.com -l 1500 -f Pinging www.cisco.com [198.133.219.25] with 1500 bytes of data: Packet needs to be fragmented but DF set.

Ping from the router: sv3-6#ping ip Target IP address: 198.133.219.25 Repeat count [5]: 1 Datagram size [100]: 1400 Extended commands [n]: y Source address or interface: FastEthernet0/0 Set DF bit in IP header? [no]: yes Sweep min size [36]: 1400 Sweep max size [18024]: 1500 Sweep interval [1]: 10

!!!!......


MTU Issues Work Around: Adjusting IP MTU & TCP MSS

ASA/PIX: mtu outside 1492

sysopt connection tcpmss 1392

IP Fragmentation and PMTUD

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml

Adjusting IP MTU, TCP MSS, and PMTUD on Windows and Sun Systems

http://www.cisco.com/en/US/tech/tk870/tk877/tk880/technologies_tech_note09186a008011a218.shtml


IPSEC Common Issues

NAT with IPSec


MTU Issues


Routing



SPIPeerLocal_idRemote_idTransform…

IPSec SA

Internet

SPIPeerLocal_idRemote_idTransform…

IPSec SA

00:01:33: %CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSec packet has invalid spi for destaddr=172.16.172.28, prot=50, spi=0xB1D1EA3F(-1311643073)


ESP SPI=0xB1D1EA3F



Dead Peer Detection

crypto isakmp keepalive <# of sec. between keepalive> <# of sec. between retries if keepalive fails>

DPD Message (R-U-There)DPD Message (R-U-There ACK)


IPSEC Common Issues

NAT with IPSec


MTU Issues


Routing



show crypto ipsec saIPSec

ASA1(config)# sh crypto ipsec sa

#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0 #pkts decaps: 32906, #pkts decrypt: 32906, #pkts verify: 32906

ASA2(config)# sh crypto ipsec sa

#pkts encaps: 32829, #pkts encrypt: 32829, #pkts digest: 32829 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

I sent encrypted packets, and got nothing back from remote host

I sent decrypted packets, and got nothing from the local host


IPSEC Common Issues

NAT with IPSec


MTU Issues


Routing



Start with configuring the two ends side by side with exact matching policies

Turn off vendor specific features: Mode config, Xauth, IKE keepalive

Interoperability Tips

IKE authentication method

Hash algorithm

DH group

ISAKMP SA lifetime

Encryption algorithm

Matching pre-shared secret

IPSec mode (tunnel or transport)

Encryption algorithm

Authentication algorithm

PFS group

IPSec SA Lifetime

Interesting traffic definition

Phase I ParametersPhase I Parameters Phase II ParametersPhase II Parameters


Other Issues - Errors

IKE Policy mismatch

Pre-shared key mismatch

Access-list mismatch

IPSec policy mismatch

IKE Pool misconfigured

IPSec peer misconfigured

Additional Considerations


IKE Policy mismatch

If there is a mismatch or if there are no common ISAKMP policies then the following error will be seen.The solution is to configure a common ISAKMP policy on both peers.

ISAKMP (0): atts are not acceptable. Next payload is 0 ISAKMP (0): no offers accepted! ISAKMP (0): SA not acceptable!


Pre-shared key mismatch

If the pre-shared keys on both the peers do not match then the following error will be seen.

1d00H:%CRPTO-4-IKMP_BAD_MESSAGE: IKE message from 172.16.172.34 failed its sanity check or is malformed

which will result in :

%CRYPTO-6-IKMP_MODE_FAILURE: Processing of Main Mode failed with peer at172.16.172.34


Access-list mismatch If the access-lists on the peer IPSEC devices do not

match that is if they are not mirror images of each other then the following error will occur :

IPSEC(validate_transform_proposal): proxy identities not supported ISAKMP: IPSec policy invalidated proposal

It is also important to note that the word “any” should not be used in the access-list .


IPSec policy mismatch If the IPSEC transform-set policies do not match ,

then the following error will be seen. Both the peer should have identical IPSEC transform-set policies.

ISAKMP: SA life duration (VPI) of 0x0 0x46 0x50 0x0ISAKMP: authenticator is HMAC-MD5 IPSEC(validate_proposal): transform propos al (prot 3, trans 2, hmac_alg 1) not supported

ISAKMP (0): atts not acceptable. Next payload is 0ISAKMP (0): SA not acceptable!


IKE Pool misconfigured

If the PIX is configured for IKE mode-config and the pool is misconfigured then the following error will be seen:

IPSEC(key_engine_delete_sas): delete all SAs shared with 171.69.89.116return status is IKMP_NO_ERR_NO_TRANS04101: ISAKMP: Failed to allocate address for client from pool

ISADB: reaper checking SA 0x80e02638, conn_id = 0 DELETE IT!


IPSec peer misconfigured

If the IPSEC peer is misconfigured under the crypto map , then the following error message will be seen

1d00h: ISAKMP: No cert, and no keys (public or pre-shared) with remote peer 172.167.172.33

1d00h: ISAKMP (0:1): purging SA


Additional Considerations - Split tunneling

We need to use “split tunneling” when using the Unity client if we want to simultaneously have a IPSEC tunnel to the PIX and also INTERNET connection.

vpngroup vpn3000 split-tunnel 160access-list 160 permit ip 192.168.2.0 255.255.255.0 30.1.1.0 55.255.255.0

Here the IPSEC tunnel will be only established between the source destination specified by the access-list.


Additional ConsiderationsIPSec Multiple peers

If there are multiple peers to a PIX , make sure that the match address access-lists for each of the peers are mutually exclusive from the match address access list for the other peers

If this is not done, the PIX will choose the wrong crypto map to try and establish a tunnel with one of the peers


Additional ConsiderationsIPSec from behind low-end firewalls

Issues With IPSec/ESP or IPSec/UDP, two VPN users to SAME IPSec VPN

server – 2nd user may be disallowed 2nd user may cause disconnection of 1st user

Solutions Multiples ISAKMP sessions Vary source port [NOT UDP 500] and keep track Based on SPI [Keep UDP 500/500]


Additional ConsiderationsDES - 3DES issue

When using SSH, if the pix has only DES key enabled and SSH client is 3DES then the following error will occur

pix520-1(config)# 315011: SSH session from 171.69.89.116 on interface outside for user "" disconnected by SSH server, reason: "Invalid cipher type" (0x06)315011: SSH session from 171.69.89.116 on interface outside for user "" disconnected by SSH server, reason: "Invalid cipher type" (0x06)

We can also use the “ sh ssh sessions” to view the current ssh connections


Q&A


Troubleshooting VPN

Documents

invalid cipher

successful

ipsec common

ipsec peers

pix asa

ipsec peers

ipsec parameters

ipsec tunnel