Republished with permission from WatchGuard Technologies, Inc.

WatchGuard LiveSecurity

 

Debugging Mobile User (IPsec) VPN

by David Piscitello, Core Competence, Inc.

 

Though VPN technology has been around longer than most people realize, it hasn't come into broad usage until the last couple of years. In addition, the IPsec standard is not exactly an example of seamless elegance. The result is that VPN technology, while improving steadily, is still complex and time-consuming to install.

My previous article, "Deploying Mobile User (IPsec) VPN," offered help and guidance for administrators in the early stages of providing remote users VPN access to a network. This article covers the next phase: if you've begun installing VPN and the connections aren't coming together, here's where you might find help. I'll suggest some preventive measures that can ease VPN implementation, and show examples of how to interpret your Firebox logs when remote access fails.

Troubleshooting MUVPN Connections
If the security policy implemented on your Firebox blocks incoming ICMP messages, it may be hard to confirm network connectivity between remote users and a Firebox before you attempt to create VPN tunnels. Have the remote user ping the closest point he can reach on the public side of the Firebox (for example, your Internet access router); as an alternative, try to ping the public IP address of the remote user from behind the Firebox (beware that the client's address may change over time!).  Other speed bumps to watch for:

  • Make sure that UDP traffic is flowing on port 500 between the two devices. 
  • Watch out for (and eliminate) any use of NAT or PAT along the route between your two devices.  
  • Beware of intermediate devices that block ports 50 or 51.

If you get this far and come up short, you’ll have to revert to using packet analysis. The good news is that Gerald Combs’ freeware packet analyzer, Ethereal, parses the unencrypted bits of IKE and IPsec, and it’s available for Windows NT/2000 and *NIX. Install this or your favorite more expensive packet analyzer on your external network, between your Firebox and Internet router. 

Refer back to your planning sheets and confirm that systems have the same security policies for IKE and IPsec.

What the logs may show when remote access fails
Differences in configurations between the MUVPN client and the Firebox may prevent your remote user from establishing IPsec tunnels and accessing your Trusted Network. Firebox and MUVPN client logs may help you diagnose the problem.

Should your remote user change or mistype his password, you’ll see the following sequence of log entries:

38228 06/13/01 15:32:50 iked[91] 
FROM 207.246.161.212 AG-HDR  ISA_SA ISA_KE ISA_NONCE ISA_ID

38238 06/13/01 15:32:50 iked[91] 
TO 207.246.161.212 AG-HDR ISA_SA ISA_KE ISA_NONCE ISA_ID ISA_HASH

38248 06/13/01 15:32:50 iked[91] 
CRYPTO ACTIVE after delay

38258 06/13/01 15:32:50 iked[91] 
FROM 207.246.161.212 IF-HDR -2B5EB394 ISA_NOTIFY

38268 06/13/01 15:32:50 iked[91] 
Received AUTH_FAILED message, mess_id=0x00000000

In this case, log entry 38268 isolates the problem for you.

In the process of managing remote users, you may run into a situation where the IKE encryption selected by the remote access client software differs from the default DES encryption used by the Firebox. This might be more commonly encountered in situations where another IPsec client (for example, the SafeNet client), is installed by a remote user. While the Firebox and SafeNet client are compatible, users who manually configure this client may choose 3DES for IKE encryption. You might see this log sequence:

40998 06/13/01 15:40:16 iked[91] 
FROM 207.246.161.212 AG-HDR  ISA_SA ISA_KE ISA_NONCE ISA_ID

41008 06/13/01 15:40:16 iked[91] 
Proposal is unacceptable. mess_id=0

41018 06/13/01 15:40:16 iked[91] 
Sending NO_PROPOSAL_CHOSEN message

41028 06/13/01 15:40:16 iked[91] 
Error processing (sa)

41038 06/13/01 15:40:16 iked[91] 
Aggressive Mode processing failed

The log entries 41008-41018 show that the IPsec client and Firebox could not agree on the parameters of the IKE Phase 1 SA.

The final example is subtle, and it’s a good one to examine. Do you recall that you assign a static IP address per remote user at the Firebox? Well, what if you change IP assignments and one of your remote users fails to update his configuration file?  Let’s look at the log entries for a situation where user “lisa” has changed her Virtual IP address on the Trusted Network from 207.86.152 .186 to 207.86.152.188:

47048 06/13/01 16:00:37 iked[91] 
FROM  207.246.161.212 AG-HDR  ISA_SA ISA_KE ISA_NONCE ISA_ID

47058 06/13/01 16:00:38 iked[91] 
TO 207.246.161.212 AG-HDR  ISA_SA ISA_KE ISA_NONCE ISA_ID ISA_HASH

47068 06/13/01 16:00:38 iked[91] 
CRYPTO ACTIVE after delay

47078 06/13/01 16:00:38 iked[91] 
FROM 207.246.161.212 AG-HDR  ISA_HASH

47088 06/13/01 16:00:38 iked[91] 
Phase 1 completed as responder

47098 06/13/01 16:00:38 iked[91] 
Deleting old phase 1 SA for 207.246.161.212

47108 06/13/01 16:00:38 iked[91] 
Deleting SA: peer 207.246.161.212

47118 06/13/01 16:00:38 iked[91]
my_cookie  22F81615A7322B8F

47128 06/13/01 16:00:38 iked[91]
his_cookie F9C35BC2D5ACCCDF

47138 06/13/01 16:00:38 iked[91] 
Drop Host 8 207.86.152.186 ipsec_users lisa succeeded

47158 06/13/01 16:00:38 iked[91] 
User "lisa" at 207.86.152.186 logged out

47168 06/13/01 16:00:38 iked[91] 
lisa unbound

47178 06/13/01 16:00:38 iked[91] 
Updated lisa000 channel for lisa rgw

47198 06/13/01 16:00:38 iked[91] 
User "lisa" at 207.86.152.186 logged in

47208 06/13/01 16:00:38 iked[91] 
Add Host 8 207.86.152.186 ipsec_users lisa succeeded

47218 06/13/01 16:00:38 iked[91] 
lisa bound to 207.246.161.212

Log entry 47198 shows user “lisa” has successfully established an IPsec tunnel to the Firebox (this log sequence also shows that “lisa” has previously connected, so you now have a picture of what the process of clearing old IPsec tunnel “state” looks like). Note that the Firebox has bound “lisa” to the Virtual IP address 207.86.152.188 upon completion of IKE Phase 2.

47618 06/13/01 16:01:47 iked[91] 
FROM 207.246.161.212 QM-HDR*-4B7E7383 ISA_HASH ISA_SA ISA_NONCE ISA_ID ISA_ID

47628 06/13/01 16:01:47 iked[91] 
Getting IPSEC preferences as Responder propnum=1, mode=(Tunnel), laddr=64.53.71.51, raddr=207.246.161.212

47638 06/13/01 16:01:47 iked[91] 
TO 207.246.161.212 QM-HDR*-4B7E7383 ISA_HASH ISA_SA ISA_NONCE ISA_ID ISA_ID

47648 06/13/01 16:01:47 iked[91] 
FROM 207.246.161.212 QM-HDR*-4B7E7383 ISA_HASH

47658 06/13/01 16:01:47 iked[91] 
Load outbound ESP SA, Algs=ESP_3DES/AUTH_ALG_HMAC_SHA1 Life=3600sec/0KB SPI=c87f1960

47668 06/13/01 16:01:47 iked[91]
207.86.152.128/26 <-> 207.86.152.188/32

47678 06/13/01 16:01:47 iked[91] 
Load inbound  ESP SA, Algs=ESP_3DES/AUTH_ALG_HMAC_SHA1 Life=3600sec/0KB SPI=01048f22

47688 06/13/01 16:01:47 iked[91] 
207.86.152.128/26 <-> 207.86.152.188/32

47698 06/13/01 16:01:47 iked[91] 
Committing SAs for channel=1 established with QM message_id=4b7e7383

47708 06/13/01 16:01:47 kernel 
ipsec: make bundle for channel 1, 1 in SA's, 1 out SA's

IKE Phase 2 completes, “lisa” and the Firebox set up IPsec SA’s for data, and life is good, right? Look carefully at log entry 47668. Now look at what happens when lisa tries to ping to the FTP server on the Trusted Network (yes, ping is enabled for ipsec_users):

47728 06/13/01 16:01:47 firewalld[74] 
deny in ipsec0
60 icmp 20 32 207.86.152.188 207.86.152.181 8 0 (Ping)

47758 06/13/01 16:01:48 firewalld[74] 
deny in ipsec0
60 icmp 20 32 207.86.152.188 207.86.152.181 8 0 (Ping)

47788 06/13/01 16:01:48 firewalld[74] 
deny in ipsec0
60 icmp 20 32 207.86.152.188 207.86.152.181 8 0 (Ping)

What’s wrong? Remember, during IKE Phase 1, the remote IPsec client identifies itself using the USER-FQDN “lisa.” The Virtual IP address has local meaning to the Firebox and MUVPN client, and is never passed in IKE. The Virtual IP address tells the client what IP address to use as its source address in encapsulated security payload (ESP) packets. It also tells the Firebox to route return packets back to the MUVPN client. The problem here is that the Firebox has a route for 207.86.152.186 for lisa. Thus, all traffic from lisa arrives with a source IP address of 207.86.152.188, but the Firebox is configured to only permit packets from .186

This is actually easier to appreciate in the MUVPN client log than the Firebox log, where each IKE parameter (called transforms) are parsed, and helps make my point that logging at both ends of IPsec VPNs is essential:

2001/06/13 16:02:56.890: AL_IKE--
Start Phase 1 negotiation with peer 64.53.71.51

2001/06/13 16:02:56.950: AL_IKE--
DestID: IPV4_ADDR--64.53.71.51 PresharedKey: xxxxxxxxxxxxxxxxx

2001/06/13 16:02:56.950: AL_IKE--
Protocol -- PROTO_ISAKMP

2001/06/13 16:02:56.950: AL_IKE--
Transform -- KEY_IKE

2001/06/13 16:02:56.950: AL_IKE--
Encryption -- DES_CBC

2001/06/13 16:02:56.950: AL_IKE--
Hash -- MD5_HASH

2001/06/13 16:02:56.950: AL_IKE--
MyID: USER_FQDN--lisa

2001/06/13 16:02:56.950: AL_IKE--
Authentication -- PRESHARED_KEY

2001/06/13 16:02:56.950: AL_IKE--
LifeType -- SECONDS

2001/06/13 16:02:56.950: AL_IKE--
LifeDuration -- 3600

2001/06/13 16:02:56.950: AL_IKE--
GroupDescription -- MODP_768

2001/06/13 16:02:56.950: AL_IKE--
AggressiveMode Exchange Selected

2001/06/13 16:02:57.220: AL_IKE--
AggressiveMode -- initiator sent out message1

2001/06/13 16:02:58.100: AL_IKE--
IKE message received from 64.53.71.51

2001/06/13 16:02:58.100: AL_IKE--
AggressiveMode -- initiator received response message1

2001/06/13 16:02:58.100: AL_IKE--
Rcvd Msg [HDR SA KE NONCE ID HASH]

2001/06/13 16:02:58.210: AL_IKE--
DestID: IPV4_ADDR--64.53.71.51 ProtocolID:17 Port:500 PresharedKey:xxxxxxxxxxxxxxxx

2001/06/13 16:02:58.210: AL_IKE--
AggressiveMode -- initiator sent out message2

2001/06/13 16:02:58.210: AL_IKE--
Phase 1 negotiation succeeded with 64.53.71.51

2001/06/13 16:02:58.320: AL_IKE--
Sending Notification CONNECTED (0x4000) to peer 64.53.71.51

2001/06/13 16:02:58.320: AL_IKE--
Phase 1 boom counter = 3600

Here, IKE Phase 1 completes. Note that the Virtual IP address never passed between the Firebox and remote IPsec client.

2001/06/13 16:04:06.760: AL_IKE--
Start Phase 2 negotiation with peer 64.53.71.51

2001/06/13 16:04:06.760: AL_IKE--
RemoteHostID: IPV4_ADDR_SUBNET-- 207.86.152.128/255.255.255.192

2001/06/13 16:04:06.760: AL_IKE--
SourceID: IPV4_ADDR--207.86.152.188

2001/06/13 16:04:06.760: AL_IKE--
Protocol -- PROTO_IPSEC_ESP, Nmbr of transforms -- 1

2001/06/13 16:04:06.760: AL_IKE--
Transform -- ESP_3DES

2001/06/13 16:04:06.760: AL_IKE--
Authentication -- HMAC_SHA

2001/06/13 16:04:06.760: AL_IKE--
LifeType -- SECONDS

2001/06/13 16:04:06.760: AL_IKE--
LifeDuration -- 3600

2001/06/13 16:04:06.760: AL_IKE--      EncapsulationMode -- TUNNEL

2001/06/13 16:04:06.760: AL_IKE--
QuickMode -- initiator sent out message1

2001/06/13 16:04:07.140: AL_IKE--
IKE message received from 64.53.71.51

2001/06/13 16:04:07.140: AL_IKE--  
QuickMode -- initiator received response message1

2001/06/13 16:04:07.140: AL_IKE--  
Rcvd Msg [HDR* HASH SA NONCE ID ID]

2001/06/13 16:04:07.190: AL_IKE--
Source ID: IPV4_ADDR--207.86.152.188

2001/06/13 16:04:07.190: AL_IKE--
RemoteHost ID: IPV4_ADDR_SUBNET-- 207.86.152.128/255.255.255.192

2001/06/13 16:04:07.250: AL_IKE--
QuickMode -- initiator sent out message2

2001/06/13 16:04:07.250: AL_IKE--
Phase 2 negotiation succeeded with 64.53.71.51

Phase 2 completes. However, the Firebox will deny incoming packets from .188 rather than accepting them because it’s bound lisa to .186. The remedy here is easy: change the MUVPN configuration so that both the Firebox and the client use the same IP address.

Closing Remarks
MUVPNs can be slightly more difficult to configure than Branch Office VPNs, for two reasons: diversity of computing platforms on which the client software is installed, and quite likely, less sophisticated users at the client end, which often results in configuration errors. Plan correctly and you can minimize the helpdesk calls and maximize your remote access security. And use all the logging facilities at your disposal.

Remember, however, that making the traffic from remote users and teleworkers secure is only part of the puzzle. Mobile worker laptop security and teleworker broadband connections need firewall and antivirus protection, OS and file system protection, and more. MUVPN is just one element in your entire defense system. ##

Resources:
Revisit Dave's earlier columns on Security and xDSL Connections: they are still relevant and helpful. 

If you missed the "prequel" to this article, it's in the LiveSecurity archive, entitled "Deploying Mobile User (IPsec) VPN."

Was this article useful to you? Is there a security topic you want our experts to write about? Let us know at lsseditor@watchguard.com.

Copyright© 2001, WatchGuard Technologies, Inc. All rights reserved. WatchGuard, LiveSecurity, Firebox and ServerLock are trademarks or registered trademarks of WatchGuard Technologies, Inc. in the United States and other countries.