Republished with permission from WatchGuard Technologies, Inc.

WatchGuard LiveSecurity

 

Incorporating Your Firebox into a Multi-vendor VPN

by David Piscitello, President, Core Competence

[Editor's note: LiveSecurity readers have responded enthusiastically to our articles specifically for beginners, but we're not about to neglect veteran network administrators, either. We think this lengthy how-to provides valuable reference material intermediate and advanced users will want to save -- or anyone using Virtual Private Networks. But we'd love to know what you think. Feel free to sound off at lsseditor@watchguard.com.]

In many organizations, multi-vendor interoperability is not an issue. Those organizations know that daily operations are more complicated when staff must contend with several different administrative interfaces. Besides, discounts follow volume, so at least initially most organizations operate networks with one brand of router, firewall or VPN security gateway.

But today, savvy organizations must plan ahead, anticipating a time when they will acquire a company, or be acquired, or when they must enter a business-to-business relationship with companies having different network equipment. In those cases, tasks like building an IPsec VPN are not so simple, even when all the equipment involved implements accepted Internet Standards and has met desired levels of certification. The tasks don’t have to be arduous and painful, however. With some careful planning and a good understanding of the IPsec capabilities of the VPN security gateways involved with firewalls, you can incorporate your Firebox into a multi-vendor VPN.

IPsec VPN Planning Sheets
Building an IPsec VPN requires careful planning. If you fail to take time here, you’ll be completely in the dark should your tunnel configurations go awry. Save yourself a lot of grief and test connectivity before you attempt to create VPN tunnels. You need to document everything you can about the topology of the network and the configuration of your security gateways, including your Firebox (with trusted, external and optional network addressing), default routes, DNS and NTP servers. Begin by creating an overall topology map, one that illustrates all the networks that are to be connected using VPN tunnels. Identify the networks by domain name, IP address and subnet mask.

Next, create a VPN Installation and Configuration Planning sheet like the one you will find here, for every security gateway to which your Firebox will establish VPN tunnels. These planning sheets will come in handy as you create a Security Policy Database on your Firebox and the remote security gateway.

The final step in planning your IPsec VPN is to document the configuration of IPsec tunnels for each security gateway (e.g., your Firebox and your business partner’s security gateway). Here’s where you need to know how IPsec and its companion, Internet Key Exchange (IKE), are implemented in your Firebox.

IKE and the Firebox
The Internet Key Exchange provides key and security session management for IPsec. IKE is also used for peer (security gateway) authentication. Using IKE, peer security gateways verify identities, then establish the parameters for what are known as security associations (SA).

IKE performs these functions in two phases. During Phase I, the Firebox and the remote security gateway authenticate each other by using Main Mode.In Main Mode, peers exchange identities, authentication methods, and credentials used to prove their identities. They also negotiate security parameters to protect IKE itself, specifying:

  • encryption algorithm, used to protect IKE messages from disclosure or capture (message confidentiality). 
  • authentication algorithm (hashed message authentication code), used to verify the source of each IKE message and also to protect the packet against modification
  • Diffie-Hellman Group, the method by which both the Firebox and remote security gateway will generate the same secret keys without revealing them. These keys are used by the encryption and authentication algorithms negotiated between the Firebox and remote security gateway

These parameters are collectively referred to as an IKE security policy. The Firebox implements only one IKE security policy, so you should prepare a second VPN Planning Worksheet like the one here. In the section called “Parameters for this tunnel: IKE,” you would enter:

  • IKE authentication method: pre-shared key (abbreviated PSS on form),
  • IKE encryption algorithm: (single) DES,
  • IKE message hash algorithm: SHA1-HMAC
  • IKE Diffie-Hellman Group: Group 1.

The pre-shared key used during IKE Phase 1 is the shared secret you enter when you configure IPsec gateways as part of your overall Branch Office IPsec VPN configuration. The pre-shared key is like a secret password; your Firebox demonstrates that it is who it says it is by using this key to digitally sign its own IP address. The IKE peer will do the same. 

Here’s the first instance where interoperability comes into play. You must configure the remote security gateway to use Main Mode SA with the same IKE security policy as the Firebox to successfully complete Phase I and create a pair of IKE security associations (one in each direction).

IPsec and the Firebox
After IKE SAs are created between the Firebox and a remote security gateway, they are used as a secure control channel for creating Phase II IPsec Security Associations. Here, parameters are negotiated for IPsec security associations that are used for data transfer. The relevant parameters for IPsec tunnels and the choices available on the Firebox through the IPsec Configuration sub-window are:

  • IPsec Encryption Algorithm: DES-CBC, 3DES-CBC (abbreviated DES and 3DES, respectively, on the form)
  • IPsec Diffie-Hellman Group: 1 (default)
  • Internet Protocol Security: ESP, AH
  • IPsec Mode: tunnel
  • IPsec Hash Algorithm: SHA1-HMAC, MD5-HMAC
  • IPsec SA Forced Key Expiration (“lifetime”): specify in time or kilobytes

Here’s the second instance where interoperability comes into play. You must configure the remote security gateway to use the same IPsec configuration parameters as the Firebox to succeed in creating IPsec security associations between the two systems. 

Unique sets of security parameters are negotiated for IKE and IPsec SAs. You may define several IPsec SAs, each handled by the same security gateway and thus negotiated using the same IKE SAs. For example, you may define an IPsec tunnel to a specific security gateway that encrypts traffic with 3DES.You may define a different IPsec tunnel that encrypts traffic to the rest of that private subnet using DES.Both are set up over the Firebox's IKE SAs, which always encrypts with DES. 

Because many of the same methods are used to protect IKE and IPsec (encryption, hashes, Diffie-Hellman), it’s easy to confuse these two security associations. If you see Phase 1 proposal, IKE transform, or ISAKMP transform in another-vendor gateway, these values must match the Firebox's built-in Phase 1 settings. If you see Phase 2 proposal or IPsec transform in another-vendor gateway, these values must match Phase 2 settings you configured in the Firebox Policy Manager under "Dynamic Security." (To adjust Dynamic Security settings, follow this path: Policy Manager => Network => Back Office VPN => IPSec => IPSec Configuration => Tunnels => select a tunnel =>Edit => Dynamic Security tab). Phase 2 must also use an exactly complementary set of host and subnet filters.

What the Log Reveals when You Succeed…
The following excerpt from my Firebox log shows a successful creation of an IPsec tunnel, following my attempt to ping the trusted network address of my business partner’s “other-vendor” security gateway (192.168.0.5) from a host on my trusted network (172.16.152.181/26), log entry 3418. You’ll find my sample VPN Planning sheets #1 (for my Firebox) and #2 especially helpful here; hopefully, these will drive home my point about planning.

3418 04/16/01 11:16:41 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

3428 04/16/01 11:16:41 iked[91] ipsec_nl_catcher: Acquiring key for channel/policy 2/0

3438 04/16/01 11:16:41 iked[91] Key acquire proxyraddr = 192.168.0.0

3448 04/16/01 11:16:41 iked[91] Key acquire proxyladdr = 172.16.152.128

3458 04/16/01 11:16:41 iked[91] ipsec_acquire_keys:laddr = 10.0.71.51, raddr = 10.0.160.211

Log entries 3418 through 3458 show the Firebox is preparing to initiate a Main Mode SA to the remote security gateway in response to my ping attempt to 192.168.0.5.

3468 04/16/01 11:16:41 iked[91] TO10.0.160.211 MM-HDRISA_SA

3488 04/16/01 11:16:42 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.18 192.168.0.5 8 0 (Any)

3498 04/16/01 11:16:42 iked[91] FROM 10.0.160.211 MM-HDR ISA_SA ISA_VENDORID

3508 04/16/01 11:16:42 iked[91] TO 10.0.160.211 MM-HDR ISA_KE ISA_NONCE

3528 04/16/01 11:16:43 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

3538 04/16/01 11:16:43 iked[91] FROM 10.0.160.211 MM-HDR ISA_KE ISA_NONCE

3548 04/16/01 11:16:43 iked[91] TO 10.0.160.211 MM-HDR* ISA_ID ISA_HASH

3558 04/16/01 11:16:44 iked[91] FROM 10.0.160.211 MM-HDR* ISA_ID ISA_HASH

3568 04/16/01 11:16:44 iked[91] Phase 1 completed as initiator

Log entries 3468-3558 show the multi-packet IKE negotiation associated with what is called an Identity Protection Exchange. The Firebox initiates the negotiation by sending a Main Mode header (MM-HDR) to propose a security association (ISA_SA), entry 3468. The remote security gateway responds (entry 3498). The Firebox next sends a Key Exchange payload (ISA_KE) containing keying material to be used in the (Diffie-Hellman) generation of keys for the IKE security association. A nonce payload is included in this IKE message to protect against replay attacks (entry 3508). Again, the remote security gateway responds in kind (entry 3538). At this point, both the Firebox and remote security gateway generate session keys, and IKE message payloads are hereafter encrypted using DES (denoted in the log entry with an asterisk). In the last exchange (3548, 3558), the Firebox and remote security gateway exchange authentication information; since we are using pre-shared secrets, this is a hash of the IP address.

Log entry 3568 records that Phase 1 has succeeded. The ping retries illustrate that the delay in creating the IKE SA exceeds the TTL for my ICMP messages.

3578 04/16/01 11:16:44 iked[91] Getting IPSEC preferences as Initiator propnum=1, mode=(Tunnel), laddr=10.0.71.51, raddr=10.0.160.211

3588 04/16/01 11:16:44 iked[91] Getting IPSEC preferences as Initiator propnum=2, mode=(Tunnel), laddr=10.0.71.51, raddr=10.0.160.211

3598 04/16/01 11:16:44 iked[91] TO 10.0.160.211 QM-HDR*-E02AC229 ISA_HASH ISA_SA ISA_NONCE ISA_ID ISA_ID

3618 04/16/01 11:16:44 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any

3628 04/16/01 11:16:44 iked[91] FROM  10.0.160.211 QM-HDR*-E02AC229 ISA_HASH ISA_SA ISA_NONCE ISA_ID ISA_ID ISA_NOTIFY

3638 04/16/01 11:16:44 iked[91] Received Unknown message, mess_id=0x00000000

3648 04/16/01 11:16:44 iked[91] Unknown Notify Message 24576

Phase II begins at log entry 3578.Security policies (preferences) we selected for the IPsec tunnel between my network and my partner’s network are negotiated using a packet exchange called Quick Mode (entry 3598). These messages are encrypted with DES (note the *), and a message hash and nonce are included for message integrity and protection against replay.

3658 04/16/01 11:16:44 iked[91] Load outbound ESP SA, Algs=ESP_3DES/AUTH_ALG_HMAC_SHA1 Life=3600sec/8192KB SPI=78f0e005

3668 04/16/01 11:16:44 iked[91] 172.16.152.128/26 <-> 192.168.0.0/24

3678 04/16/01 11:16:44 iked[91] Load inboundESP SA, Algs=ESP_3DES/AUTH_ALG_HMAC_SHA1 Life=3600sec/8192KB SPI=00043a5a

3688 04/16/01 11:16:44 iked[91] 172.16.152.128/26 <-> 192.168.0.0/24

Log entries 3658 and 3678 show the security policies we selected for the IPsec tunnel from my partner’s network to my network. Note that although in this example we have the same policy in both directions, you can in general apply different policies.If you do apply different policies, be sure that the filters are complementary at each end.

3698 04/16/01 11:16:44 iked[91] Committing SAs for channel=2 established with QM message_id=e02ac229

3708 04/16/01 11:16:44 kernel ipsec: make bundle for channel 2, 1 in SA's, 1 out SA's

3718 04/16/01 11:16:44 iked[91] TO 10.0.160.211 QM-HDR*-E02AC229 ISA_HASH

3778 04/16/01 11:16:54 iked[91] RE-TO 10.0.160.211 QM-HDR*-E02AC229 ISA_HASH

3808 04/16/01 11:17:05 iked[91] RE-TO 10.0.160.211 QM-HDR*-E02AC229 ISA_HASH

Log entries 3718-3808 illustrate how Quick Mode is used to produce IPsec session keys from the keying material exchanged during Phase I IKE. Quick Mode is also used to refresh keys when they expire (e.g., in 3600 seconds, according to our policy definition).

3898 04/16/01 11:17:16 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

3918 04/16/01 11:17:17 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

3938 04/16/01 11:17:18 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

3958 04/16/01 11:17:19 firewalld[74] allow out eth1 60 icmp 20 32 172.16.152.181 192.168.0.5 8 0 (Any)

Success! At this point my pings (and allowed applications) are traversing an IPsec tunnel. Remember that you change routing adjacencies when you establish an IPsec tunnel -- the “next hop” beyond your Firebox in a traceroute from the trusted network behind your Firebox may be your Internet router when you are not tunneling, but it will be the trusted network IP address of the remote network’s security gateway when you’ve established an IPsec tunnel in tunnel mode.

… And What to Do if You Don't
Shame on you if you didn’t confirm network connectivity between security gateways before you attempted to create VPN tunnels. Do so now. From each trusted network, ping the closest point you can reach on the public side of the other security gateway.Next, make sure that UDP traffic is flowing on port 500 between the two devices.Watch out for (and eliminate) any use of NAT or PAT along the route between your two devices.Beware of intermediate devices that block protocols 50 or 51.

What’s next?Packet analysis. If you get this far and come up short, you’ll have to revert to using a packet analyzer that parses the unencrypted bits of IKE and IPsec. Place it on your external network, between your Firebox and Internet router. Refer back to your planning sheets and confirm that both systems have the same security policies for IKE and IPsec. 

Parting Remarks
VPNs can seem like Very Particular Networks. But in truth, they are no more or less difficult to configure than a dynamic routing protocol like OSPF. The devil is in the details. And you have a better chance of keeping the devil at bay if you know the details, plan carefully, and document the plan.   ##

Copyright© 2001, WatchGuard Technologies, Inc. All rights reserved. WatchGuard, LiveSecurity, Firebox and ServerLock are trademarks or registered trademarks of WatchGuard Technologies, Inc. in the United States and other countries.