Securing IP traffic with IPsec

This short article describes how to set up the IPsec functionality in Linux 2.6 kernels, and how to use IPsec on Slackware Linux. Be aware that this is not an in-depth guide of the IPsec tools, please refer to the ipsec-tools documentation for more in-depth information about the various parameters and options that can be set.

IPsec is a collection of protocols that provide authentication, encryption and compression. The IPsec protocol is defined in several RFCs, and is a mandatory part of the new IETF standard for IP traffic (IPv6). The following protocols are defined in the IPsec standard:

In this article we will focus on using ESP to encrypt traffic, and Racoon to do the initial key exchange. To use IPsec two types of entries should be added: security associations (SAs) and security policies (SPs). Security policies define IPsec policies for certain connections. For example, you can specify to encrypt all the packages to host 192.168.1.11 with ESP. The kernel uses security associations to decide how the specified policy is going to be handled. For example, the encryption key and encryption algorithm settings for a connection are specified in a security association. We will manually set up the security policies, and let racoon take care of the security associations.

Slackware Linux has provided a Linux 2.6 kernel since Slackware Linux 10.0. The 2.6 kernel is available in Slackware 10.0 and 10.1 from the testing directory on CD2 of the Slackware Linux CD sets, or any of the official Slackware mirrors. The default Slackware 2.6 kernel has support for AH, ESP and IPcomp in for both IPv4 and IPv6. If you are compiling a custom kernel enable use at least the following options in your kernel configuration:

CONFIG_INET_AH=y
CONFIG_INET_ESP=y
CONFIG_INET_IPCOMP=y
    

Or you can compile support for IPsec protocols as a module:

CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
    

When you choose to compile IPsec support as a module, make sure that the required modules are loaded. For example, if you are going to use ESP for IPv4 connections, load the esp4 module.

Besides having kernel support you need userland tools to set up IPsec connections, and an IKE daemon for doing key exchanges. For IPsec with Linux 2.6 the Linux port of the KAME IPsec tools are the most widely used utilities. The KAME IPsec tools are not included in Slackware Linux 10.0 or 10.1. The next section describes how to install the KAME IPsec tools.

This section is divided in two parts, you can install the IPsec tools manually, or use the build script that is provided with this article.

# tar jxf ipsec-tools-x.y.z.tar.bz2
# cd ipsec-tools-x.y.z
# CFLAGS="-O2 -march=i486 -mcpu=i686" \
  ./configure --prefix=/usr \
              --sysconfdir=/etc \
              --localstatedir=/var \
              --enable-hybrid \
              --enable-natt \
              --enable-dpd \
              --enable-frag \
              i486-slackware-linux
# make
# make install
      

# tar zxf /path/to/ipsec-tools-slackbuild.tar.gz
      

At this point we can start to configure IPsec. The first thing you will want to do is to set up some IPsec policies. These policies define for what connections you would like to apply one of the IPsec protocols. Policies can be added to the security policy database (SPD) in the kernel with the setkey command. setkey can read security policies from a file, so it is a good idea to put the policies in a file, for example /etc/setkey.conf. Let's get started by looking at an example:

#!/usr/sbin/setkey -f

# Flush SPD
spdflush;

spdadd 192.168.1.10 192.168.1.11 any -P out ipsec
        esp/transport//require;

spdadd 192.168.1.11 192.168.1.10 any -P in ipsec
        esp/transport//require;
    

You should have no trouble recognizing the first line, which sets /usr/sbin/setkey to be the interpreter for this file (in case it is executed). The first thing we do is flushing the security policy database, with the spdflush command. If you have set up any other security policies that you want to keep you should not flush the security policy database.

The next to commands add policies to the security policy database. The following syntax is used for spdadd commands:

spdadd src_range dst_range upperspec policy ;
    

We can now deduct that the first command sets up a security policy for packets traveling from 192.168.1.10 to 192.168.1.11 with any protocol, and the second command sets up the same policy in reverse direction. The policy specification of the spdadd command can have a few forms, in this example we will only use this (simplified) form:

-P direction ipsec protocol/mode/src-dst/level
    

Now that the syntax of the policy is explained, it should be no trouble to read the policies of the two example SPD commands. Both policies specify that we require usage of ESP for packets that match the criteria. The first command specifies a policy for outgoing packets, the second command for incoming packets.

At this point policies are set up for packets sent out on 192.168.1.10 going to 192.168.1.11, and packets coming in from 192.168.1.11 on 192.168.1.10. On the machine with IP address 192.168.1.11 we can use the same policy with the in and out parameters swapped:

#!/usr/sbin/setkey -f

# Flush SPD
spdflush;

spdadd 192.168.1.10 192.168.1.11 any -P in ipsec
        esp/transport//require;

spdadd 192.168.1.11 192.168.1.10 any -P out ipsec
        esp/transport//require;
    

After setting up some policies you can add them to the kernel security policies database with the setkey -f command, for example: setkey -f /etc/setkey.conf. If you want to set up these policies automatically when a system boots, you can add something along these lines to /etc/rc.d/rc.inet1:

if [ -x /usr/sbin/setkey ] && [ -r /etc/setkey.conf ]; then
  /usr/sbin/setkey -f /etc/setkey.conf
fi
    

For a more extensive explanation of the setkey, read the setkey(8) manual page.

When the kernel handles a packet for which a security policy is set up, the kernel will look up the security associations database (SAD) to see what parameters (e.g. encryption key) should be used for handling the packet. We will use the racoon daemon to set up security associations.

Racoon establishes a security association after a key exchange that is done in two stages:

An example racoon configuration is listed below. This configuration is also used in the SlackBuild script that is linked from this article. Configuration of racoon is extensively described in the racoon.conf(5) manual page. In short, we define two sections, one for phase 1 (remote) and one for phase 2 (sainfo), that specify which algorithms should be used for encryption and hashing. In addition, we specify in the remote section that we would like to use pre-shared keys to verify the authenticity of the other host. As specified with the path pre_shared_key option, the /etc/racoon/psk.txt file will be used as the pre-shared key database.

# search this file for pre_shared_key with various ID key.
path pre_shared_key "/etc/racoon/psk.txt";

# racoon will look for certificate file in the directory,
# if the certificate/certificate request payload is received.
path certificate "/etc/racoon/certs";

remote anonymous
{
        exchange_mode aggressive, main;
        my_identifier address;
        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group 2;
        }
}

sainfo anonymous
{
        pfs_group 2;
        encryption_algorithm 3des, blowfish 448, rijndael;
        authentication_algorithm hmac_sha1, hmac_md5;
        compression_algorithm deflate;
}
    

The pre-shared key database has a very simple two-column format. The first column specifies the host name or IP address, the second column the shared key. Both columns are separated with whitespace. When the key is prefixed with 0x, the key is interpreted as a hexadecimal number. For example, an entry in the database could look like this (obviously, the key should be a good random string of letters, numbers and other characters):

192.168.1.11    This is some secret key
    

Racoon can now be started with the racoon command. The SlackBuild script includes a startup script for racoon. If you want to start racoon when the system boots, make the startup script executable:

# chmod a+x /etc/rc.d/rc.racoon
    

After you have added the security policies to the security policy database, and started Racoon, you can test your IPsec configuration. For instance, you could ping the other host to start with. The first time you ping the other host, this will fail:

$ ping 192.168.1.11
connect: Resource temporarily unavailable
    

The reason for this is that the security associations still have to be set up. But the ICMP packet will trigger the key exchange. ping will trigger the key exchange. You can see whether the exchange was succesful or not by looking at the Racoon log messages in /var/log/messages. A succesful key exhange looks like this:

 Apr  4 17:14:58 terrapin racoon: INFO: IPsec-SA request for 192.168.1.11 queued due to no phase1 found.
 Apr  4 17:14:58 terrapin racoon: INFO: initiate new phase 1 negotiation: 192.168.1.10[500]<=>192.168.1.11[500] 
 Apr  4 17:14:58 terrapin racoon: INFO: begin Aggressive mode. 
 Apr  4 17:14:58 terrapin racoon: INFO: received Vendor ID: DPD 
 Apr  4 17:14:58 terrapin racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
 Apr  4 17:14:58 terrapin racoon: INFO: ISAKMP-SA established 192.168.1.10[500]-192.168.1.11[500] spi:58c4669f762abf10:60593eb9e3dd7406
 Apr  4 17:14:59 terrapin racoon: INFO: initiate new phase 2 negotiation: 192.168.1.10[0]<=>192.168.1.11[0] 
 Apr  4 17:14:59 terrapin racoon: INFO: IPsec-SA established: ESP/Transport 192.168.1.11->host1ip; spi=232781799(0xddff7e7) 
 Apr  4 17:14:59 terrapin racoon: INFO: IPsec-SA established: ESP/Transport 192.168.1.10->192.168.1.11 spi=93933800(0x59950e8) 
    

After the key exchange, you can verify that IPsec is set up correctly by analyzing the packets that go in and out with tcpdump. tcpdump is available in the n diskset. Suppose that the outgoing connection to the other host goes through the eth0 interface, you can analyze the packats that go though the eth0 interface with tcpdump -i eth0. If the outgoing packets are encrypted with ESP, you can see this in the tcpdump output. For example:

# tcpdump -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
17:27:50.241067 IP terrapin.taickim.net > 192.168.1.11: ESP(spi=0x059950e8,seq=0x9)
17:27:50.241221 IP 192.168.1.11 > terrapin.taickim.net: ESP(spi=0x0ddff7e7,seq=0x9)