VPN Setup

Share on:

Here are some thoughts on setting up a VPN server on Ubuntu.

Most of this information is based on the following sites:

There are essentially three parts to setting up VPN: authentication, kernel configuration, and routing tables.


Authentication is done by the IKE protocol, which is implemented by Strongswan. Setting it up requires configuring the files /etc/ipsec.conf and /etc/ipsec.secrets.

There are two versions of IKE: IKEv1 and IKEv2. The newer version is better in a number of respects, but Mac OS X 10.9 and earlier use racoon as the IKE client, and racoon only supports IKEv1. Thus, I set up two VPN server connections with Strongswan, one for each protocol, and Strongswan automatically selects the right version upon client connection.

My goal was to set up certificate-based authentication of the server, but password-based authentication of clients. Strongswan supports PEM certificates, and so the same key that is used for website HTTPS or other TLS authentication works fine (but see below with regard to the OS X client). Both versions of IKE support various combinations of authentication protocols. For IKEv1, we want hybrid XAUTH authentication, and for IKEv2, we want EAP authentication of the remote client. In Strongswan, this is done by configuring “leftauth=pubkey” and other parameters, and then “rightauth=xauth” (for IKEv1) or “rightauth=eap-[form]” (for IKEv2).

Passwords are placed in the /etc/ipsec.secrets file. It seems that configuring an EAP password also automatically configures the XAUTH password.

Strongswan on Ubuntu is subject to Apparmor protection, which means that the Apparmor configuration needs to be adjusted to permit Strongswan to read the server’s private key if the key is placed in a location outside of /etc/ipsec.d.

For configuring IKEv2, this page provides information on the setup. The main point is that iOS only supports EAP-MSCHAPv2 as the authentication protocol.

Kernel Configuration

The kernel needs to be configured with parameters to allow for packet forwarding from the server. This is best done using a new file in /etc/sysctl.d.

Routing Tables

Finally, the routing tables need to be set up to accept both the authentication packets and the encrypted VPN data packets, and to forward the VPN data to their destinations.

Client Configuration

The IKEv2 client in iOS appears to work out of the box. For OS X 10.9, a specialized racoon configuration file is needed. Out of the box, the OS X Cisco VPN client supports pre-shared keys plus passwords and mutual certificates plus passwords, but it does not support hybrid authentication without a certificate on the client side.

When a VPN connection is initiated in OS X, the system creates a new configuration file in /var/run/racoon, which the default configuration file /etc/racoon/racoon.conf will read. The key is to start by obtaining the automatically created file (just initiate a VPN connection and copy whatever file shows up in /var/run/racoon), and then rewrite it and adjust /etc/racoon/racoon.conf to include that adjusted file.

The main adjustment is to remove references to the client certificate, and under each proposal block to change the authentication_method parameter to hybrid_rsa_client. Additionally, because of a bug with OS X VPN connections timing out due to rekeying, I changed the lifetime parameter to a larger value.

Finally (and this was the most frustrating bug), the server certificate as parsed by the OS X client does not include bundled intermediate certificates. (I’m not sure if this is the fault of Strongswan not sending intermediate certificates, or racoon ignoring them when sent.) So this means that you have to manually import the intermediates into the OS X keychain before the server certificate will be accepted.