Migrating from IKEv1 to IKEv2 on AIX with Configuration Examples

White Papers

Abstract

This document provides clear, hands-on guidance to simplify your transition to IKEv2.
It outlines how to configure IPsec tunnels on AIX using both certificates and pre-shared keys, offering practical steps, command references, and XML examples. Designed for AIX versions 6.1 and later,
this guide helps streamline IPsec implementation with IKEv2 and ensures a smoother, more secure migration from IKEv1.

Content

Internet Protocol Security (IPsec) provides robust security at the Internet Protocol (IP) layer, safeguarding data transmitted over networks. This guide details two distinct methods for establishing IKEv2 IPsec tunnels between AIX systems (versions 6.1, 7.1, 7.2 and 7.3): one utilizing digital certificates and the other employing pre-shared keys. A foundational understanding IPsec concepts is recommended.

Prerequisites and Key Concepts
Method 1: IKEv2 Tunnels using Certificates
- Generating Certificates on AIX with GSKit
- Configuring the IKEv2 Tunnel with Certificates (XML)
Method 2: IKEv2 Tunnels using Pre-Shared Keys
- Configuring the IKEv2 Tunnel with Pre-Shared Keys (XML)
Managing IPsec Tunnels and Daemons

1. Prerequisites and Key Concepts

Before proceeding, familiarize yourself with the following terms and assumptions:

Supported AIX Versions: AIX 6.1, 7.1, 7.2, 7.3.

System IPs: For illustrative purposes, this guide uses 1.1.1.1 for one AIX system (Host1) and 2.2.2.2 for another AIX system (Host2). Replace these with your actual environment IPs.
Source and Destination System Matrix: Packet direction is crucial for IPsec configuration.
Table 1: Source and Destination IPs
| System | Packet direction | Source | Destination |
|---------------|------------------|-----------------|-----------------|
| On AIX (Host1)| Incoming | 2.2.2.2 (Host2) | 1.1.1.1 (Host1) |
| On AIX (Host1)| Outgoing | 1.1.1.1 (Host1) | 2.2.2.2 (Host2) |
| On AIX (Host2)| Incoming | 1.1.1.1 (Host1) | 2.2.2.2 (Host2) |
| On AIX (Host2)| Outgoing | 2.2.2.2 (Host2) | 1.1.1.1 (Host1) |

Interpret Table 1 from left to right. For example, the first row means: On the AIX system designated as Host1, when a packet is incoming, its source IP is 2.2.2.2 (Host2) and its destination IP is 1.1.1.1 (Host1).
Initiator and Responder:
- Initiator: The system that initiates the tunnel connection.
- Responder: The system that responds to the initiator's request.
- Either AIX system can act as the initiator. Tunnels can be activated by initiating communication (e.g., ping) from one system to the other, or by using the ike cmd=activate command on the AIX initiator.
IPsec Device Configuration: Ensure IPsec devices are configured and available on AIX.
- Verify with: lsdev -Cc ipsec. The output should show ipsec_v4 Available.
- If not available, configure using smitty ipsec4 (for IPv4) or smit ipsec6 (for IPv6):
  1. Launch smitty ipsec4.
  2. Select Start/Stop IP Security and press Enter.
  3. Select Start IP Security and press Enter.
  4. On the next screen, retain default settings and press Enter.
  5. A COMMAND STATUS screen showing ipsec_v4 Available indicates success.
IBM GSKit V8: Certificate generation relies on IBM GSKit V8 filesets.
You can verify the installed GSKit filesets using lpp command.
For example: The gsk8capicmd utility is typically located at /usr/opt/ibm/gsk8/bin/gsk8capicmd, often with a symbolic link at /usr/bin/gsk8capicmd.

2. Method 1: IKEv2 Tunnels using Certificates

This method involves generating digital certificates for authentication between AIX systems.

Generating Certificates on AIX with GSKit

The following steps outline the creation of key databases (kdbs) and certificates. AIX IPsec uses a key database named ikekey.kdb. All certificates and key database(kdbs) will initially be created in a directory named /GSK_CERTS and later moved to the appropriate system location.

The following is the set of key databases (kdbs) and certificates that we will be creating as part of the setup in this document.

image-20250601191823-1

Create the Key Database for Host1 (ikekey.kdb)
This kdb will store Host1's private key, its certificate, and the trusted Root CA certificate.
```
gsk8capicmd -keydb -create -db /GSK_CERTS/ikekey.kdb -pw 123456 -type cms -stash
```
(Note: "123456" is used for simplicity. Employ a strong password in production.)
This command creates ikekey.crl, ikekey.kdb, ikekey.rdb, and ikekey.sth.
Create the Root CA Key Database (rootCA.kdb)
This kdb will hold the Root Certificate Authority (CA) certificate and its private key.
```
gsk8capicmd -keydb -create -db /GSK_CERTS/rootCA.kdb -pw 123456 -type cms -stash
```
This creates rootCA.crl, rootCA.kdb, rootCA.rdb, and rootCA.sth.

Create the Root CA Certificate
Create the self-signed Root CA certificate within rootCA.kdb.

gsk8capicmd -cert -create -db /GSK_CERTS/rootCA.kdb -pw 123456 -size 1024 \
            -dn "C=IN,O=IBM,CN=ipsecroot" -label Root_CA_Full -default_cert yes -ca true

Extract the Public Part of the Root CA
AIX IPsec requires only the public part of the Root CA in its ikekey.kdb to trust certificates signed by it.
```
gsk8capicmd -cert -extract -db /GSK_CERTS/rootCA.kdb -stashed -label Root_CA_Full \
            -format ascii -target /GSK_CERTS/rootCA_Pub.arm
```

Add the Public Root CA to Host1's ikekey.kdb

gsk8capicmd -cert -add -db /GSK_CERTS/ikekey.kdb -stashed -label Root_CA \
            -file /GSK_CERTS/rootCA_Pub.arm -format ascii -trust enable

You can verify by listing certificates:
gsk8capicmd -cert -list -db /GSK_CERTS/ikekey.kdb -pw 123456

Create a Certificate Signing Request (CSR) for Host1
This CSR will be signed by the Root CA. Test_Cert2 is used as the label for Host1's certificate in this example.

gsk8capicmd -certreq -create -db /GSK_CERTS/ikekey.kdb -pw 123456 -label Test_Cert2 \
            -dn "C=IN,ST=KA,L=BA,O=IBM,OU=ISL,CN=test2" -size 1024 \
            -file /GSK_CERTS/cert2_csr.arm

Sign Host1's Certificate Request with the Root CA
Use the full Root CA (from rootCA.kdb where the private key resides) to sign the CSR.

gsk8capicmd -cert -sign -db /GSK_CERTS/rootCA.kdb -pw 123456 -label Root_CA_Full \
            -target /GSK_CERTS/Test_Cert2.cer -format ascii -expire 100 \
            -file /GSK_CERTS/cert2_csr.arm

Receive the Signed Certificate into Host1's ikekey.kdb
```
gsk8capicmd -cert -receive -file /GSK_CERTS/Test_Cert2.cer -db /GSK_CERTS/ikekey.kdb \
            -pw 123456 -type cms -format ascii
```
ikekey.kdb now contains Root_CA (public part) and Test_Cert2 (Host1's signed certificate).
Move Host1's Key Database Files
Move the ikekey.* files for Host1 to the standard AIX IPsec location.
```
/usr/bin/mv /GSK_CERTS/ikekey* /etc/security/
```

Create a Certificate for the Remote AIX System (Host2)
We will create this certificate on the current AIX system (Host1) and sign it using the same Root CA. For organizational purposes, we'll use a temporary KDB named Rikekey.kdb (where 'R' signifies remote). Test_Cert1 is used as the label for Host2's certificate.

gsk8capicmd -keydb -create -db /GSK_CERTS/Rikekey.kdb -pw 123456 -type cms -stash

Add the public Root CA to Rikekey.kdb:

gsk8capicmd -cert -add -db /GSK_CERTS/Rikekey.kdb -stashed -label Root_CA \
            -file /GSK_CERTS/rootCA_Pub.arm -format ascii -trust enable

Create CSR for Host2 (Test_Cert1):

gsk8capicmd -certreq -create -db /GSK_CERTS/Rikekey.kdb -pw 123456 -label Test_Cert1 \
            -dn "C=IN,ST=KA,L=BA,O=IBM,OU=ISL,CN=test1" -size 1024 \
            -file /GSK_CERTS/cert1_csr.arm

Sign Host2's CSR with the Root CA:

gsk8capicmd -cert -sign -db /GSK_CERTS/rootCA.kdb -pw 123456 -label Root_CA_Full \
            -target /GSK_CERTS/Test_Cert1.cer -format ascii -expire 100 \
            -file /GSK_CERTS/cert1_csr.arm

Receive Host2's signed certificate into Rikekey.kdb:

gsk8capicmd -cert -receive -file /GSK_CERTS/Test_Cert1.cer -db /GSK_CERTS/Rikekey.kdb \
            -pw 123456 -type cms -format ascii

Distribute Certificate Files
- The certificate generation commands (steps 1-10) are typically run on one system (e.g., the designated initiator, Host1).
- The ikekey.* files (containing Test_Cert2) from /GSK_CERTS/ are already moved to /etc/security/ on the initiator (Host1).
- The Rikekey.* files (containing Test_Cert1) from /GSK_CERTS/ need to be securely copied to the /etc/security/ directory of the responder (Host2).
- Crucially, on Host2, rename the copied Rikekey.* files to ikekey.* (e.g., mv Rikekey.kdb ikekey.kdb, mv Rikekey.sth ikekey.sth, etc.).

Configuring the IKEv2 Tunnel with Certificates (XML)

Create an XML file (e.g., AIX-AIX-Certificates-IKEv2.xml) on each AIX system. The following is a sample XML for one of the systems (e.g., the initiator). You will need a corresponding XML on the other system with local and remote identities swapped.

Important Note on XML Configuration:
The sample XML below configures a system with local IP 2.2.2.2 (using certificate CN=test2) and remote IP 1.1.1.1 (using certificate CN=test1).

IKELocalIdentity and IPSecLocalIdentity must reflect the IP address and certificate distinguished name (DN) of the AIX system where this XML file is loaded.
IKERemoteIdentity and IPSecRemoteIdentity must reflect the IP address and certificate DN of the peer AIX system.
Adjust all IP addresses and ASN1_DN values in the XML to match your specific environment and the certificates you generated (Test_Cert2 for the local host, Test_Cert1 for the remote host, as per the generation steps).

<?xml version="1.0"?>
<AIX_VPN Version="2.1">
   <IKEProtection
         IKE_Flags_UseCRL="No"
         IKE_KeyOverlap="10"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMinMinutes="1"
         IKE_Role="Both"
         IKE_Version="2"
         IKE_XCHGMode="None">
      <IKETransform IKE_Encryption="AES-CBC-256"/>
      <IKETransform IKE_PRF="PRF_HMAC_SHA1"/>
      <IKETransform IKE_Hash="SHA"/>
      <IKETransform IKE_DHGroup="2"/>
      <IKETransform IKE_AuthenticationMethod="RSA_signatures"/>
      <IKETransform IKE_KeyRefreshMinutes="10"/>
   </IKEProtection>
   <IKETunnel
         IKE_Flags_AutoStart="No"
         IKE_Flags_MakeRuleWithOptionalIP="Yes"
         IKE_ProtectionRef="P1Pol"
         IKE_TunnelName="P1">
      <IKELocalIdentity>
         <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test2">
            <IPV4_Address Value="2.2.2.2"/>
         </ASN1_DN>
      </IKELocalIdentity>
      <IKERemoteIdentity>
         <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test1">
            <IPV4_Address Value="1.1.1.1"/>
         </ASN1_DN>
      </IKERemoteIdentity>
   </IKETunnel>
   <IPSecProposal IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Authentication="HMAC-SHA"
            ESP_EncapsulationMode="Transport"
            ESP_Encryption="ESP_AES_256"
            ESP_ExtendedSeqNum="0"
            ESP_KeyRefreshKB="0"
            ESP_KeyRefreshMinutes="2"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Flags_UseCommitBit="No"
         IPSec_Flags_UseLifeSize="No"
         IPSec_InitiatorDHGroup="0"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_ResponderDHGroup="NO_PFS"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMinMinutes="1"
         IPSec_Role="Both"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_Flags_AutoStart="No"
         IPSec_Flags_OnDemand="No"
         IPSec_ProtectionRef="P2Pol"
         IPSec_TunnelName="P2">
      <IPSecLocalIdentity EndPort="65535" Port="0" Protocol="0">
         <IPV4_Address_Range From_IPAddr="2.2.2.2" To_IPAddr="2.2.2.2"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity EndPort="65535" Port="0" Protocol="0">
         <IPV4_Address_Range From_IPAddr="1.1.1.1" To_IPAddr="1.1.1.1"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

Load this XML file into the IKE database (see "Managing IPsec Tunnels and Daemons" section for commands).

3. Method 2: IKEv2 Tunnels using Pre-Shared Keys

This method uses a shared secret (a pre-shared key) for authentication, which is simpler to set up than certificates but generally considered less secure for larger deployments.

Configuring the IKEv2 Tunnel with Pre-Shared Keys (XML)

Create an XML file (e.g., AIX-AIX-PRESHARED-IKEV2.xml) on each AIX system. The pre-shared key (Value="12345" in the example) must be identical on both systems.

Important Note on XML Configuration:
The sample XML below is for Host1 (IP 1.1.1.1) connecting to Host2 (IP 2.2.2.2).

IKELocalIdentity and IPSecLocalIdentity must reflect the IP address of the AIX system where this XML file is loaded.
IKERemoteIdentity, IPSecRemoteIdentity, and IKEPresharedRemoteID must reflect the IP address of the peer AIX system.
A corresponding XML with swapped local/remote IP addresses will be needed on Host2.

<?xml version="1.0"?>
<AIX_VPN Version="2.0">
   <IKEProtection
         IKE_Role="Both"
         IKE_Version="2"
         IKE_XCHGMode="Main"
         IKE_KeyOverlap="10"
         IKE_Flags_UseCRL="No"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinMinutes="1">
      <IKETransform
            IKE_Encryption="AES-CBC-256"
            IKE_PRF="PRF_SHA2_256"
            IKE_Hash="SHA2_256"
            IKE_DHGroup="2"
            IKE_AuthenticationMethod="Preshared_key"/>
   </IKEProtection>
   <IKETunnel
         IKE_TunnelName="P1"
         IKE_ProtectionRef="P1Pol"
         IKE_Flags_AutoStart="No"
         IKE_Flags_MakeRuleWithOptionalIP="Yes">
      <IKELocalIdentity>
         <IPV4_Address Value="1.1.1.1"/>
      </IKELocalIdentity>
      <IKERemoteIdentity>
         <IPV4_Address Value="2.2.2.2"/>
      </IKERemoteIdentity>
   </IKETunnel>
   <IKEPresharedKey Value="12345" Format="ASCII">
      <IKEPresharedRemoteID>
         <PK_IPV4_Address Value="2.2.2.2"/>
      </IKEPresharedRemoteID>
   </IKEPresharedKey>
   <IPSecProposal IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Encryption="ESP_AES_256"
            ESP_KeyRefreshKB="0"
            ESP_Authentication="HMAC-SHA"
            ESP_ExtendedSeqNum="0"
            ESP_EncapsulationMode="Transport"
            ESP_KeyRefreshMinutes="480"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Role="Both"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_InitiatorDHGroup="5"
         IPSec_ResponderDHGroup="GROUP_5"
         IPSec_Flags_UseLifeSize="No"
         IPSec_Flags_UseCommitBit="No"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinMinutes="1"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_TunnelName="P2"
         IPSec_ProtectionRef="P2Pol"
         IPSec_Flags_OnDemand="No"
         IPSec_Flags_AutoStart="No">
      <IPSecLocalIdentity>
         <IPV4_Address_Range To_IPAddr="1.1.1.1" From_IPAddr="1.1.1.1"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity>
         <IPV4_Address_Range To_IPAddr="2.2.2.2" From_IPAddr="2.2.2.2"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

Load this XML file into the IKE database:

/usr/sbin/ikedb -x
/usr/sbin/ikedb -p AIX-AIX-PRESHARED-IKEV2.xml

(The ikedb -x command unloads any existing configuration.)

4. Managing IPsec Tunnels and Daemons

Once the XML configuration is prepared and certificate files (if used) are in place, manage the IPsec daemons and tunnels using these commands:

Starting IKEv2 Daemons (Recommended for IKEv2-only):
When using only IKEv2, you don't need to start isakmpd (IKEv1) or iked (broker) daemons. Only tmd (Tunnel Manager Daemon), ikev2d (IKEv2 Daemon), and cpsd (Complex Parsing Subsystem Daemon) are required.
```
stopsrc -g ike
# Add a short delay here if scripting, e.g., sleep 5
startsrc -s tmd
startsrc -s ikev2d
startsrc -s cpsd
```
Loading/Unloading XML Configuration:
- Unload current XML configuration:
```
/usr/sbin/ikedb -x
```
- Load new XML configuration (replace <sample.xml> with your filename):
```
/usr/sbin/ikedb -p <sample.xml>
```
Viewing Loaded XML Configuration:
```
/usr/sbin/ikedb -g
```
General IPsec Daemon Management (Starts all IKE daemons including IKEv1):
- Stop all IPsec daemons:
```
stopsrc -g ike
```
- Start all IPsec daemons:
```
startsrc -g ike
```
(Note: Ensure a brief delay between stopping and starting daemons, especially in scripts.)

Starting Daemons Individually (Alternative):

startsrc -s tmd
startsrc -s ikev2d
# startsrc -s iked  (if broker is needed for IKEv1/IKEv2 interop)
# startsrc -s isakmpd (for IKEv1)

Listing IPsec Filters:
```
lsfilt -aO
```
Listing Tunnel Definitions:
```
lstun -aO
```
Activating Tunnels (run on Initiator):
Replace P1 with the IKE_TunnelName from your XML.
```
ike cmd=activate tunnel P1
```
Alternatively, if IKE_Flags_AutoStart is Yes in the XML and appropriate traffic is initiated, tunnels may start automatically.
Listing Active Tunnels:
```
ike cmd=list
```
Removing/Tearing Down Tunnels:
- Remove all active IKE tunnels:
```
ike cmd=remove all
```
- Remove tunnel definitions (filter rules might persist):
```
rmtun -v 4 -t all
```

Additional Information:

IBM AIX Knowledge Center – IPsec Overview
https://www.ibm.com/docs/en/aix/7.2?topic=security-internet-protocol-ipsec
IBM AIX – Configuring IP Security (IPsec)
https://www.ibm.com/docs/en/aix/7.2?topic=services-configuring-ip-security
IBM GSKit (Global Security Kit) Documentation
https://www.ibm.com/docs/en/ibm-gskit
AIX ikedb Command Reference
https://www.ibm.com/docs/en/aix/7.2?topic=i-ikedb-command

--------------------------------------------------------------------------------------------------------------------------------------------Authors: Mansi Jaiswal,Tilak R Nayak
Operating System: AIX and VIOS
Hardware: Power
Feedback: aix_feedback@wwpdl.vnet.ibm.com, mansjais@in.ibm.com, tilak-r.nayak@in.ibm.com,

[{"Type":"MASTER","Line of Business":{"code":"LOB64","label":"Printer Support"},"Business Unit":{"code":"BU065","label":"Technical Support Services"},"Product":{"code":"PT0055","label":"0273"},"ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":""}]

Product Synonym

AIX IPsec,IBM AIX IKEv1,IBM AIX IKEv2,IPsec on AIX,IBM AIX 6.1/7.1/7.2/7.3 IPsec

Tips