Introduction
Terminologies and assumptions
IKEv1 tunnels between AIX and Windows using pre-shared keys
IKEv1 tunnels between AIX and Windows using certificates
Summary
Related topics
Downloadable resources
Comments

IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

IKEv1 IPsec tunnels between AIX 6.1 or later versions and Windows 2012

IKEv1 tunnels using certificates and pre-shared keys

Shailaja Mallya

and Shivaprasad Manuwacharya

Published on December 01, 2017

Content series:

This content is part # of # in the series: IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

https://www.ibm.com/developerworks/library/?series_title_by=**auto**

Stay tuned for additional content in this series.

This content is part of the series:IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

Stay tuned for additional content in this series.

Internet Protocol Security (IPsec) as its name suggests provides security at the Internet Protocol (IP) layer. This tutorial requires a basic understanding of what IPsec is and how it can be used to protect data over the network. You can refer to Knowledge Center or other resources such as wiki to get acquainted with IPsec.

This tutorial discusses two different methods of establishing IPsec tunnels between IBM® AIX® (6.1 / 7.1 / 7.2) and Microsoft Windows 2012 systems. The methods involve pre-shared keys and certificates using IKEv1 between AIX and Windows systems. Table 1 provides a short description of the main topics covered in this tutorial and what they entail.

Table 1. Main topics covered

Content	Description
Terminologies and assumptions	This section provides a note on the important terms used in this tutorial and some assumptions on which the setup is configured.
IKEv1 tunnels using pre-shared keys	This section explains the required Internet Key Exchange (IKE) XML file to be updated on AIX. It also has a detailed explanation for using GUI on Windows 2012 for IKEv1.
IKEv1 tunnels using certificates	This section explains the required IKE XML file to be updated on AIX. Almost all the steps to be run on Windows 2012 are the same as mentioned in the "IKEv1 tunnels using pre-shared key" section. The two methods differ by a single step on Windows. Only this single step has been highlighted in this section.

Terminologies and assumptions

This section explains a few terms, such as initiator and responder, and highlights a few assumptions that this tutorial is based on.

In this tutorial, for illustration purposes, we have mentioned the AIX system's IP as 1.1.1.1 and the Windows system's IP as 2.2.2.2. These need to be replaced with the appropriate IPs in your environment.

Source and destination system matrix:

Table 2. Source and destination IPs

System	Packet direction	Source	Destination
On AIX	Incoming	2.2.2.2 (Windows)	1.1.1.1 (AIX)
On AIX	Outgoing	1.1.1.1 (AIX)	2.2.2.2 (Windows)
On Windows	Incoming	1.1.1.1 (AIX)	2.2.2.2 (Windows)
On Windows	Outgoing	2.2.2.2 (Windows)	1.1.1.1 (AIX)

The source is always the system that creates and sends a packet. The destination is always the system that receives it. This table (Table 2) needs to be read from left to right. For example, the first row is interpreted as follows:

'On AIX' system, when a packet is 'incoming', the source mentioned in the packet is '2.2.2.2 (Windows)' and the destination mentioned in this packet is '1.1.1.1 (AIX)'

Initiator is the system that initiates a tunnel connection and the responder is the system that responds to the initiator's request.
Either the Windows or the AIX system can be the initiator. You can activate the tunnels from Windows by pinging or communicating with AIX. Or, you can run the ike cmd=activate command on AIX and the tunnels will be active. If one of these methods don't work, try the other.
For this setup, you need to have the IPsec devices configured on AIX.
Running the lsdev -Cc ipsec command on AIX shows the ipsec_v4 device as available. Else run smitty ipsec4.
In the smitty panel:
1. Select Start/Stop IP Security and press Enter.
2. Select Start IP Security and press Enter.
3. On the next screen, retain the default settings and press Enter.
4. On the COMMAND STATUS screen, the message, ipsec_v4 Available, indicates successful configuration of the device.

IKEv1 tunnels between AIX and Windows using pre-shared keys

The following XML file needs to be created on the AIX system. Let's name it AIX-Windows-PreShared-IKEv1.xml. Add this XML file to the IKE database on AIX using the following command:

/usr/sbin/ikedb -x
/usr/sbin/ikedb -p AIX-Windows-PreShared-IKEv1.xml


<?xml version="1.0"?>
<AIX_VPN
      Version="2.1">
   <IKEProtection
         IKE_Role="Both"
	       IKE_Version="1"
         IKE_XCHGMode="Main"
         IKE_KeyOverlap="10"
         IKE_Flags_UseCRL="No"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinMinutes="60">
      <IKETransform
            IKE_Encryption="3DES-CBC"
            IKE_Hash="SHA"
            IKE_DHGroup="1"
            IKE_AuthenticationMethod="Preshared_key"
            IKE_KeyRefreshMinutes="240"/>
   </IKEProtection>
   <IKETunnel
         IKE_TunnelName="P1"
         IKE_ProtectionRef="P1Pol"
         IKE_Flags_AutoStart="Yes"
         IKE_Flags_MakeRuleWithOptionalIP="No">
      <IKELocalIdentity>
         <IPV4_Address
               Value="1.1.1.1"/>
      </IKELocalIdentity>
      <IKERemoteIdentity>
         <IPV4_Address
               Value="2.2.2.2"/>
      </IKERemoteIdentity>
   </IKETunnel>
   <IKEPresharedKey
         Value="12345"
         Format="ASCII">
      <IKEPresharedRemoteID>
         <PK_IPV4_Address
               Value="2.2.2.2"/>
      </IKEPresharedRemoteID>
   </IKEPresharedKey>
   <IPSecProposal
         IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Encryption="ESP_3DES"
            ESP_KeyRefreshKB="0"
            ESP_Authentication="HMAC-SHA"
            ESP_ExtendedSeqNum="0"
            ESP_EncapsulationMode="Transport"
            ESP_KeyRefreshMinutes="30"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Role="Both"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_InitiatorDHGroup="0"
         IPSec_ResponderDHGroup="NO_PFS"
         IPSec_Flags_UseLifeSize="No"
         IPSec_Flags_UseCommitBit="No"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinMinutes="30"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_TunnelName="P2"
         IPSec_ProtectionRef="P2Pol"
         IPSec_Flags_OnDemand="No"
         IPSec_Flags_AutoStart="Yes">
      <IPSecLocalIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="1.1.1.1"
               From_IPAddr="1.1.1.1"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="2.2.2.2"
               From_IPAddr="2.2.2.2"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

To keep it simple, let's start only the IKEv1 daemon using the following commands:

stopsrc -g ike
startsrc -s tmd ; startsrc -s isakmpd

When we start only isakmpd (IKEv1 daemon), we don't need to start the ikev2d (IKEv2) or iked (broker) daemons. Only the tmd daemon (tunnel manager daemon) is required.

You need to perform the following steps on the Windows 2012 system to create the IKEv1 IPsec policy. The overall tasks can be divided into the following five high-level steps:

Add IPsec snap-in to the Microsoft Management Console (MMC)
Create the IPsec policy.
Create the IPsec filter rules.
1. Define IP tunneling attributes.
2. Define authentication methods.
3. Define filter actions.
Specify other tunnel-related settings.
Assign the IPsec policy.

Add IPsec snap-in to the MMC

Perform the following steps to add an IPsec snap-in to the MMC:

Open the MMC.
Click File -> Add/Remove Snap-in.
Figure 1. MMC File menu option
View image at full size
From the Available snap-ins list, select IP Security Policy Manager and click Add. Then, click OK.
Figure 2. Adding the IP Security Policy Management snap-in
View image at full size
In the Select Computer or Domain dialog box, select Local computer and click Finish.
Figure 3. Selecting the computer or domain to manage snap-in
View image at full size
Clicking Finish takes you back to the Add or Remove Snap-ins dialog box. Click OK.

Create the IPsec policy

Perform the following steps to create the IPsec policy:

In the main console, right-click IP Security Policies on Local Computer and then click Create IP Security Policy.
Figure 4. Right-click menu options for IPsecurity policies snap-in
View image at full size
Figure 5. Welcome page of IP Security Policy Wizard
View image at full size
On the welcome page of the IP Security Policy Wizard, click Next. On the IP Security Policy Name page, click Next to create a policy named Policy1.
Figure 6. IP security policy naming
View image at full size
On the Request for Secure Communication page, click Next.
Figure 7. IP security policy wizard
View image at full size
On the wizard completion page, select the Edit properties check box and click Finish.
Figure 8. IP Security Policy Wizard completion
View image at full size

Create the IPsec filter rules

Perform the following steps to create the IPsec filter rules:

To set the policy properties, select the Use Add Wizard check box at the lower-right corner and then click Add.
Figure 9. Adding policy properties to the newly created policy
View image at full size
On the welcome page of the Security Rule Wizard, click Next.
Figure 10. Security Rule Wizard welcome page
View image at full size
1. Define the IP tunneling attributes.
  1. Select the This rule does not specify a tunnel option and click Next.
    Figure 11. Tunnel Endpoint page
    View image at full size
  2. On the Network Type page, select All network connections and click Next.
    Figure 12. Selecting the network type
    View image at full size
  3. On the IP Filter List page, click Add.
    Figure 13. Adding IP filters
    View image at full size
  4. You can give a meaningful name to the filter list. In this example, we have entered Policy 1 IP Filter List 1. Note that Use Add Wizard check box is selected and click Add.
    Figure 14. Creating an IP filter policy
    View image at full size
  5. On the welcome page of the IP Filter Wizard, click Next.
    Figure 15. IP Filter Wizard welcome page
    View image at full size
  6. On the IP Filter Description and Mirrored property page, select the Mirrored. Match packets with the exact opposite source and destination option to apply the same rule on both the incoming and outgoing packets on Windows. Then click Next.
    Figure 16. IP Filter Description and Mirrored property page
    View image at full size
  7. On the IP Traffic Source page, from the Source address drop-down list, select A specific IP Address or Subnet.
    Figure 17. IP traffic source selection
    View image at full size
    Also, provide the source IP below the drop-down list (the partner system's IP = AIX system's IP) and click Next.
  8. On the IP Traffic Destination page, from the Destination address drop-down list, select A specific IP Address or subnet, add the required destination IP, and click Next.
    Figure 18. IP traffic destination selection
    View image at full size
  9. On the IP Protocol Type page, in this example, we are selecting Any as the protocol type. You can select the required protocol as per your needs. After that, click Next.
    Figure 19. IP traffic protocol type
    View image at full size
  10. On the wizard completion page, click Finish.
    Figure 20. IP Filter Wizard completion
    View image at full size
  11. Notice that you are directed to the IP Filter List dialog box. Click OK in dialog box.
    Figure 21. IP Filter List dialog box
    View image at full size
2. Define IP filter actions.
  1. On the IP Filter List page, select Policy 1 IP Filter List 1 and click Next.
    Figure 22. IP Filter List page
    View image at full size
  2. On the Filter Action page, if you see an existing filter action, you can select it and click Edit. Else you can add a new filter action if you do not want to modify an existing one. In this example, we are adding a new one. Notice that the Use Add Wizard check box is checked. Click Add.
    Figure 23. Filter Action page
    View image at full size
  3. As Use Add Wizard is selected, the IP Security Filter Action Wizard opens. Click Next on the welcome page.
    Figure 24. IP Security Filter Action Wizard welcome page
    View image at full size
  4. We will change the default name New filter action to Policy 1 filter action 1. Then, click Next.
    Figure 25. Filter Action Name page
    View image at full size
  5. On the Filter Action General Options page, select Negotiate security and click Next.
    Figure 26. Filter Action General Options page
    View image at full size
  6. On the Communicating with computers that do not support IPsec page, select Do not allow unsecured communication and click Next.
    Figure 27. IP filter action on communicating with IPsec unaware systems
    View image at full size
  7. On the IP Traffic Security page, select Integrity and encryption and click Next.
    Figure 28. IP Traffic Security page
    View image at full size
  8. On the wizard completion page, click Finish.
    Figure 29. IP filter action completion page
    View image at full size
  9. Notice that you are directed to the Filter Action page. On this page, select the new filter action, Policy 1 filter action 1. Notice that the Use Add Wizard check box is selected and click Next.
    Figure 30. Back at the Filter Action page
    View image at full size
3. Define authentication methods.
  1. On the Authentication Method page, select Use this string to protect the key exchange (pre-shared key) and click Next. Note that this key is the same one that is mentioned in the XML file updated on AIX.
    Figure 31. Selecting the authentication method
    View image at full size
  2. On the Security Rule Wizard completion page, click Finish.
    Figure 32. Security rule completion page
    View image at full size
  3. In the Policy1 Properties dialog box, click the General tab.
    Figure 33. Policy1 Properties dialog box
    View image at full size
4. Specify other tunnel-related settings.
  1. On the General tab, click Settings to specify additional settings.
    Figure 34. General tab of Policy1 properties
    View image at full size
  2. In the Key Exchange Settings dialog box, set the time after which a new key must be regenerated. Then click Methods.
    Figure 35. Policy properties: Key exchange settings
    View image at full size
  3. In the Key Exchange Security Methods dialog box, select the rule that is displayed and click Edit.
    You can change the algorithms used for encryption and integrity checking and you can set the Diffie-Hellman group level here. Make sure that the IKE security algorithms match the ones mentioned in the XML file on AIX. The settings we use are the ones shown in the following screen captures.
    After the required algorithms have been chosen, click OK and proceed to the next steps.
    Figure 36. Key exchange settings: Key Exchange Security Methods dialog box
    View image at full size
    Figure 37. Key Exchange Security Methods dialog box
    View image at full size
  4. In the Key Exchange Security Methods dialog box, click OK. Also, in the Policy1 Properties dialog box, click OK.
    Figure 38. Back to Policy1 Properties
    View image at full size
5. Assign the IPsec policy.
  1. After the policy is created, notice that it is not initially assigned to the system.
    Figure 39. Console root with IP security snap-in
    View image at full size
  2. Right-click Policy1 and click Assign.
    Figure 40. Assigning the IP security Policy
    View image at full size
  3. Notice that the Policy Assigned state now changes to Yes from No.
    Figure 41. IP security Policy assigned
    View image at full size

IKEv1 tunnels between AIX and Windows using certificates

Follow the certificate generation steps on the AIX system and the certificate import steps on Windows explained in their respective sections earlier.

Refer to the tutorial Generating certificates in AIX and importing certificates to Windows for IKE IPsec tunnels for creating the certificates on AIX and importing to the Windows operating system before proceeding with the following steps.

You need to create the following XML file in the AIX system. Let us name it AIX-Windows-Certificates-IKEv1.xml. Add this XML to the IKE database on AIX using the following commands:

/usr/sbin/ikedb -x
/usr/sbin/ikedb -p AIX-Windows-Certificates-IKEv1.xml

<?xml version="1.0"?>
<AIX_VPN
      Version="2.1">
   <IKEProtection
         IKE_Role="Both"
         IKE_Version="1"
         IKE_XCHGMode="Main"
         IKE_KeyOverlap="10"
         IKE_Flags_UseCRL="No"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinMinutes="60">
      <IKETransform
            IKE_Encryption="3DES-CBC"
            IKE_Hash="SHA"
            IKE_DHGroup="1"
            IKE_AuthenticationMethod="RSA_signatures"
            IKE_KeyRefreshMinutes="240"/>
   </IKEProtection>
	<IKETunnel
         IKE_TunnelName="P1"
         IKE_ProtectionRef="P1Pol"
         IKE_Flags_AutoStart="Yes"
         IKE_Flags_MakeRuleWithOptionalIP="Yes">
         <IKELocalIdentity>
             <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test2">
         <IPV4_Address
               Value="1.1.1.1"/>
        </ASN1_DN>
      </IKELocalIdentity>
      <IKERemoteIdentity>
             <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test1">
         <IPV4_Address
               Value="2.2.2.2"/>
        </ASN1_DN>
      </IKERemoteIdentity>

   </IKETunnel>
   <IPSecProposal
         IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Encryption="ESP_3DES"
            ESP_KeyRefreshKB="0"
            ESP_Authentication="HMAC-SHA"
            ESP_ExtendedSeqNum="0"
            ESP_EncapsulationMode="Transport"
            ESP_KeyRefreshMinutes="30"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Role="Both"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_InitiatorDHGroup="0"
         IPSec_ResponderDHGroup="NO_PFS"
         IPSec_Flags_UseLifeSize="No"
         IPSec_Flags_UseCommitBit="No"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinMinutes="30"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_TunnelName="P2"
         IPSec_ProtectionRef="P2Pol"
         IPSec_Flags_OnDemand="No"
         IPSec_Flags_AutoStart="Yes">
      <IPSecLocalIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="1.1.1.1"
               From_IPAddr="1.1.1.1"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="2.2.2.2"
               From_IPAddr="2.2.2.2"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

To keep it simple, let’s start only the IKEv1 daemon using the following commands:

stopsrc -g ike
startsrc -s tmd ; startsrc -s isakmpd ; startsrc -s cpsd

When we start only isakmpd (IKEv1 daemon), we don’t need to start the ikev2d (IKEv2) or iked (broker) daemons. Only the tmd daemon is required.

Perform all the steps mentioned for the Windows system in the section, IKEv1 tunnels between AIX and windows using pre-shared keys except the steps showed in Figure 31. Instead of using pre-shared specify the settings as shown in Figure 42 for the certificates.

Figure 42. Authentication Method page in Security Rule Wizard

View image at full size

Summary

This tutorial explained how to establish IKEv1 tunnels between AIX 6.1/ 7.1/ 7.2 and Windows 2012 systems using certificates and pre-shared keys. The following tutorial explains how to establish tunnels using IKEv2 between AIX and Windows Operating system:

IKEv2 IPsec tunnels between AIX 6.1 or later versions and Windows 2012

Part 2 and part 3 in this series, together, explain four different ways to establish tunnels between AIX 6.1/ 7.1/ 7.2 and Windows 2012 systems. These four methods are setting up IKEv1 and IKEv2 tunnels using pre-shared keys and certificates between AIX and Windows.

The configuration steps for Windows (explained in this tutorial) were tried in the lab and worked for the test team. These steps are not endorsed by Microsoft nor by IBM. You can consider the steps mentioned in this tutorial as a guidance to get you started. There is always light at the end of the tunnel. So, enjoy configuring tunnels!

Contents

IKEv1 IPsec tunnels between AIX 6.1 or later versions and Windows 2012

Content series:

This content is part # of # in the series: IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

This content is part of the series:IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

Table 1. Main topics covered

Terminologies and assumptions

Table 2. Source and destination IPs

IKEv1 tunnels between AIX and Windows using pre-shared keys

Add IPsec snap-in to the MMC

Figure 1. MMC File menu option

Figure 2. Adding the IP Security Policy Management snap-in

Figure 3. Selecting the computer or domain to manage snap-in

Create the IPsec policy

Figure 4. Right-click menu options for IPsecurity policies snap-in

Figure 5. Welcome page of IP Security Policy Wizard

Figure 6. IP security policy naming

Figure 7. IP security policy wizard

Figure 8. IP Security Policy Wizard completion

Create the IPsec filter rules

Figure 9. Adding policy properties to the newly created policy

Figure 10. Security Rule Wizard welcome page

Figure 11. Tunnel Endpoint page

Figure 12. Selecting the network type

Figure 13. Adding IP filters

Figure 14. Creating an IP filter policy

Figure 15. IP Filter Wizard welcome page

Figure 16. IP Filter Description and Mirrored property page

Figure 17. IP traffic source selection

Figure 18. IP traffic destination selection

Figure 19. IP traffic protocol type

Figure 20. IP Filter Wizard completion

Figure 21. IP Filter List dialog box

Figure 22. IP Filter List page

Figure 23. Filter Action page

Figure 24. IP Security Filter Action Wizard welcome page

Figure 25. Filter Action Name page

Figure 26. Filter Action General Options page

Figure 27. IP filter action on communicating with IPsec unaware systems

Figure 28. IP Traffic Security page

Figure 29. IP filter action completion page

Figure 30. Back at the Filter Action page

Figure 31. Selecting the authentication method

Figure 32. Security rule completion page

Figure 33. Policy1 Properties dialog box

Figure 34. General tab of Policy1 properties

Figure 35. Policy properties: Key exchange settings

Figure 36. Key exchange settings: Key Exchange Security Methods dialog box

Figure 37. Key Exchange Security Methods dialog box

Figure 38. Back to Policy1 Properties

Figure 39. Console root with IP security snap-in

Figure 40. Assigning the IP security Policy

Figure 41. IP security Policy assigned

IKEv1 tunnels between AIX and Windows using certificates

Figure 42. Authentication Method page in Security Rule Wizard

Summary

Related topics

Downloadable resources