Contents


IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

IKEv1 IPsec tunnels between AIX 6.1 or later versions and Windows 2012

IKEv1 tunnels using certificates and pre-shared keys

Comments

Content series:

This content is part # of # in the series: IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

Stay tuned for additional content in this series.

This content is part of the series:IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2

Stay tuned for additional content in this series.

Internet Protocol Security (IPsec) as its name suggests provides security at the Internet Protocol (IP) layer. This tutorial requires a basic understanding of what IPsec is and how it can be used to protect data over the network. You can refer to Knowledge Center or other resources such as wiki to get acquainted with IPsec.

This tutorial discusses two different methods of establishing IPsec tunnels between IBM® AIX® (6.1 / 7.1 / 7.2) and Microsoft Windows 2012 systems. The methods involve pre-shared keys and certificates using IKEv1 between AIX and Windows systems. Table 1 provides a short description of the main topics covered in this tutorial and what they entail.

Table 1. Main topics covered
ContentDescription
Terminologies and assumptions This section provides a note on the important terms used in this tutorial and some assumptions on which the setup is configured.
IKEv1 tunnels using pre-shared keys This section explains the required Internet Key Exchange (IKE) XML file to be updated on AIX. It also has a detailed explanation for using GUI on Windows 2012 for IKEv1.
IKEv1 tunnels using certificates This section explains the required IKE XML file to be updated on AIX. Almost all the steps to be run on Windows 2012 are the same as mentioned in the "IKEv1 tunnels using pre-shared key" section. The two methods differ by a single step on Windows. Only this single step has been highlighted in this section.

Terminologies and assumptions

This section explains a few terms, such as initiator and responder, and highlights a few assumptions that this tutorial is based on.

  • In this tutorial, for illustration purposes, we have mentioned the AIX system's IP as 1.1.1.1 and the Windows system's IP as 2.2.2.2. These need to be replaced with the appropriate IPs in your environment.
  • Source and destination system matrix:
    Table 2. Source and destination IPs
    SystemPacket directionSourceDestination
    On AIXIncoming2.2.2.2 (Windows)1.1.1.1 (AIX)
    On AIXOutgoing1.1.1.1 (AIX)2.2.2.2 (Windows)
    On WindowsIncoming1.1.1.1 (AIX)2.2.2.2 (Windows)
    On WindowsOutgoing2.2.2.2 (Windows)1.1.1.1 (AIX)

    The source is always the system that creates and sends a packet. The destination is always the system that receives it. This table (Table 2) needs to be read from left to right. For example, the first row is interpreted as follows:

    'On AIX' system, when a packet is 'incoming', the source mentioned in the packet is '2.2.2.2 (Windows)' and the destination mentioned in this packet is '1.1.1.1 (AIX)'

  • Initiator is the system that initiates a tunnel connection and the responder is the system that responds to the initiator's request.

    Either the Windows or the AIX system can be the initiator. You can activate the tunnels from Windows by pinging or communicating with AIX. Or, you can run the ike cmd=activate command on AIX and the tunnels will be active. If one of these methods don't work, try the other.

  • For this setup, you need to have the IPsec devices configured on AIX.

    Running the lsdev -Cc ipsec command on AIX shows the ipsec_v4 device as available. Else run smitty ipsec4.

    In the smitty panel:

    1. Select Start/Stop IP Security and press Enter.
    2. Select Start IP Security and press Enter.
    3. On the next screen, retain the default settings and press Enter.
    4. On the COMMAND STATUS screen, the message, ipsec_v4 Available, indicates successful configuration of the device.

IKEv1 tunnels between AIX and Windows using pre-shared keys

The following XML file needs to be created on the AIX system. Let's name it AIX-Windows-PreShared-IKEv1.xml. Add this XML file to the IKE database on AIX using the following command:

/usr/sbin/ikedb -x
/usr/sbin/ikedb -p AIX-Windows-PreShared-IKEv1.xml


<?xml version="1.0"?>
<AIX_VPN
      Version="2.1">
   <IKEProtection
         IKE_Role="Both"
	       IKE_Version="1"
         IKE_XCHGMode="Main"
         IKE_KeyOverlap="10"
         IKE_Flags_UseCRL="No"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinMinutes="60">
      <IKETransform
            IKE_Encryption="3DES-CBC"
            IKE_Hash="SHA"
            IKE_DHGroup="1"
            IKE_AuthenticationMethod="Preshared_key"
            IKE_KeyRefreshMinutes="240"/>
   </IKEProtection>
   <IKETunnel
         IKE_TunnelName="P1"
         IKE_ProtectionRef="P1Pol"
         IKE_Flags_AutoStart="Yes"
         IKE_Flags_MakeRuleWithOptionalIP="No">
      <IKELocalIdentity>
         <IPV4_Address
               Value="1.1.1.1"/>
      </IKELocalIdentity>
      <IKERemoteIdentity>
         <IPV4_Address
               Value="2.2.2.2"/>
      </IKERemoteIdentity>
   </IKETunnel>
   <IKEPresharedKey
         Value="12345"
         Format="ASCII">
      <IKEPresharedRemoteID>
         <PK_IPV4_Address
               Value="2.2.2.2"/>
      </IKEPresharedRemoteID>
   </IKEPresharedKey>
   <IPSecProposal
         IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Encryption="ESP_3DES"
            ESP_KeyRefreshKB="0"
            ESP_Authentication="HMAC-SHA"
            ESP_ExtendedSeqNum="0"
            ESP_EncapsulationMode="Transport"
            ESP_KeyRefreshMinutes="30"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Role="Both"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_InitiatorDHGroup="0"
         IPSec_ResponderDHGroup="NO_PFS"
         IPSec_Flags_UseLifeSize="No"
         IPSec_Flags_UseCommitBit="No"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinMinutes="30"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_TunnelName="P2"
         IPSec_ProtectionRef="P2Pol"
         IPSec_Flags_OnDemand="No"
         IPSec_Flags_AutoStart="Yes">
      <IPSecLocalIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="1.1.1.1"
               From_IPAddr="1.1.1.1"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="2.2.2.2"
               From_IPAddr="2.2.2.2"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

To keep it simple, let's start only the IKEv1 daemon using the following commands:

stopsrc -g ike
startsrc -s tmd ; startsrc -s isakmpd

When we start only isakmpd (IKEv1 daemon), we don't need to start the ikev2d (IKEv2) or iked (broker) daemons. Only the tmd daemon (tunnel manager daemon) is required.

You need to perform the following steps on the Windows 2012 system to create the IKEv1 IPsec policy. The overall tasks can be divided into the following five high-level steps:

  1. Add IPsec snap-in to the Microsoft Management Console (MMC)
  2. Create the IPsec policy.
  3. Create the IPsec filter rules.
    1. Define IP tunneling attributes.
    2. Define authentication methods.
    3. Define filter actions.
  4. Specify other tunnel-related settings.
  5. Assign the IPsec policy.

Add IPsec snap-in to the MMC

Perform the following steps to add an IPsec snap-in to the MMC:

  1. Open the MMC.
  2. Click File -> Add/Remove Snap-in.
    Figure 1. MMC File menu option
  3. From the Available snap-ins list, select IP Security Policy Manager and click Add. Then, click OK.
    Figure 2. Adding the IP Security Policy Management snap-in
  4. In the Select Computer or Domain dialog box, select Local computer and click Finish.
    Figure 3. Selecting the computer or domain to manage snap-in

    Clicking Finish takes you back to the Add or Remove Snap-ins dialog box. Click OK.

Create the IPsec policy

Perform the following steps to create the IPsec policy:

  1. In the main console, right-click IP Security Policies on Local Computer and then click Create IP Security Policy.
    Figure 4. Right-click menu options for IPsecurity policies snap-in
    Figure 5. Welcome page of IP Security Policy Wizard
  2. On the welcome page of the IP Security Policy Wizard, click Next. On the IP Security Policy Name page, click Next to create a policy named Policy1.
    Figure 6. IP security policy naming
  3. On the Request for Secure Communication page, click Next.
    Figure 7. IP security policy wizard
  4. On the wizard completion page, select the Edit properties check box and click Finish.
    Figure 8. IP Security Policy Wizard completion

Create the IPsec filter rules

Perform the following steps to create the IPsec filter rules:

  1. To set the policy properties, select the Use Add Wizard check box at the lower-right corner and then click Add.
    Figure 9. Adding policy properties to the newly created policy
  2. On the welcome page of the Security Rule Wizard, click Next.
    Figure 10. Security Rule Wizard welcome page
    1. Define the IP tunneling attributes.
      1. Select the This rule does not specify a tunnel option and click Next.
        Figure 11. Tunnel Endpoint page
      2. On the Network Type page, select All network connections and click Next.
        Figure 12. Selecting the network type
      3. On the IP Filter List page, click Add.
        Figure 13. Adding IP filters
      4. You can give a meaningful name to the filter list. In this example, we have entered Policy 1 IP Filter List 1. Note that Use Add Wizard check box is selected and click Add.
        Figure 14. Creating an IP filter policy
      5. On the welcome page of the IP Filter Wizard, click Next.
        Figure 15. IP Filter Wizard welcome page
      6. On the IP Filter Description and Mirrored property page, select the Mirrored. Match packets with the exact opposite source and destination option to apply the same rule on both the incoming and outgoing packets on Windows. Then click Next.
        Figure 16. IP Filter Description and Mirrored property page
      7. On the IP Traffic Source page, from the Source address drop-down list, select A specific IP Address or Subnet.
        Figure 17. IP traffic source selection

        Also, provide the source IP below the drop-down list (the partner system's IP = AIX system's IP) and click Next.

      8. On the IP Traffic Destination page, from the Destination address drop-down list, select A specific IP Address or subnet, add the required destination IP, and click Next.
        Figure 18. IP traffic destination selection
      9. On the IP Protocol Type page, in this example, we are selecting Any as the protocol type. You can select the required protocol as per your needs. After that, click Next.
        Figure 19. IP traffic protocol type
      10. On the wizard completion page, click Finish.
        Figure 20. IP Filter Wizard completion
      11. Notice that you are directed to the IP Filter List dialog box. Click OK in dialog box.
        Figure 21. IP Filter List dialog box
    2. Define IP filter actions.
      1. On the IP Filter List page, select Policy 1 IP Filter List 1 and click Next.
        Figure 22. IP Filter List page
      2. On the Filter Action page, if you see an existing filter action, you can select it and click Edit. Else you can add a new filter action if you do not want to modify an existing one. In this example, we are adding a new one. Notice that the Use Add Wizard check box is checked. Click Add.
        Figure 23. Filter Action page
      3. As Use Add Wizard is selected, the IP Security Filter Action Wizard opens. Click Next on the welcome page.
        Figure 24. IP Security Filter Action Wizard welcome page
      4. We will change the default name New filter action to Policy 1 filter action 1. Then, click Next.
        Figure 25. Filter Action Name page
      5. On the Filter Action General Options page, select Negotiate security and click Next.
        Figure 26. Filter Action General Options page
      6. On the Communicating with computers that do not support IPsec page, select Do not allow unsecured communication and click Next.
        Figure 27. IP filter action on communicating with IPsec unaware systems
      7. On the IP Traffic Security page, select Integrity and encryption and click Next.
        Figure 28. IP Traffic Security page
      8. On the wizard completion page, click Finish.
        Figure 29. IP filter action completion page
      9. Notice that you are directed to the Filter Action page. On this page, select the new filter action, Policy 1 filter action 1. Notice that the Use Add Wizard check box is selected and click Next.
        Figure 30. Back at the Filter Action page
    3. Define authentication methods.
      1. On the Authentication Method page, select Use this string to protect the key exchange (pre-shared key) and click Next. Note that this key is the same one that is mentioned in the XML file updated on AIX.
        Figure 31. Selecting the authentication method
      2. On the Security Rule Wizard completion page, click Finish.
        Figure 32. Security rule completion page
      3. In the Policy1 Properties dialog box, click the General tab.
        Figure 33. Policy1 Properties dialog box
    4. Specify other tunnel-related settings.
      1. On the General tab, click Settings to specify additional settings.
        Figure 34. General tab of Policy1 properties
      2. In the Key Exchange Settings dialog box, set the time after which a new key must be regenerated. Then click Methods.
        Figure 35. Policy properties: Key exchange settings
      3. In the Key Exchange Security Methods dialog box, select the rule that is displayed and click Edit.

        You can change the algorithms used for encryption and integrity checking and you can set the Diffie-Hellman group level here. Make sure that the IKE security algorithms match the ones mentioned in the XML file on AIX. The settings we use are the ones shown in the following screen captures.

        After the required algorithms have been chosen, click OK and proceed to the next steps.

        Figure 36. Key exchange settings: Key Exchange Security Methods dialog box
        Figure 37. Key Exchange Security Methods dialog box
      4. In the Key Exchange Security Methods dialog box, click OK. Also, in the Policy1 Properties dialog box, click OK.
        Figure 38. Back to Policy1 Properties
    5. Assign the IPsec policy.
      1. After the policy is created, notice that it is not initially assigned to the system.
        Figure 39. Console root with IP security snap-in
      2. Right-click Policy1 and click Assign.
        Figure 40. Assigning the IP security Policy
      3. Notice that the Policy Assigned state now changes to Yes from No.
        Figure 41. IP security Policy assigned

IKEv1 tunnels between AIX and Windows using certificates

Follow the certificate generation steps on the AIX system and the certificate import steps on Windows explained in their respective sections earlier.

Refer to the tutorial Generating certificates in AIX and importing certificates to Windows for IKE IPsec tunnels for creating the certificates on AIX and importing to the Windows operating system before proceeding with the following steps.

You need to create the following XML file in the AIX system. Let us name it AIX-Windows-Certificates-IKEv1.xml. Add this XML to the IKE database on AIX using the following commands:

/usr/sbin/ikedb -x
/usr/sbin/ikedb -p AIX-Windows-Certificates-IKEv1.xml
<?xml version="1.0"?>
<AIX_VPN
      Version="2.1">
   <IKEProtection
         IKE_Role="Both"
         IKE_Version="1"
         IKE_XCHGMode="Main"
         IKE_KeyOverlap="10"
         IKE_Flags_UseCRL="No"
         IKE_ProtectionName="P1Pol"
         IKE_ResponderKeyRefreshMaxKB="200"
         IKE_ResponderKeyRefreshMinKB="1"
         IKE_ResponderKeyRefreshMaxMinutes="1440"
         IKE_ResponderKeyRefreshMinMinutes="60">
      <IKETransform
            IKE_Encryption="3DES-CBC"
            IKE_Hash="SHA"
            IKE_DHGroup="1"
            IKE_AuthenticationMethod="RSA_signatures"
            IKE_KeyRefreshMinutes="240"/>
   </IKEProtection>
	<IKETunnel
         IKE_TunnelName="P1"
         IKE_ProtectionRef="P1Pol"
         IKE_Flags_AutoStart="Yes"
         IKE_Flags_MakeRuleWithOptionalIP="Yes">
         <IKELocalIdentity>
             <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test2">
         <IPV4_Address
               Value="1.1.1.1"/>
        </ASN1_DN>
      </IKELocalIdentity>
      <IKERemoteIdentity>
             <ASN1_DN Value="/C=IN/ST=KA/L=BA/O=IBM/OU=ISL/CN=test1">
         <IPV4_Address
               Value="2.2.2.2"/>
        </ASN1_DN>
      </IKERemoteIdentity>

   </IKETunnel>
   <IPSecProposal
         IPSec_ProposalName="P2Prop">
      <IPSecESPProtocol
            ESP_Encryption="ESP_3DES"
            ESP_KeyRefreshKB="0"
            ESP_Authentication="HMAC-SHA"
            ESP_ExtendedSeqNum="0"
            ESP_EncapsulationMode="Transport"
            ESP_KeyRefreshMinutes="30"/>
   </IPSecProposal>
   <IPSecProtection
         IPSec_Role="Both"
         IPSec_KeyOverlap="10"
         IPSec_ProposalRefs="P2Prop "
         IPSec_ProtectionName="P2Pol"
         IPSec_InitiatorDHGroup="0"
         IPSec_ResponderDHGroup="NO_PFS"
         IPSec_Flags_UseLifeSize="No"
         IPSec_Flags_UseCommitBit="No"
         IPSec_ResponderKeyRefreshMaxKB="200"
         IPSec_ResponderKeyRefreshMinKB="1"
         IPSec_ResponderKeyRefreshMaxMinutes="43200"
         IPSec_ResponderKeyRefreshMinMinutes="30"/>
   <IPSecTunnel
         IKE_TunnelName="P1"
         IPSec_TunnelName="P2"
         IPSec_ProtectionRef="P2Pol"
         IPSec_Flags_OnDemand="No"
         IPSec_Flags_AutoStart="Yes">
      <IPSecLocalIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="1.1.1.1"
               From_IPAddr="1.1.1.1"/>
      </IPSecLocalIdentity>
      <IPSecRemoteIdentity
            Port="0"
            EndPort="65535"
            Protocol="0">
         <IPV4_Address_Range
               To_IPAddr="2.2.2.2"
               From_IPAddr="2.2.2.2"/>
      </IPSecRemoteIdentity>
   </IPSecTunnel>
</AIX_VPN>

To keep it simple, let’s start only the IKEv1 daemon using the following commands:

stopsrc -g ike
startsrc -s tmd ; startsrc -s isakmpd ; startsrc -s cpsd

When we start only isakmpd (IKEv1 daemon), we don’t need to start the ikev2d (IKEv2) or iked (broker) daemons. Only the tmd daemon is required.

Perform all the steps mentioned for the Windows system in the section, IKEv1 tunnels between AIX and windows using pre-shared keys except the steps showed in Figure 31. Instead of using pre-shared specify the settings as shown in Figure 42 for the certificates.

Figure 42. Authentication Method page in Security Rule Wizard

Summary

This tutorial explained how to establish IKEv1 tunnels between AIX 6.1/ 7.1/ 7.2 and Windows 2012 systems using certificates and pre-shared keys. The following tutorial explains how to establish tunnels using IKEv2 between AIX and Windows Operating system:

IKEv2 IPsec tunnels between AIX 6.1 or later versions and Windows 2012

Part 2 and part 3 in this series, together, explain four different ways to establish tunnels between AIX 6.1/ 7.1/ 7.2 and Windows 2012 systems. These four methods are setting up IKEv1 and IKEv2 tunnels using pre-shared keys and certificates between AIX and Windows.

The configuration steps for Windows (explained in this tutorial) were tried in the lab and worked for the test team. These steps are not endorsed by Microsoft nor by IBM. You can consider the steps mentioned in this tutorial as a guidance to get you started. There is always light at the end of the tunnel. So, enjoy configuring tunnels!

Related topics


Downloadable resources


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=1054632
ArticleTitle=IPsec tunnel configuration between IBM AIX and Microsoft Windows, Part 2: IKEv1 IPsec tunnels between AIX 6.1 or later versions and Windows 2012
publish-date=12012017