How to configure IP security rules

In the following example, we are creating rules to deny 231.12.21.132:45566 (UDP) connection with IP <development IP address> for port 45566

IMPORTANT
-- For IPsec, you must set up allow rules first before setting up deny rules.
-- After rules are added to the filter table or modified, they are not active yet.
 There is a separate action to activate the rules (step 9)
Read through carefully before you start. Try this in your test environment.

IP Security
===========
Installing IP Security and Setting Up a Couple of Filter Rules
This document walks you through installing IPsec software and then creating a couple of filter rules so that 231.12.21.132:45566 (UDP) connection with IP <development IP address> is denied.

1. Install the software:
 bos.msg.en_US.net.IPsec   IP Security Messages - US
 bos.net.IPsec.keymgt      IP Security Key Management
 bos.net.IPsec.rte           IP Security
To check:
# lslpp -l | grep IPsec
 

2. Install the latest fixes for IPsec filesets that you installed (to be at the same as the TL/SP level of the AIX system)

3. Reboot

4. Make sure the default policy is permit, so all IP traffic is permitted, then you add the deny rules for all IP addresses you want to deny access.
# mkfilt -z P
This should return:
Default rule for IPv4 in ODM has been changed.
Default rule for IPv6 in ODM has been changed.
Device IPsec_v4 is in Defined status.
Filter activation for IPv4 not performed.

5. Start IP Security:
# smitty IPsec4
Start/Stop IP Security
Start IP Security ->
----------------------------------------------------------
Start IP Security
Type or select values in entry fields.
Press Enter AFTER making all wanted changes.
[Entry Fields]
Start IP Security [Now and After Restart] +
Deny All Non_Secure IP Packets [no] +
-----------------------------------------------------------
The previous values will:
- Let the system know whether you want to have IPsec started automatically after a restart.
- Let the system know whether you want to allow or deny everything by default.
For this example, I leave both settings to their default values.
Check that IPsec is available.
# lsdev -Cc IPsec ->
IPsec_v4 Available IP Version 4 Security Extension

6. There are now two filter rules created for you, run the following command to check your filter rules:
# lsfilt -v4
Beginning of IPv4 filter rules.
Rule 1:
Rule action: permit
Source Address: 0.0.0.0
Source Mask: 0.0.0.0
Destination Address: 0.0.0.0
Destination Mask: 0.0.0.0
Source Routing: no
Protocol: udp
Source Port: eq 4001
Destination Port: eq 4001
Scope: both
Direction: both
Logging control: no
Fragment control: all packets
Tunnel ID number: 0
Interface: all
Auto-Generated: yes
Rule 2:
*** Dynamic filter placement rule for IKE tunnels ***
Logging control: no
Rule 0:
Rule action: permit
Source Address: 0.0.0.0
Source Mask: 0.0.0.0
Destination Address: 0.0.0.0
Destination Mask: 0.0.0.0
Source Routing: yes
Protocol: all
Source Port: any 0
Destination Port: any 0
Scope: both
Direction: both
Logging control: no
Fragment control: all packets
Tunnel ID number: 0
Interface: all
Auto-Generated: no
End of IPv4 filter rules.
What does that mean?
- Rule 0 says to accept everything, which is the result of leaving the "Deny All Non_Secure IP Packets" option in Step 4 to "no".
- Rule 1 is for IPsec itself.

7. Add a filter rule to allow/permit all other port 45566 requests to <host IP address>:
Add an IP Security Filter Rule
Type or select values in entry fields.
Press Enter AFTER making all wanted changes.
[Entry Fields]
* Rule Action [permit] +
* IP Source Address [0.0.0.0]
* IP Source Mask [0.0.0.0]
IP Destination Address [<development host IP address>]
IP Destination Mask [255.255.255.255]
* Apply to Source Routing? (PERMIT/inbound only) [yes] +
* Protocol [udp] +
* Source Port/ICMP Type Operation [any] +
* Source Port Number ICMP Type # 45566
* Destination Port/ICMP Code Operation [eq] +
* Destination Port Number ICMP Type # 45566
* Routing [both] +
* Direction [both] +
* Log Control [no] +
* Fragmentation Control [all packets] +
* Tunnel ID +#
* Interface [all] +
Filter rule 3 for IPv4 was added successfully.

8. Adding a filter rule to deny port 45566 requests specifically from 231.12.21.132:45566 (UDP),
# smitty IPsec4
Advanced IP Security Configuration
Configure IP Security Filter Rules
Add an IP Security Filter Rule ->
Add an IP Security Filter Rule
Type or select values in entry fields.
Press Enter AFTER making all wanted changes.
[Entry Fields]
* Rule Action [deny] +
* IP Source Address [231.12.21.132]
* IP Source Mask [255.255.255.255]
IP Destination Address [<development host IP address>]
IP Destination Mask [255.255.255.255]
* Apply to Source Routing? (PERMIT/inbound only) [yes] +
* Protocol [udp] +
* Source Port ICMP Type Operation [any] +
* Source Port Number ICMP Type # 45566
* Destination Port / ICMP Code Operation [eq] +
* Destination Port Number ICMP Type # 45566
* Routing [both] +
* Direction [both] +
* Log Control [no] +
* Fragmentation Control [all packets] +
* Tunnel ID +#
* Interface [all] +
Filter rule 4 for IPv4 was successfully.

9. Activate the filter rules by backing out to the "Advanced IP Security Configuration" screen:
# smitty IPsec4
Advanced IP Security Configuration
Activate/Update/Deactivate IP Security Filter Rule
Activate/Update

10. If you apply all the previous steps, 231.12.21.132 is now denied access to <development host IP address> for port 45566.

Be sure to do step 9 each time you make changes to the rules.