This article describes how OpenStack handles networking with iptables, chains, and rules, much like other systems. But first, let's look at the structure of iptables as a refresher of the technology used in this article.

iptable structure

An iptable is a user-space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall; iptable specifically refers to IPv4.

To set up a Linux firewall, rules are used, each of which specifies what to match within a packet and what to do with such a packet. Chains are a list of rules.

The previous incarnation of iptables, ipchains, added the concept of chains of rules; iptables extended this further into tables. So the structure of iptables is: iptables > tables > chains > rules.

The iptable has four built-in tables:

• Filter table: The default table with the following chains:
  • INPUT for packets coming to the local server.
  • OUTPUT for packets generated locally and going out of the local server.
  • FORWARD for packets routed through the local server.
• NAT table (network address translation):
  • PREROUTING: Used for destination NAT, it alters packets IP addresses before routing.
  • POSTROUTING: Used for source NAT, it alters packets IP addresses after routing.
  • OUTPUT: The NAT for locally generated packets on the firewall.
• Mangle table: For specialized packet alteration:
  • PREROUTING
  • OUTPUT
  • FORWARD
  • INPUT
  • POSTROUTING
• Raw table: For configuration exemptions:
  • PREROUTING
  • OUTPUT

Back to top

The iptable within OpenStack

Within OpenStack, you'll find iptable chains and rules predominantly in the Cloud Compute-Nova module, a cloud computing fabric controller (the main part of an IaaS system) written in Python that uses many external libraries. This article details the nova-network FlatDHCPManager component, as well as other OpenStack components, necessary for networking tasks.

When it starts, OpenStack defines some of the OpenStack chains. These chains form a chain fabric with Linux's built-in chains. Another task at startup is to define some rules for fixed network range and the metadata service. Once a network is created and used, nova-network sets up some rules. When one instance (also known as a server and VMs) is created, nova-compute creates one instance-specific chain and set up rules under this chain to assure the instance's connectivity. Regarding floating IPs, OpenStack also uses some rules to make it work. In addition, OpenStack's security group and its rules are embodied by iptables rules.

Back to top

OpenStack primer

OpenStack, a global collaboration of developers and cloud computing technologists producing the open standard cloud operating system for both public and private clouds, is free open source software released under the terms of the Apache license. Cloud service providers, enterprises, and government organizations can take advantage of the freely available, Apache-licensed software to build massively scalable cloud environments.

OpenStack currently consists of six core software projects:

• Cloud Compute-Nova
• Cloud Storage-Swift
• Image Service-Glance (delivery and registration)
• Identity Service-Keystone
• Dashboard-Horizon
• Network Connectivity-Quantum

These projects, along with a vibrant ecosystem of technology providers and future projects, deliver a pluggable framework and operating system for public and private clouds.

There are more than 10 binaries in the Nova project, among which 3 are related to VMs connectivity:

• nova-api provides the metadata service for VMs.
• nova-compute sets up the network environment for VMs.
• nova-network sets up the network environment for the entire cloud ecosystem, tasks such as IP allocation, DHCP settings and so on.

The Nova module primarily consists of a set of Python daemons, though it requires and integrates with a number of native system components for databases, messaging, and virtualization capabilities. It uses a special metadata service to enable virtual machine instances to retrieve instance-specific data. Instances access the metadata service at http://169.254.169.254.

The metadata includes public SSH keys (identified by key/pair name when a user requests a new instance), user data (passed as the user_data parameter in the API call or by the --user_data flag in the Nova boot command). The binary nova-api implements the metadata service.

OpenStack is complex set of components. To learn more about the system and how one of the other components works, follow the extensive OpenStack resources in the Resources section.

Before you move onto rules and chains in Nova, let's take a look at IP addressing modes.

Back to top

IP addressing in Nova

Every VM is automatically assigned a private IP address from each available nova-network. These IP addresses are called fixed IPs. You can optionally assign public IP addresses to instances. OpenStack uses the term floating IP to refer to an IP address (typically public) that can be dynamically added to a running virtual instance.

There are multiple strategies available for implementing fixed IPs:

• Flat mode
• Flat DHCP mode
• VLAN DHCP mode
• Nova-network with quantum mode

Flat mode

Flat mode is the simplest networking mode. Each instance receives a fixed IP from the pool. All instances are attached to the same bridge (br100) by default. The bridge must be configured manually. The networking configuration is injected into the instance before it is booted. And there is no floating IP feature in this mode.

Flat DHCP mode

This is similar to the flat mode in that all instances are attached to the same bridge. In this mode Nova does a bit more configuration; it will attempt to bridge into an Ethernet device (eth0 by default). It will also run dnsmasq as a dhcpserver listening on this bridge. Instances receive their fixed IPs by doing a dhcpdiscover. Moreover, floating IP feature is provided.

VLAN DHCP mode

This is the default networking mode and supports the most features. For multiple machine installation, it requires a switch that supports host-managed VLAN tagging. In this mode, Nova will create a VLAN and bridge for each project (same as a tenant). The project gets a range of private IPs that are only accessible from inside the VLAN. For a user to access the instances in his project, a special VPN instance (code-named "cloudpipe") needs to be created. Nova generates a certificate and key for the user to access the VPN and starts the VPN automatically.

Nova-network with quantum mode

In this mode, the DHCP server can also be launched automatically, but floating IP is not supported. There is a quantum agent running on each compute host to connect virtual instances to quantum network. The network topology can be much sophisticated.

In Nova, a security group is a named collection of network access rules, like firewall policies. These access rules specify which incoming network traffic should be delivered to all VM instances in the group; all other incoming traffic being discarded. Users can modify rules for a group at any time. The new rules are automatically enforced for all running instances and instances launched from then on. A security group can be thought of as a security profile or a security role, such as "webappserver."

Back to top

Rules and chains of iptables

As described earlier, an iptable is a user space application program that allows a system administrator to configure the tables provided by the Linux kernel firewall and the chains and rules it stores.

Also noted previously, several different tables can be defined; among those used in OpenStack are filter and NAT. Each table contains a number of built-in chains and can also contain user-defined chains. Each chain is a list of rules that can match a set of packets. Each rule specifies what to do with a packet that matches — this is called a target, which might be a jump to a user-defined chain in the same table.

From here on, I'll discuss the various key chains and rules.

Chains created when nova-network starts

As shown in Figure 1, NetworkManager is a big class that connects with many other classes or modules. Among these classes or modules, one of them is linux_net. The linux_net contains an IptablesManager object, the __init__() method of which initializes the chains defined in OpenStack system.

Tables of IPv4 packet filter rules in the Linux kernel have some built-in chains as mentioned earlier: PREROUTING, INPUT, FORWARD, OUTPUT and POSTROUTING. In general, a packet arriving at a Linux host go to PREROUTING chain. After passing it, the kernel makes a routing decision. If the packet is for the Linux host machine, it goes to INPUT chain, then if accepted, goes to the target process. If the packet is not for the Linux host machine, it goes to FORWARD chain, then to POSTROUTING chain, and then leave the host machine. The packets generated by local processes go to OUTPUT chain first, then if accepted, go to POSTROUTING chain.

Besides these built-in chains, OpenStack system creates some others chains that are hooked into the chain system. OpenStack chains consist of two types — unwrapped and wrapped chains.

Examples of unwrapped chains include nova-filter-top and nova-postrouting-bottom. The nova-filter-top is added at the top of the FORWARD and OUTPUT chains. Its name is not wrapped, so it's shared between the various Nova workers. It's intended for rules that need to live at the top of the FORWARD and OUTPUT chains. It's in both the IPv4 and IPv6 set of tables.

Examples of wrapped chains include the green colored chains. The name of these chains has the process name as a suffix. For example, nova-network creates the nova-network-PREROUTING chain.

For IPv4 and IPv6, the built-in INPUT, OUTPUT, and FORWARD filter chains are wrapped, meaning that the "real" INPUT chain has a rule that jumps to the wrapped INPUT chain, etc. Additionally, there's a wrapped chain named local which is jumped to from nova-filter-top.

For IPv4, the built-in PREROUTING, OUTPUT, and POSTROUTING NAT chains are wrapped in the same way as the built-in filter chains. Additionally, there's an sNAT chain and a float-sNAT chain that are applied after the POSTROUTING chain.

Rules when nova-network starts

Figure 3 shows part of the startup process of FlatDHCPManager. The __init__() method that constructs an IptableManager object to create many OpenStack chains was previously described. So let's focus on init_host(). The init_host() method calls the initialize() method of LinuxNet3 which calls methods of linux_net to set up some iptables rules in a NAT table.

(A larger version of Figure 3.)

Let's look at these rules.

Rule for metadata host

Among rules created by init_host() of linux_net, one rule is to allow IPs under FLAGS.fixed_range to access metadata_host (it is 192.168.1.90 in this case).

-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 192.168.1.90/32 -j ACCEPT

The ensure_metadata_ip() adds 169.254.169.254/32 to lo device with the following command:

# ip addr add 169.254.169.254/32 scope link dev lo

Then metadata_forward() adds a dNAT rule to route packets from 169.254.169.254/32 to metadata_host:

-A nova-network-PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT
--to-destination 192.168.1.90:8775

If the nova-network and nova-api are not running on the same host, you must define the metadata_host on the nova-network host to point to the nova-api host.

Rule to access dmz

FLAGS.dmz_cidr defines a list of dmz (perimeter networking) CIDRs (classless inter-domain routing). By default, it is an empty list. In this case it is 10.128.0.0/24. So, the rule is:

-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 10.128.0.0/24 -j ACCEPT

Rule for VMs to connect each other

Another rule is to ensure VMs with two fixed IPs can talk with each other:

-A nova-network-POSTROUTING -s 10.0.0.0/8 -d 10.0.0.0/8 -m conntrack ! --ctstate DNAT 
-j ACCEPT

Rule for access outside of fixed subnet

The add_snat_rule() method adds an sNAT rule into wrapped chain sNAT of the NAT table. You can see ip_range and FLAGS.routing_source_ip in Figure 3. The ip_range's value is defined by FLAGS.fixed_range. In this case, it is 10.0.0.0/8. FLAGS.routing_source_ip is FLAGS.my_ip by default, which is defaulted by function flags._get_my_ip(). In this case, FLAGS.my_ip is 192.168.1.90. After that, you get a rule such as:

-A nova-network-snat -s 10.0.0.0/8 -j SNAT --to-source 192.168.1.90

For fixed IPs to access outside, you must create a network that is a subnet of FLAGS.fixed_range. That way, with the default gateway pointed to one IP of br100 on the nova-network machine, the VMs with this subnet's IPs are able to access an outside network.

Rules for each network

To set up a network, nova-network creates some rules in the filter table. To trigger the nova-network's operation on a given network, run following commands:

1. Create a network and set the host:

   # ./bin/nova-manage network create mynet 10.10.10.0/24

   
   
2. Boot a server:

   nova boot --image a3fb743d-42df-49ba-b9c4-8042ebbd344e --flavor 1 myserver

   
   

After executing these commands, you have these rules:

1. Allow a forwarded traffic pass bridge so the IP on br100 can work as a gateway:

   -A nova-network-FORWARD -i br100 -j ACCEPT -A nova-network-FORWARD -o br100 -j ACCEPT

   
   
2. Allow DHCP and DNS traffic to come into the local dnsmasq process:

   -A nova-network-INPUT -i br100 -p udp -m udp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 67 -j ACCEPT -A nova-network-INPUT -i br100 -p udp -m udp --dport 53 -j ACCEPT -A nova-network-INPUT -i br100 -p tcp -m tcp --dport 53 -j ACCEPT

   
   

Note: 67 is DHCP port and 53 is DNS port.

Rules on nova-compute host

The nova-compute module also creates wrapped chains. On these chains it creates some rules in the filter table. Let's look at those.

Rules to allow forwarded traffic pass bridge

These rules on the nova-compute host allow VMs to connect with nova-network host and VMs on other compute hosts.

-A nova-compute-FORWARD -i br100 -j ACCEPT 
-A nova-compute-FORWARD -o br100 -j ACCEPT

Chain and rules for each instance

For each instance, it creates one chain and some rules in the filter table (Figure 4).

You can see:

1. nova-compute creates one chain for each instance. In Figure 4, the chain's name is nova-compute-inst-1.
2. All traffic targeted to this instance, no matter whether it is forwarded or generated from local process, go into this instance's specific chain.
3. All traffic from IPs of the subnet that the instance resides in is allowed.
4. All DHCP traffic from the specified DHCP server is allowed.
5. All other traffic is dropped.

nova-api

On starting, nova-api creates a rule in the filter table to allow others to access the nova-api service.

-A nova-api-INPUT -d 192.168.1.90/32 -p tcp -m tcp --dport 8775 -j ACCEPT

Floating IPs

To see how floating IPs are implemented, first associate a floating IP to the instance's fixed IP. The fixed IP of the instance previously created is 10.10.10.2.

Create a floating IP in the default pool

To create a floating IPs in the default pool:

# nova-manage floating create --ip_range=192.168.1.232/30

To allocate a floating IP from this pool:

# Nova floating-ip-create

You've got an IP of 192.168.1.233. Now assign it to the instance that has an ID of 8f773639-c04f-4885-9349-ac7d6a799843:

# nova add-floating-ip 8f773639-c04f-4885-9349-ac7d6a799843 192.168.1.233

Bind floating IP to public interface

FLAGS.public_interface is used to bind floating IPs. After you run the nova add-floating-ip command, you can see you have the following floating IP under the public_interface:

# ip addr list dev wlan0
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 08:11:96:75:91:54 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.90/16 brd 192.168.255.255 scope global wlan0
    inet 192.168.1.233/32 scope global wlan0
    inet6 fe80::a11:96ff:fe75:9154/64 scope link 
       valid_lft forever preferred_lft forever

Rules in NAT table for floating IPs

After the instance gets a floating IP on the nova-network host, these rules apply:

-A nova-network-OUTPUT -d 192.168.1.233/32 -j DNAT --to-destination 10.10.10.2
-A nova-network-PREROUTING -d 192.168.1.233/32 -j DNAT --to-destination 10.10.10.2 
-A nova-network-float-snat -s 10.10.10.2/32 -j SNAT --to-source 192.168.1.233

You can see that dNAT rule is used to translate the floating IP into the instance's fixed IP. If a packet arrives at the nova-network host with the floating IP as target IP, the target IP is translated. And then you have a sNAT rule that translates traffic from the instance's fixed IP into the floating IP. Since all the traffic from VMs to outside of the fixed network is pointed to gateway, which is set by the nova-network's dnsmasq process, with this sNAT rule the traffic out from VMs can be masked as from the floating IP successfully. In addition, you have a dNAT rule in a wrapped OUTPUT chain that allows the local process on nova-network to access VMs with floating IPs.

Ping VMs with floating IP

To ping VMs with floating IPs, you need some more rules. Remember that on the nova-compute host, you have a specific chain for each instance; the rules in it just allow the traffic from IPs in the fixed subnet. If you ping a floating IP, the traffic will be dropped by those rules since the source IP of the ping packet is not within the fixed subnet. Obviously, you need add a rule to allow icmp traffic.

To add a rule allowing ping, use OpenStack's security group rule concept:

# nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0

After that, you can see you have one more rule created under that instance's specific chain:

In the same way, you can enable SSH to the VMs with floating IP.

Back to top

In conclusion

Rules from iptables are used extensively by OpenStack network. Its security group and floating IPs concepts are just the start of using the iptables rules. Dive into the resources that follow to learn more about using OpenStack's IaaS environment.

Resources

Learn

• Explore the OpenStack system with resources from the developer community:
  • Cloud Compute-Nova module:
    • At OpenStack
    • The Nova project developer's site at GitHub
    • OpenStack Compute Administration Manual
  • Network Connectivity-Quantum at OpenStack
  • Image Services-Glance
    • Project documentation
    • At OpenStack
  • Identity Services-Keystone
    • At OpenStack
    • Project documentation
  • Cloud Storage-Swift at OpenStack
  • Dashboard-Horizon at OpenStack
  • OpenStack manuals:
    • For developers
    • APIs
    • Admin guides
  • Answers to common questions
  
  
• The author explains how to use the OpenStack nova-scheduler to help decide which compute node host should launch an image instance. For more on OpenStack from the author, see the blog.
  
  
• developerWorks has an OpenStack wiki.
  
  
• Explore developerWorks Cloud Computing, where you will find valuable community discussions and learn about new technical resources related to the cloud.
  
  
• Stay current with developerWorks technical events and webcasts focused on a variety of IBM products and IT industry topics.
  
  
• Attend a free developerWorks Live! briefing to get up to speed quickly on IBM products and tools as well as IT industry trends.
  
  
• Follow developerWorks on Twitter.
  
  
• Watch developerWorks on-demand demos ranging from product installation and setup demos for beginners to advanced functionality for experienced developers.
  
  

Get products and technologies

• Evaluate IBM products in the way that suits you best: Download a product trial, try a product online, use a product in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.
  
  

Discuss

• Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• iptable structure
• The iptable within OpenStack
• OpenStack primer
• IP addressing in Nova
• Rules and chains of iptables
• In conclusion
• Resources
• About the author
• Comments

Next steps from IBM

Learn more, join the cloud community

---

• Community: Cloud computing conversations led by IBM community members
• Buy: Try IBM software on IBM SmartCloud Enterprise

Dig deeper into Cloud computing on developerWorks

IBM SmartCloud trial. No charge.

Unleash the power of hybrid cloud computing today!

---

Public and private solutions in one trial.

Special offers