Opening firewall ports for intra-app communication

This topic applies to you if you implement two or more Sterling Order Management System Software applications in an integrated manner such that they share the same database, and some of the applications have to communicate through firewalls.

If this scenario applies to you, you have to open firewall ports to allow intra-application communication.

Intra-app communication

"Intra-app communication" refers to management messages and notifications that flow between the Sterling Order Management System Software applications. Some of these intra-app communications include:
  • Database cache notifications
  • System management console commands

The intra-app management messages and notifications are implemented using Remote Method Invocation (RMI). When an IBM® agent or application server starts up, it first creates an RMI method that listens on a port. Next, it registers that RMI service in the YFS_HEARTBEAT table. Agents or application servers that need to send management messages to its peers iterate through the YFS_HEARTBEAT table, picks up the RMI information, and then calls the RMI service of its peers.

By default, the port the RMI method listens-on is random. To enable intra-app communication, one would have to open up a wide range of ports.

Deterministic RMI port feature

The Deterministic RMI Port feature allows you to specify a range of ports that your JVMs can bind to so that you can control the number of firewall ports to open. To specify the range, set the rmi.portrange parameter in the YFS_PROPERTIES file.

For example, if you set rmi.portrange=27001-27050, each JVM will randomly pick a number in that range to bind to. If that port is busy or in use, the JVM will then try each number in the range in sequence until it finds a free port. If it reaches the end of the range, it will wrap around to the beginning of the range and continue trying. If it reaches its original starting point, the JVM gives up and will not start up.

For backward compatibility, RMI will be allowed to choose any valid port number. To do this, either do not set the rmi.portrange parameter or set rmi.portrange=0.

Configuration recommendations

On one hand, you should specify a small port range to reduce the number of ports you open. However, you also need to specify a big enough port range to accommodate the highest concurrent ports used from any single computer node governed by that port range. When estimating your port ranges, consider the following:
  • You need to estimate the largest number of JVMs you expect to start in a single computer node or more precisely from one IP address. Set rmi.portrange to a number that is larger than that estimate.
  • You need to configure the firewall that sits between the Sterling Order Management System Software applications to allow that intra-app communications across the port range above.

You can use the customer_overrides.properties file to selectively set your port range to suit your operational needs. For example, you may run a few (for example, up to 5) JVMs on computer nodes that are dedicated to running application servers. Conversely, you may run a large number (for example, 20) of JVMs on computer nodes that are dedicated to running the Sterling Integration Servers. Using customer_overrides.properties, you could set a low value for the application servers and a higher number for the integration servers.

How port selection works

The application selects a starting point at random in the list of ports, and tries to create an RMI stub at that port number. If the operation fails because the port is in use, the application moves on to the next port in sequence, wrapping at the beginning. If the application reaches the starting point without a successful export, it will throw an exception and exit.

Note: A random starting point is used to reduce the chance of conflicts when multiple JVMs are running on the same server.

For tracking purposes, an INFO-level message with the port number is printed after binding to a specific port when a range is requested. This does not apply when the port number 0 (default) is used, because that is internal to the RMI implementation.

Troubleshooting

If your JVMs cannot start because they cannot find a free port, you can use network tools like netstat to see which port numbers are busy. You can also use tools like lsof to find out which process is listening on the port number and the peer it is talking to.

Limitations

The following limitations apply to use of an RMI port range:

  • The internal and external IP addresses of the hosts (from the point of view of the network zones) must be identical. Effectively, this means that Network Address Translation (NAT) or Secure NAT (SNAT) cannot be used. The reason is that JVMs register their RMI stub with their network address. When a JVM picks up an RMI stub to call, the network address must be reachable from that JVM.

  • All existing RMI configuration requirements apply. For example, a server's hostname must resolve to the external IP address.