IBM Spectrum Scale port usage

The nodes in a IBM Spectrum Scale cluster communicate with each other using the TCP/IP protocol. The port number used by the main GPFS daemon (mmfsd) is controlled with the tscTcpPort configuration parameter. The default port number is 1191.

You can specify a different port number using the mmchconfig command:

mmchconfig tscTcpPort=PortNumber

When the main GPFS daemon (mmfsd) is not running on the primary and backup configuration server nodes, a separate service (mmsdrserv) is used to provide access to the configuration data to the rest of the nodes in the cluster. The port number used for this purpose is controlled with the mmsdrservPort parameter. By default, mmsdrserv uses the same port number as the one assigned to the main GPFS daemon. If you change the daemon port number, you must specify the same port number for mmsdrserv using the following command:

mmchconfig mmsdrservPort=PortNumber

Do not change the mmsdrserv port number to a number different from that of the daemon port number.

Certain commands (mmadddisk, mmchmgr, and so on) require an additional socket to be created for the duration of the command. The port numbers assigned to these temporary sockets are controlled with the tscCmdPortRange configuration parameter. If an explicit range is not specified, the port number is dynamically assigned by the operating system from the range of ephemeral port numbers. If you want to restrict the range of ports used by IBM Spectrum Scale commands, use the mmchconfig command:

mmchconfig tscCmdPortRange=LowNumber-HighNumber

In a remote cluster setup, if IBM Spectrum Scale on the remote cluster is configured to use a port number other than the default, you have to specify the port number to be used with the mmremotecluster command:

mmremotecluster update ClusterName -n tcpPort=PortNumber,Node,Node...

For related information, see the topic Firewall recommendations for internal communication among nodes.

Table 1 provides IBM Spectrum Scale port usage information:

Table 1. IBM Spectrum Scale port usage
Descriptor	Explanation
Service provider	IBM Spectrum Scale
Service name	mmfsd mmsdrserv
Port number	1191 While executing certain commands, IBM Spectrum Scale may need to create additional sockets whose dynamic port numbers are assigned by the operating system. Such sockets are used by commands to exchange data with GPFS daemons running on other nodes. The port numbers that are used correspond to the ephemeral ports of the operating system. To control which ports are used by the commands (so that firewall rules can be written to allow incoming traffic only on those ports), you can restrict the port range to a specific range by setting the tscCmdPortRange configuration variable.
Protocols	TCP/IP
Source port range	The source port range is chosen by the operating system on the client side.
Is the service name/number pair in the default /etc/services file shipped with AIX® and Linux distributions?	See the IBM Spectrum Scale FAQ in IBM® Knowledge Center (www.ibm.com/support/knowledgecenter/STXKQY/gpfsclustersfaq.html).
Is the service name/number pair added to /etc/services by a product?	No
Binaries that listen on the ports	/usr/lpp/mmfs/bin/mmfsd /usr/lpp/mmfs/bin/mmsdrserv
Can the service be configured to use a different port?	Yes. To change the main port used by IBM Spectrum Scale, enter: `mmchconfig tscTcpPort=PortNumber` Note: If you change the main port (daemon port) number, you must change the mmsdrserv port to the same number. To change the mmsdrserv port number to match the daemon port number, use: `mmchconfig mmsdrservPort=PortNumber` To change the range of port numbers used for command execution, use: `mmchconfig tscCmdPortRange=LowNumber-HighNumber` To specify a port number when connecting to remote clusters, use the mmremotecluster command.
When is the service required? What depends on the service?	On the IBM Spectrum Scale primary and secondary cluster configuration servers, either mmsdrserv or mmfsd needs to be running at all times to provide access to IBM Spectrum Scale configuration data to the rest of the cluster. On other nodes, mmfsd must be running in order to mount a IBM Spectrum Scale file system. Depending on the IBM Spectrum Scale configuration, a node either has to be a member of the IBM Spectrum Scale cluster or possess an authorized SSL key in order to establish a connection.
When the daemon starts and its port is already in use (for example, another resource has bound to it already), how does the daemon behave?	The daemon shuts down and tries to start over again. Most GPFS daemon down error messages are in the mmfs.log.previous log for the instance that failed. If the daemon restarted, it generates a new mmfs.log.latest log. Begin problem determination for these errors by examining the operating system error log. IBM Spectrum Scale records file system or disk failures using the error logging facility provided by the operating system: syslog facility on Linux and errpt facility on AIX. See the IBM Spectrum Scale: Problem Determination Guide for further information.
Is there an administrator interface to query the daemon and have it report its port number?	Yes; issue this command: `mmlsconfig tscTcpPort`
Is the service/port registered with the Internet Assigned Numbers Authority (IANA)?	Yes `gpfs 1191/tcp General Parallel File System gpfs 1191/udp General Parallel File System # Dave Craft <gpfs@ibm.com> November 2004`

Note: Ports configured for the IBM Spectrum Scale remote shell command (such as ssh) or the remote file copy command (such as scp) and the ICMP echo command (network ping) also must be unblocked in the firewall for IBM Spectrum Scale to function properly.