Question & Answer
How do you Limit Port Usage in IBM Tivoli Monitoring Version 6?
ITM uses TCP/IP[Note 1] to communicate between and within ITM processes. It uses port numbers to perform this activity. Customers who monitor port usage closely have questions about what ports are used and also what options are available to control and limit those ports. This document explains all these questions. If there is a topic you know well, please skip to the end sections which explain exactly how to control the ports. The presentation uses standard default port numbers but almost anything can be configured.
In TCP/IP communications a port is a number from 0 to 65535. A process can ask to be notified if another process - on this or another server - wants to connect. A call to a "bind" routine links the incoming communication to the listening process.
A network interface is a hardware or software construction that has one or more IP addresses. That can be IPV4 or IPV6 addresses. In simple workstations there is typically a single hardware part which implements the network interface and usually a software network interface for localhost or 127.0.0.1, which is only known locally. Servers may have many network interfaces and a single hardware interface can even have multiple IP addresses.
When an external process connects to the listening port, that produces a socket connection. The connection can be read or written from either side and the TCP/IP software transmits the data.
ITM Usage of TCP/IP
ITM does not control outbound communication. It writes data to a target ip address and port number and lets TCP/IP calculate the best way to do that work. All TCP/IP environments have "route" commands which lets the system administrator control what network interface gets used for the communication.
By default, ITM listens on all interfaces using a BIND call. The KDEB_INTERFACELIST environment variable can be used to limit the interfaces which are surveyed. That is not often required and will not be mentioned further here.
ITM and Location Broker
Each ITM process uses a communication string which is KDE_TRANSPORT or the earlier KDC_FAMILIES which names the protocols supported and the modifiers. See this technote for reference:
- Sitworld: ITM Protocol Usage and Protocol Modifiers
- KDE_TRANSPORT=KDC_FAMILIES="ip.pipe port:1918 ip use:n ip.spipe use:n sna use:n HTTP:1920"
This is composed from the data captured during configuration.
The initial connection supplies a Location Broker function which allows the ITM service to look up a service and the ip address, port number and other items needed for a connection. Location Brokers run at each TEMS and a hub TEMS maintains a master list - the Global Location Broker.
When a service registers, that information is added to the Location Broker data. For example a TEMS will register all the available network interfaces. If KDEB_INTERFACELIST is supplied, the ip addresses listed will be registered using that order. In this way a service can tell a user what ip address and port to use.
The important thing to remember is that services register and users look up the data. The decentralized approach makes the process more resilient and higher performing because there are fewer choke points.
ITM Port Usage - Agents
In a default configuration, agents use the following sockets,
1) Connection to a TEMS port 1918. The communications string defines the protocols used. The CT_CMSLIST environment variable names the servers where a TEMS is running. The initial connection at port 1918 gives access to the Location Broker data and thus indirectly to the TEMS. However from the standpoint of configuration, this is a socket to a 1918 listening port on the server running the TEMS.
2) Agent listening port at 1918+N*4096. If there is just one agent installed, the listening port will be 1918+4096 or 6014. If there are more than one agent installed, the agents contend for the listening ports. That means incidentally that there are a maximum of 15 agents using the default configuration. The listening port is used for several purposes including retrieval of real time data and receiving broadcasts about new WPA address.
3) Agent connection to Warehouse Proxy Agent or WPA. This can be at any port but recently was defaulted to 63358 or 1918+15*4096. The information is actually found from the Location Broker. The WPA registers at the hub TEMS, the Global Location Broker information is propagated to the Local Location Broker at the remote TEMS and the agent looks it up as part of the Location Broker data.
4) Internal Web Server ports. By default all Agents and most ITM processes start an internal web server. The default ports used are 1920 and 3661 [http and https]. There are several purposes including the ability to start and stop diagnostic tracing dynamically. The first ITM process owns the 1920/3661 and the others register with it. If that process stops the 1920 ownership switches to another ITM process if possible.
6) Ephemeral ports. ITM makes use of ports which are received from TCP/IP as "the next free port". These are used to communicate between ITM sub-systems.
7) Local ports. These are on address 0.0.0.0 and 127.0.0.1 which are not internet capable addresses. They are used to maintain awareness between ITM processes such as handling the internal web server switch process.
Almost every single number you see in the above description is configurable except for 4096 and local ports. So if you must make a change you can always make that change. The rest of the technote shows how to limit or control port usage.
ITM Port Usage - Servers
In most ways servers like TEMS/TEPS/S&P/WPA are also agents. You use the same communications string to control.
TEMS/TEPS must have the internal web server present for normal operations.
The TEMS have an added control named gbl_site.txt which identifies which TEMS the TEMS connects to. That is unrelated to this technote.
Changing the Agent Configuration
To implement changes you need to make a permanent change to the Agent configuration. If you make a communications string change to one ITM process it often has to be made to all ITM processes on that server. Here are two technotes that help explain that process. The first relates to just changing the communications string:
- Updating Linux/Unix agent KDC_FAMILIES configuration
The second is a more general technote which has been vetted by L3 and Development as valid long term.
- Linux/Unix Persistent Configuration Changes and Windows comment
Limiting ITM Agent Internal Web Server ports.
The internal web server start can be canceled. To do that just add
to the start of the communications string.
This will eliminate the 1920/3661 ports and also the ephemeral ports associated with the web server.
Alternatively, you can also eliminate specific HTTP ports by adding
HTTPS:0 to the communications string.
NOTE: this cannot be done on the TEPS or the TEMS process without breaking important ITM functions.
Limiting ITM listening ports
If you add EPHEMERAL to the connection string like this
ip.pipe port:1918 EPHEMERAL:Y use:y
then the agent will not use a listening port or a WPA port. There is a limitation: if historical data is going to be collected either 1) a WPA is required on the TEMS the agent reports to or 2) historical data collection must be at the TEMS. This can actually be configured at the TEMS by adding EPHEMERAL:INBOUND in to the TEMS connection string.
This also makes setting up connections through firewalls easier since only the base port 1918 needs to be permitted. There is no significant performance impact with this choice. This modifier can be made to one agent but not another on the same server.
Controlling Ephemeral ports
These ports cannot be eliminated but you can configure them to certain number ranges as described here:
- Avoiding Conflicts with ITM process port number usage
Universal Agent Ports
The Universal Agent is an ITM component that lets you extend ITM by writing your own agents.
UA uses all the same ports and by default will also use port 1919 to communicate with collectors [IANA registered]. Each data collector process will use an ephemeral port to form the socket is created.
KUMP_LOCAL_DATA=Y configures non-socket communication on a single server. In a very few cases that configuration causes collection issues.
Please consider use of Agent Builder instead which is being actively developed.
The tacmd createNode Function uses Ports
This function gets used for the first install of OS Agent on a system. Linux/Unix uses SSH/RSH/REXEC from the hub TEMS to the target agent. For example, SSH usually uses port 22. During agent createNode processing the service port and port 1918 from agent to hub will be used. Afterwards the agent will usually connect to a remote TEMS.
This document describes ITM port usage and also ways you can eliminate or control port usage.
In z/OS the SNA communications can be used for communication. This document does not apply to that option. One of the Portal Client to Portal Server communication option uses part HTTP and part CORBA communications, which is also unrelated to this document. Some Portal Client options can use ports not described here. The ITM EIF facility and the LDAP facility can use ports not described here.
17 June 2018