health network-performance

Perform network health checks on an IBM Software Hub deployment on a Red Hat OpenShift cluster with iPerf. These checks include detecting issues that are related to network performance and network access failures. You must log in to the Red Hat OpenShift Container Platform as a cluster administrator to use this command.

Prerequisites

Log in to the cluster.
Log in to Red Hat® OpenShift® Container Platform as a cluster administrator.
${OC_LOGIN}
Remember: OC_LOGIN is an alias for the oc login command.
Verify the config file.

A Kubernetes config file must be at either ~/.kube/config or ~/auth/kubeconfig for this command to work. The config file must have the target cluster as the current context.

Before you begin

Before you run the cpd-cli health network-performance command, follow these steps to enable access to the ibm-network-performance image:

  1. If you are using a private registry, the network-performance image is automatically mirrored when you mirror the IBM® Software Hub images.
    Note: The following examples use the recommended installation environment variables.

    It is strongly recommended that you use a script to create environment variables with the correct values for your environment. For more information, see Setting up installation environment variables.

    • Use the --image-prefix and --image-tag when you run the cpd-cli health network-performance command:
      cpd-cli health network-performance \
--image-prefix=${PRIVATE_REGISTRY_LOCATION}/cpopen/cpd \
--image-tag=${VERSION}.${IMAGE_ARCH}
  2. For online environments, the cluster will use the default network-performance image located in icr.io/cpopen/cpd when you run the cpd-cli health network-performance command.

Syntax

cpd-cli health network-performance \
[--image-prefix=<image-registry-prefix>] \
[--image-tag=<image-tag> \
[--log-level=debug|trace] \
[--minbandwidth=<MBytes/sec>] \
[--node-list=<comma-separated
node-names>] \
[--save] \
[--verbose]

Arguments

The command has no arguments.

Options

Table 1: Command options
Option Description

--help

-h

 Display command help.
Status
Optional.
Syntax
--help
Default value
No default.
Valid values
Not applicable.

--image-prefix

 Specify the registry that the image should be pulled from.
Status
Optional. Required only when the image is pulled from a private container registry.
Syntax
--image-prefix=<image-registry-prefix>
Default value
icr.io/cpopen/cpd
Valid values
  • icr.io/cpopen/cpd
  • ${PRIVATE_REGISTRY_LOCATION}

--image-tag

 Specify the tag that identifies the image in the repository.
Status
Optional. Used to specify an image tag for the image name in a registry that is specified by the --image-prefix option. You can specify the --image-tag=<image-tag> option when you specify the --image-prefix option.
Syntax
--image-tag=<image-tag>
Default value
No default.
Valid values

The --image-prefix is the image registry. The <image-tag> is the specific image tag for an image in a registry. You can provide this option if you are using a local mirrored image where the full location of an image is: <registry url path>/<image name>:<image tag>.
--log-level The command log level.
Status
Optional.
Syntax
--log-level=debug|trace
Default value
No default.
Valid values
debug
Debug messages are written to the log.
trace
Trace messages are written to the log.

--minbandwidth

-b

 Minimum acceptable bandwidth between nodes in MBytes/sec.
Status
Optional.
Syntax
--minbandwidth=<MBytes/sec>
Default value
300
Valid values
A floating point integer that is greater than or equal to 300 MBytes/sec.

If you provide a value that is less than 300 MBytes/sec, the default value is used.

--node-list

 A comma-separated list of valid node names that you can use to run the network performance test on a subset of nodes in the cluster.
Status
Optional.
Syntax
--node-list=<comma-separated node-names>
Default value
The test is conducted on all nodes in a cluster.
Valid values

A comma-separated list of valid node names that you can use to run the network performance test on a subset of nodes in the cluster. Do not include spaces between nodes names. Use the following command to get the list of node names for a cluster:

oc get nodes -o custom-columns=NAME:.metadata.name --no-headers
--save Save output, resource YAML files, and pod container logs to the local file system.
Status
Optional.
Syntax
--save
Default value
false
Valid values
true|false
--verbose Display detailed information about resources in table format.
Status
Optional.
Syntax
--verbose
Default value
false
Valid values
true|false

Examples

Standard run
cpd-cli health network-performance
Note: Some output has been truncated for documentation purposes only.


############################################################################

                         NETWORK PERFORMANCE REPORT
                         
############################################################################

[32m[SUCCESS...] [0m All nodes that will be tested are ready

Namespace ibm-network-performance created

Daemonset ibm-network-performance-ds created successfully in namespace ibm-network-performance
Daemonset ibm-network-performance-ds started successfully in namespace ibm-network-performance
[32m[SUCCESS...] [0m All pods were successfully created and their containers are running

server node: worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
 worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 455 MBytes/sec.
 worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 456 MBytes/sec.

server node: worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
 worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 450 MBytes/sec.
 worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 453 MBytes/sec.

server node: worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
 worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 443 MBytes/sec.
 worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 415 MBytes/sec.

Deleting Namespace ibm-network-performance ...

################################   Summary   ###############################

[32m[SUCCESS...] [0m All nodes that will be tested are ready
[32m[SUCCESS...] [0m All pods were successfully created and their containers are running
[32m[SUCCESS...] [0m All nodes tested met the minimum expected network throughput of 350 MBytes/sec

Inter-node communication throughput (MBytes/sec):

          	0 (client)	1 (client)	2 (client)
0 (server)	-         	443       	415       
1 (server)	456       	-         	455       
2 (server)	453       	450       	-         

Legend (Node Information):

NodeID	Node Internal IP	Node Name                                             
0     	10.254.32.103   	worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
1     	10.254.49.192   	worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
2     	10.254.28.135   	worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com

Network Performance Check Result: [32m[SUCCESS...] [0m

Network performance info gathered successfully!
Verbose run
cpd-cli health network-performance --verbose
Note: Some output has been truncated for documentation purposes only.


############################################################################

                         NETWORK PERFORMANCE REPORT
                         
############################################################################

[32m[SUCCESS...] [0m All nodes that will be tested are ready

Namespace ibm-network-performance created

Daemonset ibm-network-performance-ds created successfully in namespace ibm-network-performance
Daemonset ibm-network-performance-ds started successfully in namespace ibm-network-performance
[32m[SUCCESS...] [0m All pods were successfully created and their containers are running

server node: worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
[2024-09-17T21:35:55Z] command: iperf3 -c 10.254.32.104 -f M
[2024-09-17T21:35:55Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-lgw9d, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.32.104, port 5201
[  5] local 10.254.49.194 port 53838 connected to 10.254.32.104 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   488 MBytes   488 MBytes/sec  365    977 KBytes       
[  5]   1.00-2.00   sec   484 MBytes   484 MBytes/sec    0   1.22 MBytes       
[  5]   2.00-3.00   sec   536 MBytes   536 MBytes/sec    0   1.39 MBytes       
[  5]   3.00-4.00   sec   519 MBytes   519 MBytes/sec    0   1.49 MBytes       
[  5]   4.00-5.00   sec   455 MBytes   455 MBytes/sec  1115    839 KBytes       
[  5]   5.00-6.00   sec   439 MBytes   439 MBytes/sec    0   1.12 MBytes       
[  5]   6.00-7.00   sec   440 MBytes   440 MBytes/sec    0   1.29 MBytes       
[  5]   7.00-8.00   sec   450 MBytes   450 MBytes/sec    0   1.37 MBytes       
[  5]   8.00-9.00   sec   438 MBytes   438 MBytes/sec    0   1.41 MBytes       
[  5]   9.00-10.00  sec   445 MBytes   445 MBytes/sec    0   1.43 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.58 GBytes   469 MBytes/sec  1480             sender
[  5]   0.00-10.04  sec  4.58 GBytes   467 MBytes/sec                  receiver

iperf Done.

 worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 469 MBytes/sec.
[2024-09-17T21:36:06Z] command: iperf3 -c 10.254.32.104 -f M
[2024-09-17T21:36:06Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-t6tjl, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.32.104, port 5201
[  5] local 10.254.28.136 port 51282 connected to 10.254.32.104 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   416 MBytes   416 MBytes/sec  244    887 KBytes       
[  5]   1.00-2.00   sec   400 MBytes   400 MBytes/sec    0   1.11 MBytes       
[  5]   2.00-3.00   sec   408 MBytes   407 MBytes/sec   48   1.26 MBytes       
[  5]   3.00-4.00   sec   399 MBytes   399 MBytes/sec  240   1.09 MBytes       
[  5]   4.00-5.00   sec   410 MBytes   410 MBytes/sec    0   1.25 MBytes       
[  5]   5.00-6.00   sec   395 MBytes   395 MBytes/sec    1   1.31 MBytes       
[  5]   6.00-7.00   sec   404 MBytes   404 MBytes/sec    0   1.37 MBytes       
[  5]   7.00-8.00   sec   401 MBytes   401 MBytes/sec   96   1.15 MBytes       
[  5]   8.00-9.00   sec   400 MBytes   400 MBytes/sec   96   1.26 MBytes       
[  5]   9.00-10.00  sec   401 MBytes   401 MBytes/sec   48   1.33 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.94 GBytes   403 MBytes/sec  773             sender
[  5]   0.00-10.04  sec  3.94 GBytes   402 MBytes/sec                  receiver

iperf Done.

 worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 403 MBytes/sec.

server node: worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
[2024-09-17T21:37:16Z] command: iperf3 -c 10.254.49.194 -f M
[2024-09-17T21:37:16Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-jxhpn, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.49.194, port 5201
[  5] local 10.254.32.104 port 57930 connected to 10.254.49.194 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   445 MBytes   445 MBytes/sec  426   1.11 MBytes       
[  5]   1.00-2.00   sec   458 MBytes   458 MBytes/sec    0   1.30 MBytes       
[  5]   2.00-3.00   sec   448 MBytes   447 MBytes/sec    0   1.36 MBytes       
[  5]   3.00-4.00   sec   439 MBytes   439 MBytes/sec    0   1.41 MBytes       
[  5]   4.00-5.00   sec   432 MBytes   433 MBytes/sec    0   1.44 MBytes       
[  5]   5.00-6.00   sec   451 MBytes   451 MBytes/sec  220   1.14 MBytes       
[  5]   6.00-7.00   sec   460 MBytes   460 MBytes/sec    0   1.31 MBytes       
[  5]   7.00-8.00   sec   446 MBytes   446 MBytes/sec  398   1.12 MBytes       
[  5]   8.00-9.00   sec   440 MBytes   440 MBytes/sec    0   1.27 MBytes       
[  5]   9.00-10.00  sec   458 MBytes   457 MBytes/sec    0   1.37 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.37 GBytes   448 MBytes/sec  1044             sender
[  5]   0.00-10.04  sec  4.37 GBytes   446 MBytes/sec                  receiver

iperf Done.

 worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 448 MBytes/sec.
[2024-09-17T21:37:27Z] command: iperf3 -c 10.254.49.194 -f M
[2024-09-17T21:37:27Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-t6tjl, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.49.194, port 5201
[  5] local 10.254.28.136 port 49678 connected to 10.254.49.194 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   451 MBytes   451 MBytes/sec  193    940 KBytes       
[  5]   1.00-2.00   sec   436 MBytes   436 MBytes/sec    0   1.17 MBytes       
[  5]   2.00-3.00   sec   459 MBytes   459 MBytes/sec    0   1.32 MBytes       
[  5]   3.00-4.00   sec   451 MBytes   451 MBytes/sec  196   1.16 MBytes       
[  5]   4.00-5.00   sec   458 MBytes   457 MBytes/sec    0   1.32 MBytes       
[  5]   5.00-6.00   sec   462 MBytes   462 MBytes/sec    0   1.39 MBytes       
[  5]   6.00-7.00   sec   465 MBytes   465 MBytes/sec  144   1.18 MBytes       
[  5]   7.00-8.00   sec   448 MBytes   448 MBytes/sec    0   1.31 MBytes       
[  5]   8.00-9.00   sec   421 MBytes   421 MBytes/sec    0   1.36 MBytes       
[  5]   9.00-10.00  sec   450 MBytes   450 MBytes/sec   48   1.40 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.39 GBytes   450 MBytes/sec  581             sender
[  5]   0.00-10.04  sec  4.39 GBytes   448 MBytes/sec                  receiver

iperf Done.

 worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 450 MBytes/sec.

server node: worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
 wait about 1 minute for the iperf server to be up and running...
[2024-09-17T21:38:37Z] command: iperf3 -c 10.254.28.136 -f M
[2024-09-17T21:38:37Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-jxhpn, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.28.136, port 5201
[  5] local 10.254.32.104 port 43482 connected to 10.254.28.136 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   389 MBytes   389 MBytes/sec  136    829 KBytes       
[  5]   1.00-2.00   sec   440 MBytes   440 MBytes/sec   48   1.12 MBytes       
[  5]   2.00-3.00   sec   445 MBytes   445 MBytes/sec    0   1.28 MBytes       
[  5]   3.00-4.00   sec   450 MBytes   450 MBytes/sec    0   1.38 MBytes       
[  5]   4.00-5.00   sec   462 MBytes   462 MBytes/sec    0   1.44 MBytes       
[  5]   5.00-6.00   sec   454 MBytes   454 MBytes/sec    0   1.47 MBytes       
[  5]   6.00-7.00   sec   442 MBytes   442 MBytes/sec    0   1.48 MBytes       
[  5]   7.00-8.00   sec   445 MBytes   445 MBytes/sec  107   1.19 MBytes       
[  5]   8.00-9.00   sec   455 MBytes   455 MBytes/sec    0   1.34 MBytes       
[  5]   9.00-10.00  sec   442 MBytes   442 MBytes/sec    0   1.40 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.32 GBytes   443 MBytes/sec  291             sender
[  5]   0.00-10.04  sec  4.32 GBytes   441 MBytes/sec                  receiver

iperf Done.

 worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 443 MBytes/sec.
[2024-09-17T21:38:47Z] command: iperf3 -c 10.254.28.136 -f M
[2024-09-17T21:38:47Z] Namespace: ibm-network-performance, PodName: ibm-network-performance-ds-lgw9d, ContainerName: ibm-network-performance-ds
Connecting to host 10.254.28.136, port 5201
[  5] local 10.254.49.194 port 48044 connected to 10.254.28.136 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   437 MBytes   437 MBytes/sec  101   1.11 MBytes       
[  5]   1.00-2.00   sec   434 MBytes   434 MBytes/sec  240   1.29 MBytes       
[  5]   2.00-3.00   sec   456 MBytes   456 MBytes/sec    0   1.40 MBytes       
[  5]   3.00-4.00   sec   446 MBytes   446 MBytes/sec    0   1.44 MBytes       
[  5]   4.00-5.00   sec   450 MBytes   450 MBytes/sec    0   1.45 MBytes       
[  5]   5.00-6.00   sec   452 MBytes   453 MBytes/sec    0   1.48 MBytes       
[  5]   6.00-7.00   sec   455 MBytes   455 MBytes/sec   48   1.50 MBytes       
[  5]   7.00-8.00   sec   432 MBytes   432 MBytes/sec    0   1.51 MBytes       
[  5]   8.00-9.00   sec   448 MBytes   447 MBytes/sec    0   1.52 MBytes       
[  5]   9.00-10.00  sec   455 MBytes   455 MBytes/sec   96   1.52 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.36 GBytes   447 MBytes/sec  485             sender
[  5]   0.00-10.04  sec  4.36 GBytes   445 MBytes/sec                  receiver

iperf Done.

 worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com client -> server worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com: 447 MBytes/sec.

Deleting Namespace ibm-network-performance ...

################################   Summary   ###############################

[32m[SUCCESS...] [0m All nodes that will be tested are ready
[32m[SUCCESS...] [0m All pods were successfully created and their containers are running
[32m[SUCCESS...] [0m All nodes tested met the minimum expected network throughput of 350 MBytes/sec

Inter-node communication throughput (MBytes/sec):

          	0 (client)	1 (client)	2 (client)
0 (server)	-         	469       	403       
1 (server)	448       	-         	450       
2 (server)	443       	447       	-         

Legend (Node Information):

NodeID	Node Internal IP	Node Name                                             
0     	10.254.32.104   	worker1.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
1     	10.254.49.194   	worker2.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com
2     	10.254.28.136   	worker3.sre-wld-3m12w-fips-nfs-452-exp.cp.example.ibm.com

Network Performance Check Result: [32m[SUCCESS...] [0m

Network performance info gathered successfully!
Standard run with the --save option 
cpd-cli health network-performance --save
When you use the --save option, the network-performance command saves the command's output and the verbose output to a compressed file with a name such as Network-Performance-YYYY-MM-DD_HH-MM-SS.zip. The compressed file is saved to the cpd-cli-workspace directory.