Private network problems can be caused by disconnected,
misconnected, or damaged Ethernet cables. They can also be caused by a management console (MC or
HMC), CEC enclosure, or intelligent power distribution unit (iPDU) problem or loss of power.
This
MAP procedure uses visual symptoms to locate and isolate the failure. It uses information from
related open serviceable events to help with locating visual symptoms and isolation.
Display open serviceable events on both management consoles (MCs).
Note: Serviceable events with the following SRCs are displayed only on the MC that created them.
They are not replicated on the partner MC.
BE17xxxx
BEB0001x
BEB10012
BEB20010
BEB20020
BEB20021
BEFxxxxx
Exxxxxxx
Are there any open serviceable events that list a FRU that is related to a network or
communication heartbeat issue?
A power problem that causes an active Ethernet port to not have power needs to have a separate
open serviceable event.
FRUs that contain an active Ethernet port are part of the management consoles, CEC
enclosures, rack control hubs, and iPDUs. Rack control hubs are not listed in a FRU list. They are
isolated by using visual symptoms.
Cable FRUs and rack control hub FRUs are not listed in a FRU list. They are isolated
by using visual symptoms.
Note: The second copy is used when the related serviceable events cannot directly repair the
problem.
Exit this procedure. Answer that you have not isolated the problem yet.
Note: This copy of the MAP is closed to so you can work with the serviceable events.
Display and repair the related serviceable events.
Note: If the problem is not repaired, return here in the second copy of the MAP you just opened, and
continue.
Use Table 1 to find your purpose for using this
MAP.
Table 1. Entry for storage facility private network problems
Purpose
Go to:
You were sent here from a serviceable event with an SRC (system
reference code) of: B3xxxxxx, BE193001, BE230010, BE230011, BE230012, BE230016,
BE230019, BEB10016, BEB10017
If you are doing or just completed a service action that
affected the Ethernet cables, verify that those cables are correctly connected by using the cable
location code labels and LED indicators. Refer to the network point-to-point figures in MAP7200 Section-6 Cable diagrams for private Ethernet networks (black and gray). If no problems are found, continue to the next
step.
Each CEC enclosure uses four Ethernet connections. One connection goes from each HMC to the CEC
FSP (flexible service processor) ports T1 and T2 (labeled "HMC1" and "HMC2"). A second connection
goes from each HMC to the LPAR Ethernet adapter ports T1 and T2.
If both LEDs on a CEC FRU are off, the probable failing FRU is in the CEC enclosure.
Each iPDU has one Ethernet connection.
Each Ethernet rack control hub (at the bottom rear of the rack) has 10 ports (9 are used if an
expansion rack is present, otherwise 8 are used). This FRU is not directly listed in a serviceable
event FRU list.
Each management console has two Ethernet cable connections for the private networks.
If you just fixed a cable problem, exit this procedure and close any related open serviceable
events.
If you did not just fix a cable problem, there are no visual symptoms. Go to step 8.
An active Ethernet port LED with a cable connected is off.
Normally, if the active Ethernet port LED is off at one end of the cable
interconnection, the active partner Ethernet power LED is off at the other end.
The possible failing FRUs are:
The FRU containing the port LED at one end of a cable connection.
The FRU containing the port LED at the other end of a cable connection.
The cable between the ports.
Normally, if the cables and couplers pass the visual inspection, they are not the problem.
To replace a FRU containing the port LED that is off, go to step 7.
Display open serviceable events and their FRU lists.
Is the FRU that you want to replace listed in a serviceable event FRU list?
Yes. Exit this procedure and use that serviceable event to replace the FRU. If that does not
fix the problem, replace the remaining FRUs from step 6.
No. If you want to replace or correct Ethernet cable connection, exit this MAP. Ethernet
cables can be hot-plugged, but there is no HMC guided procedure for hot-plugging a cable. After the
cable issue is corrected, use visual symptoms to ensure that the problem is fixed.
Most storage facility private network problems are reported with SRCs of BEB1xxxx. There are a
few exceptions where the problems are reported with SRCs of B3xxxxxx that are from the POWER® products. When this occurs, use Table 2 to determine the equivalent storage
facility BEB1xxxx SRC and the appropriate action.
Table 2. Actions for B3xxxxxx SRCs
SRC in serviceable event and definition
Equivalent storage facility SRC and definition
Action
B3010002 - HMC or partition connection monitoring fault
If MAP7200 Section-3 Visual checks does not list a visual symptom, you are instructed to use
the Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10021 in place of B3030001
If the gray network fails, substitute SRC BEB10022 in place of B3030001
B3030002 - A single partition HMC link failed.
BEB10041 (black) Network Surveillance LINK_PART_HMC_REDUND: Single HMC lost link to single
partition on a system, the path through the 172.16-BLACK network is not available, but the other
network is okay.
BEB10042 (gray) - Network Surveillance LINK_PART_HMC_REDUND: Single HMC lost link to single
partition on a system, the path through the 172.17-GRAY network is not available, but the other
network is okay.
BEB10043 - Not sure which network lost link.
The B3030002 SRC does not specify which private network (black or gray) failed.
If MAP7200 Section-3 Visual checks does not list a visual symptom, you are instructed to use
the Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10041 in place of B3030002.
If the gray network fails, substitute SRC BEB10042 in place of B3030002.
B3030003 - Multiple partition HMC links failed.
BEB10050 - Network Surveillance LINK_M_PART_HMC: Single HMC lost links to
multiple partitions on single system; both paths are not available; FSP to HMC link is still
working
B3030004 - All partition links for a single system to HMC failed.
BEB10060 - Network Surveillance LINK_A_PART_HMC: Single HMC lost links to all
partitions on single system; both paths are not available; FSP to HMC link is still working
B3030008 - One HMC link to more than one HMC occurred.
BEB10100 - Network Surveillance LINK_HMC_HMC: Lost HMC to HMC links; both
paths are not available BEB10101 - Network
Surveillance: The Partner HMC is in the Offline state in the HMC peer domain BEB10102 -
Network Surveillance: The Partner HMC is not properly configured in the HMC peer domain
B303000A - The HMC host links to all managed systems.
BEB10130 - Network Surveillance LINK_HMC_ALL: Lost HMC link to multiple HMCs;
both paths are not available. This condition does not apply to the storage facility management
console (HMC)
If MAP7200 Section-3 Visual checks does not list a visual symptom, you are instructed to use
the Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10011 in place of B303000E
If the gray network fails, substitute SRC BEB10012 in place of B303000E
B303000F - A single partition HMC link failure occurred on a redundant
path.
BEB10041 (black) Network Surveillance LINK_PART_HMC_REDUND: Single HMC lost link to single
partition on a system, the path through the 172.16-BLACK network is not available, but the other
network is okay
BEB10042 (gray) Network Surveillance LINK_PART_HMC_REDUND: Single HMC lost link to single
partition on a system, the path through the 172.17-GRAY network is not available, but the other
network is okay
BEB10043 Not sure which network has lost link.
The B303000F SRC does not specify which private network (black or gray) failed.
If MAP7200 Section-3 Visual checks does not list a visual symptom, you are instructed to use
the Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10041 in place of B303000F
If the gray network fails, substitute SRC BEB10042 in place of B303000F
B3030010 - A single HMC link to one HMC failure has occurred on a redundant
path.
B3100500 - Device Driver Message: mmm dd hh:mm:ss DR-RC02-OPENSYS kernel:
e1000: ethN: e1000_watchdog_task: NIC Link is Down Model
98x: DR-RC02-OPENSYS kernel: r8169: ethN: r8169_watchdog_task: NIC Link is
Down
BEB10011 (black) Network Surveillance NIC_FAILURE: Single HMC physical link is unavailable on
ethernet port - eth0 (172.16-BLACK network)
BEB10012 (gray) Network Surveillance NIC_FAILURE: Single HMC physical link unavailable on
ethernet port - eth3 (172.17-GRAY network)
The B3100500 SRC does not specify which private network (black or gray) failed.
If MAP7200 Section-3 Visual Checks does not list a visual symptom, you are instructed to use the
Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10011 in place of B3100500
If the gray network fails, substitute SRC BEB10012 in place of B3100500
BEB10014 (black) Network Surveillance NIC_FAILURE: Single HMC physical link unavailable on
ethernet port - eth0 on the USB adapter (172.16-BLACK network)
BEB10015 (gray) Network Surveillance NIC_FAILURE: Single HMC physical link unavailable on
ethernet port - eth3 on the USB adapter (172.17-GRAY network)
The B3100501 SRC does not specify which private network (black or gray)
failed.
If MAP7200 Section-3 Visual Checks does not list a visual symptom, you are instructed to use the
Network Topology Tool to determine which network fails.
If the black network fails, substitute SRC BEB10014 in place of B3100501
If the gray network fails, substitute SRC BEB10015 in place of B3100501
The private network is online. Only the connection to the iPDU is failing.
Possible FRUs and causes include:
The iPDU lost input power. All indicator LEDs are off. Work with the customer to restore
power.
The iPDU Ethernet port has an internal error. There are no iPDU power output problems.
Replace the iPDU. If it is not listed in an open serviceable event, exit this MAP and go to MAP1230 Replace a FRU without using a serviceable event.
If the iPDU was swapped from another rack location without using a proper FRU exchange
procedure, it is possible that its static IP address is incorrect. Either contact the next level of
support or try the exchange parts procedure for the rack control hub by using MAP1230 Replace a FRU without using a serviceable event.
The private network is offline. All connections on that network are
failing.
Normally, the only single point of failure FRU that can cause the entire private network to
be offline is the rack control hub. It either failed internally or lost input power. Observe the LED
indicators.
Ensure the Ethernet cable from the management console to the
rack control hub is connected at both ends. See Figure 1, Figure 2.
If the rack control hub LED indicators look normal, it is possible that the management
console needs to be rebooted. If that does not restore the network communication, it might need to
be replaced. It is recommended to call the next level of support before you replace the management
console.
The iPDU communication is restored. Exit this MAP and close any related
open serviceable events.
The iPDU communication heartbeat test reached the failure threshold.
Yes. Exit this MAP and repair the serviceable event.
