APAR status
Closed as program error.
Error description
After the daily probe portion of the data collection ran, an alert for Storage System Removed Port occurred for every FC port in a particular DS8000 storage device running 7.8.1.51.159. The ports were not actually removed and there was no impact to the SAN. Similar alerts were not generated for other storage devices. The probe logs showed that the probe had been collecting FC ports and hosts info and then no longer could on a subsequent probe. Working... [EP_DS_CLI_Probe.getLSSandStorageVolumes] [LOG] HWN021733I 4,574 Volumes Found. [EP_DS_CLI_Probe.getIOPorts] [LOG] HWN021713I Collecting fibre channel ports for storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getVolGroups] [LOG] HWNEP0015I Collecting volume groups on storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getConnectedHosts] [LOG] HWNEP0016I Collecting hosts on storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getConnectedHosts] [LOG] HWNEP0017I 110 Hosts Found. Failing... [EP_DS_CLI_Probe.getLSSandStorageVolumes] [LOG] HWN021733I 4,532 Volumes Found. [EP_DS_CLI_Probe.getIOPorts] [LOG] HWN021713I Collecting fibre channel ports for storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getVolGroups] [LOG] HWNEP0015I Collecting volume groups on storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getConnectedHosts] [LOG] HWNEP0016I Collecting hosts on storage system 2107-xxxxxxx. [EP_DS_CLI_Probe.getConnectedHosts] [LOG] HWNEP0017I 0 Hosts Found. The ESSNI niTransmission_0.log shows "Successful: false;" occurring when requesting the data rather than "Successful: true;". NIPhysicalConfigurationService.getFib reChannelPorts(IBM.2107-xxxxxxx):Successful: false; Return code: be740006; Resource: null" NIHostConfigurationService.getHo sts(IBM.2107-xxxxxxx):Successful: false; Return code: be740006; Resource: 110 instances of com.ibm.storage.ess.ni.resource.NISCSIHostImpl" This is caused by an ESSNI memory leak and required recycling ESSNI so that IBM Storage Insights could collect the FC ports and hosts again. The fix for the issue is expected to be in later microcode: R8.5 SP5.2 - 88.55.9.2. https://www.ibm.com/support/pages/ds8000-code-recommendation No fix for IBM Storage Insights is needed.
Local fix
Recycle ESSNI
Problem summary
**************************************************************** * USERS AFFECTED: * * IBM Spectrum Control & Storage Insights users monitoring * * DS8000 storage systems * **************************************************************** * PROBLEM DESCRIPTION: * * For DS8000 storage systems that run microcode levels * * earlier than R8.5 SP5.2 - 88.55.9.2, after a probe you * * might receive alerts that fibre channel ports were removed. * * In most cases, the alerts are invalid and the ports were not * * * * actually removed. This problem occurs because of a * * memory leak in the ESSNI network layer. * * * * * * * * * **************************************************************** * RECOMMENDATION: * ****************************************************************
Problem conclusion
To fix this problem when it occurs, contact IBM Support to help recycle ESSNI before the next probe. For a permanent fix, update your DS8000 to the latest microcode level. For information about recommended microcode levels for DS8000, see: https://www.ibm.com/support/pages/ds8000-code-recommendation.
Temporary fix
Comments
APAR Information
APAR number
IT36044
Reported component name
STORAGE INSIGHT
Reported component ID
5608TPCSI
Reported release
541
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2021-02-25
Closed date
2021-04-05
Last modified date
2021-04-05
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
STORAGE INSIGHT
Fixed component ID
5608TPCSI
Applicable component levels
[{"Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSQRB8","label":"IBM Storage Insights"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"541"}]
Document Information
Modified date:
08 February 2022