Troubleshooting
Problem
In an IBM Spectrum Protect storage environment, multiple spoke servers are monitored from a single hub server. The activity log of the hub server logs ANR2740E and ANR9999D error messages. Remote monitoring of the spoke servers stops.
Diagnosing The Problem
The hub server activity log shows the following messages:
ANR0530W Transaction failed for session x for node <server-name> (Platform) - internal server error detected. (SESSION: x)
ANR1991I Remote monitoring ended for server <server-name>. (SESSION: x)
ANR0486W Session 7 for node <server-name> (Platform) terminated - internal error detected. (SESSION: x)
ANR9999D Thread<n> issued message 486 from: (SESSION: x)
ANR9999D Thread<n> 7ffedd6cae08 outMsgfTrFL()+38 output.c:1276 (SESSION: x)
ANR9999D Thread<n> 7ffedd4c6ddd smServClose()+2ed smserv.c:3450 (SESSION: x)
ANR9999D Thread<n> 7ffedd4e29b3 RemSrvMonThread()+1233 smremmon.c:1638 (SESSION: x)
ANR9999D Thread<n> 7ffedcc07d41 startThread()+141 pkthread.c:3692 (SESSION: x)
ANR9999D Thread<n> 7ffeeae34f7f beginthreadex()+107 (SESSION: x)
ANR9999D Thread<n> 7ffeeae35126 endthreadex()+192 (SESSION: x)
ANR9999D Thread<n> 7ffef45e13d2 BaseThreadInitThunk()+22 (SESSION: x)
ANR9999D Thread<n> 7ffef58454f4 RtlUserThreadStart()+34 (SESSION: x)
ANR2740E Alert monitor could not start or has failed and stopped.
The server dsmffdc.log reports the following errors:
[ FFDC_GENERAL_SERVER_ERROR ]: (monutil.c:2187) Unable to add item to queue(00000089664D3000), itemSize(199) because of rc=2141.
[ FFDC_GENERAL_SERVER_ERROR ]: (monutil.c:2896) error 2141 sending grid 4 notification to OC.
These errors are typically caused by insufficient resources such as a lack of memory or other system resources. In this case, the volume of monitoring data was too large.
Resolving The Problem
To resolve the problem, complete the following steps:
1. To verify the configured value of the Status Refresh Interval, issue the query monitorsetting command.
2. If you are monitoring multiple spoke servers, consider increasing the number of minutes between status monitoring queries from the default setting of five (5) minutes to a higher value.
Use the set statusrefreshinterval administrative command to set the interval. For example, to set the status refresh interval to 15 minutes, issue the following command:
set statusrefreshinterval 15
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg22011282