Troubleshooting Domino agent - KGB5032I: Not Responding

Body

 ITCAM agent for Lotus Domino Availability workspace shows server status 'KGB5032I: Not Responding'.

You have confirmed that the Domino server is running and functioning fine, so why is the monitoring agent is displaying wrong status?

In this blog entry, we discuss some of the common scenarios for this problem, and steps to resolve those.

This problem may be caused by an incorrect setting in agent configuration, or the Domino server may not be reachable.

Troubleshooting:

Check the logs generated by Domino monitoring agent for error messages. Below are list of some commonly seen problems in the logs.

Domino agent generates multiple logs. The logs pertinent for this problem are as follows ( here INST refers to the agent instance name):

kgbstart_INST.log

host_gb_INST_KGBCLIENT_INST_hexnumber-01.log

Review these logs to troubleshoot the problem.

Below are a few examples of troubleshooting:

Case 1) Verify notes.ini path in agent configuration:

In kgbstart*.log, you may find that the agent client process (kgbclient) is exiting, and restarting repeatedly:

Sun 04/23/2017  7:47:50.09 : kgbstart.bat calling for instance INST
Sun 04/23/2017  7:47:50.09 : KGB_NOTESINIPATH=F:\Domino
Sun 04/23/2017  7:47:50.09 : NotesProgram=C:\Lotus\Domino
...
Sun 04/23/2017  7:47:50.17 : The domino server is started after 0 seconds
Sun 04/23/2017  7:47:50.18 : starting "E:\IBM\ITM\TMAITM6_x64"\KGBCLIENT_INST.EXE from "F:\Domino. Domino PID:"", notes.ini driver:F"
Sun 04/23/2017  7:47:50.20 : KGBCLIENT_INST.EXE exited with return code(-1073741515)
Sun 04/23/2017  7:47:50.20 : timeout exit=0 maxexit=5
Sun 04/23/2017  7:47:53.01 : restarting client

---------

Resolution:

This log snippet shows that the the kgbclient process was not able to start successfully with the configuration provided.

To confirm the notes.ini path, run "sh stat server" on the Domino Server. Check output of this command to check the path in use. The following line from "sh stat server" output shows that the notes.ini path is as follows:

[0D20:0007-17D4]   Server.Path.Configfile = E:\Domino\notes.ini   

The agent configuration file host_gb_INST.cfg shows KGB_NOTESINIPATH=F:\\Domino\\notes.ini setting.

Edit the configuration (by using itmcmd on Unix/Linux, and MTEMS on Windows) to correct the configuration, then restart the agent.

Case 2) Verify Domino server name in agent configuration:

If kgbstart*.log does not show that the kgbclient is exiting, it means that agent is successfully able to start the kgbclient process. You can also check the tasklist (Windows), ps -ef (Unix) for kgbclient process.

kgbstart*.log will show following when the kgbclient process us started successfully, no message for 'kgbclient exited' message of case 1) will appear after this :

Wed 05/03/2017  4:20:59.65 : starting "E:\IBM\ITM\TMAITM6_x64"\KGBCLIENT_INST.EXE from "E:\lotus\domino. Domino PID:"", notes.ini driver:E"

Check kgbclient log, look for error coming from NSPingServer call:

You may see the following message

(5908ECFC.0000-21FC:kgbclient.cpp,4081,"kgbclient::pingServerCluster") NSPingServer failed on server:CN=INST-01\INST/O=ABCDE/C=AU with rc:2055, Message:The server is not responding. The server may be down or you may be experiencing network problems. Contact your system administrator if this problem persists.

---------

Resolution:

In this case there the server name setting is incorrect in agent configuration. Check the 'sh stat server' output:

Server name:            INST-01/INST/ABCDE/AU - XY Notes Mail

The server name in configuration (check cfg file) is:  KGB_SERVER=INST-01\\INST/ABCDE/AU

Notice the typo of backslash after INST-01. Reconfigure the agent (do not directly change cfg file) to correct the server name. Restart the agent.

Case 3) Verify Domino server is reachable:

You have corrected the configuration of Domino agent - corrected the notes.ini path and also corrected the typo in Domino server name. However, the agent availability workspace is still showing Domino server availability as 'KGB5032I: Not Responding'.

Check the latest kgbclient log, look for any errors from NSPingServer call. You may find following errors:

        (590D14B8.000C-EC4:kgbclient.cpp,4081,"kgbclient::pingServerCluster") NSPingServer failed on server:CN=INST-01/OU=INST/O=ABCDE/C=AU with rc:2562, Message:Remote system no longer responding

Or you may find following error:

          (57922C15.0023-1:kgbclient.cpp,3974,"kgbclient::pingServerCluster") NSPingServer failed on server: CN=INST-01/OU=INST/O=ABCDE/C=AU with rc:7267, Message:The remote server is not a known TCP/IP host.

---------

Resolution:

This message indicates that the Domino server is not reachable. Check following with the help of Domino administrator:

- Run 'trace server' in Domino console. Correct problems seen in the trace.

      Run Domino command "trace CN=INST-01/OU=INST/O=ABCDE/C=AU" in Domino console on the system where the Domino server and Domino agent are installed. Check output to find reason for failure to reach the server.  

Alternately, in the Lotus Notes Preferences - Notes Ports and click Trace to discover where it breaks down.

- Check if there is a firewall blocking access to Domino Server port. You may need to open the port to allow kgbclient to connect to it.

- Check ip address to which Domino server is bound. Make sure the ip address is defined in the hosts directory ( Unix: /etc/hosts ;  Windows: C:\WINDOWS\system32\drivers\etc\hosts).

     Add following entry to the hosts file for the Domino server name to resolve to the Domino Server ip address. example:

      <ip><tab><Servername><tab><Fully Qualified Host Name>                  

     e.g.

     9.53.114.33        INST-01        INST-01/INST/ABCDE/AU

Restart the agent.

Check Domino agent availability workspace to check if the problem is resolved.

Subscribe and follow us for all the latest information directly on your social feeds:

  

Check out all our other posts and updates:
Academy Blogs:	https://goo.gl/U7cYYY
Academy Videos:	https://goo.gl/FE7F59
Academy Google+:	https://goo.gl/Kj2mvZ
Academy Twitter :	https://goo.gl/GsVecH