APM V8 - AIX OS agent upgrade fails with KCI1384E KCI1021E KCI1021E could not unload file

Body

Product:

APM V8 OS agent on AIX

Problem

During upgrade of UNIX OS agent on AIX to APM 8.1.4.0.5 following errors were returned and the upgrade failed.

The upgrade generated APMADV_Agents_install.log which showed:

KCI1021E KCI1021E could not unload file /tmp/APMADV_AIX_Agent_Install_8.1.4.0.5/.apm/inst/core-framework/unix/axaix526.tgz
KCI1384E Installation failed.

KCI5047E Installation of core-framework failed.
ERROR: Failed to install Monitoring Agent for UNIX OS.

Or the upgrade fails on unload of uxaix526.tgz:
Installing components.
    Monitoring Agent for UNIX OS [version: 06351403]
KCI1021E KCI1021E could not unload file /tmp/APMADV_AIX_Agent_Install_8.1.4.0.5/.apm/inst/os-agent/unix/uxaix526.tgz
KCI1384E Installation failed.

Solution

The cause of these errors in upgrade has been found to be a process still using binaries of OS agent, and thus installer is not able to apply the fix to upgrade the agent.

Use following steps to check if this is the problem you have:

1) Check to see if the OS agent is running.. Run following command with a user with access to the APM directory.

/opt/IBM/APM/bin/cinfo -r

If agent is running, then stop it.

2) Make sure that no process for OS agent is running by running following command:

ps -ef |grep APM
You may find LogMonitoring process is still running out of the UNIX OS agent directory, for example:
 root 34275506        1   0   Oct 15      -  0:06 /opt/IBM/APM/aix526/ux/bin/fcp_daemon LogMonitoring

If you find this process running, then kill that process:

 kill -9 <pid>

3) You may also have to run the slibclean command ( available on AIX OS ) to clear any currently unused modules in kernel and library memory.

https://www.ibm.com/support/knowledgecenter/no/ssw_aix_71/com.ibm.aix.cmds5/slibclean.htm

4) Try upgrade the agent again.