Issues with SKLM DB2/WAS start up after reboot On LINUX OS

Resolving The Problem

We have issues on RHEL 6,RHEL 7 and SLES , where customer complains about SKLM or the WAS coming up much earlier than DB2 and this leads to a racing condition.
The below provides a way to get rid of this:

The start and stop of SKLM WAS is controlled by a SecurityKeyLifecycleManager_was.init script which you would find in /etc/init.d directory. There are startup and kill scripts which are symbolic links to this main script SecurityKeyLifecycleManager_was.init.

These are startup scripts which start on various run levels after your server reboots. If you look into the following directories /etc/rc2.d, /etc/rc3.d, /etc/rc4.d , /etc/rc5.d, on your Linux server you'll find something like this:

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 K01SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 S04SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 K01SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 S04SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 K01SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 S04SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 K01SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

0 lrwxrwxrwx 1 root root 39 Jul 9 12:45 S04SecurityKeyLifecycleManager_was.init -> ../SecurityKeyLifecycleManager_was.init

All the above are startup or kill scripts and have some numbers like S04 or K01, this basically determines the sequence of their startup or kill at that runlevel.

As a solution to this, they can use the scripts which are provided with the SKLM 2.6 installer, these are startup scripts for DB2 and will take care of starting or killing DB2 when the server reboots.


RHEL

For RHEL use the following script (attached below) <path of db2sklmv26_rhel> which will help you start and stop DB2. Similar to SLES the steps remains same.
Rename the script to db2sklmv26, and place this in /etc/init.d, so the file path would be /etc/init.d/db2sklmv26

Modify the DB2_INSTALL_PATH=/opt/IBM/DB2SKLMV26/ in the script, if it is not the default path. File permission should be 755 for this file and owner and group should be root.

Then after this create symbolic links to this file in /etc/rc.d/rc*.d

like:

ls -al /etc/rc.d/rc2.d/S02db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:10 /etc/rc.d/rc2.d/S02db2sklmv26 -> /etc/init.d/db2sklmv26

ls -al /etc/rc.d/rc3.d/S02db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:10 /etc/rc.d/rc3.d/S02db2sklmv26 -> /etc/init.d/db2sklmv26

ls -al /etc/rc.d/rc4.d/S02db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:10 /etc/rc.d/rc4.d/S02db2sklmv26 -> /etc/init.d/db2sklmv26

ls -al /etc/rc.d/rc5.d/S02db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:10 /etc/rc.d/rc5.d/S02db2sklmv26 -> /etc/init.d/db2sklmv26

Notice the Startup script should be at a much lower number than of /etc/rc.d/rc2.d/S98SecurityKeyLifecycleManager_was.init (For example: S02 and S98 correspondingly for Db2 and SKLM)

Similarly, the Kill scripts:

ls -al /etc/rc.ds/rc0.d/K01db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:39 /etc/rc.d/rc0.d/K01db2sklmv26 -> /etc/init.d/db2sklmv26

ls -al /etc/rc.d/rc6.d/K01db2sklmv26
lrwxrwxrwx 1 root root 22 Sep 29 05:40 /etc/rc.d/rc6.d/K01db2sklmv26 -> /etc/init.d/db2sklmv26

Comment out the following line, in /etc/init/db2fmcd.conf,

exec /opt/IBM/DB2SKLMV26/bin/db2fmcd

db2sklmv26_rhel

SLES

Execute the following commands to configure the IBM Security Key Lifecycle Manager DB2 instance owner to start automatically:
<DB_home_dir>/sqllib/db2profile
DB_HOME/instance/db2iauto -on sklmdb26


In the /etc/inittab file :
1. Uncomment the following line
fmc:2345:respawn:/opt/IBM/DB2SKLMV26/bin/db2fmcd #DB2 Fault MonitorCoordinator
2. Append the following line after that :

slp:2345:wait:/bin/sleep 60
tt:23456789:wait:/opt/IBM/WebSphere/AppServer/bin/startServer.sh server1

So your /etc/inittab looks like this (see highlighted portion):

lbssklmapp1:~ # tail -f /etc/inittab
# mo:35:respawn:/usr/lib/fax/faxgetty /dev/modem

# vbox (voice box) getty
# I6:35:respawn:/usr/sbin/vboxgetty -d /dev/ttyI6
# I7:35:respawn:/usr/sbin/vboxgetty -d /dev/ttyI7

# end of /etc/inittab
fmc:2345:respawn:/opt/IBM/DB2SKLMV26/bin/db2fmcd #DB2 Fault Monitor Coordinator
slp:2345:wait:/bin/sleep 60
tt:23456789:wait:/opt/IBM/WebSphere/AppServer/bin/startServer.sh server