Exempting Db2 Data Management Console by using SED on AIX
This topic provides step-by-step instructions for exempting Db2 Data Management Console using Stack/heap Execution Disable (SED) on AIX Java.
Symptoms
While installing the data management console on an AIX server, the installation process reports
some errors of Illegal instruction(coredump)
You get the following
message:
./ibm-datamgmtconsole/bin/check.sh[23]: 31850890 Illegal instruction(coredump)
./ibm-datamgmtconsole/bin/check.sh[23]: 12517816 Illegal instruction(coredump)
./ibm-datamgmtconsole/bin/check.sh[23]: 40632666 Illegal instruction(coredump)
Causes
The Stack Execution Disable Protection (SED) is enabled on the AIX server. It prevents buffer overflow attacks by not executing code in data areas of memory.
Resolving the problem
To resolve this issue, exempt the Db2 Data Management Console from SED's buffer overflow protection.
- Stop the Db2® Data Management Console.
- Go to the Db2 Data Management Console installation
directory
cd <your DMC installation directory>
- Run the following command to exempt all the executable applications under the JRE directory
(
./java_tmp/jre/bin
) of the console:find java_tmp/jre/bin -type f -perm 0755 -exec sedmgr -c exempt {} \;
- Verify that all the executable applications are
exempted.
find java_tmp/jre/bin -type f -perm 0755 -exec sedmgr -d {} \;
Expected output:java_tmp/jre/bin/java : exempt java_tmp/jre/bin/jextract : exempt java_tmp/jre/bin/jjs : exempt java_tmp/jre/bin/jpackcore : exempt java_tmp/jre/bin/jrunscript : exempt java_tmp/jre/bin/keytool : exempt java_tmp/jre/bin/pack200 : exempt java_tmp/jre/bin/rmid : exempt java_tmp/jre/bin/rmiregistry : exempt java_tmp/jre/bin/traceformat : exempt java_tmp/jre/bin/unpack200 : exempt
- Run the
setup.sh
script to install the Db2 Data Management Console.