Security Bulletin: Vulnerability in Apache Log4j affects the components (Elastic Search and Hadoop) of IBM Financial Crimes Insight for Claims Fraud

Workarounds and Mitigations

The recommended solution is to apply the fix for Elastic Search and Hadoop as in steps below as soon as possible.

Steps for Elastic Search:

To fix the log4j vulnerability in Elastic Search for IBM Financial Crimes Insight for Claims Fraud, complete the following steps:

1. Log into OpenShift cluster using oc login from Ambari server.
2. Ensure all Elastic Search pods are healthy and Running.

   oc get po | grep fci-elasticsearch

3. Set the JVM property to apply log4j fix. To set, complete the following commands.

   oc patch sts fci-elasticsearch-master -p '{"spec":{"template":{"spec":{"containers":[{"name":"elasticsearch","env":[{"name":"ES_JAVA_OPTS","value":"-Dlog4j2.formatMsgNoLookups=true"}]}]}}}}'
   oc patch sts fci-elasticsearch-data -p '{"spec":{"template":{"spec":{"containers":[{"name":"elasticsearch","env":[{"name":"ES_JAVA_OPTS","value":"-Dlog4j2.formatMsgNoLookups=true"}]}]}}}}'
   oc patch sts fci-elasticsearch-client -p '{"spec":{"template":{"spec":{"containers":[{"name":"elasticsearch","env":[{"name":"ES_JAVA_OPTS","value":"-Dlog4j2.formatMsgNoLookups=true"}]}]}}}}'

   The Elastic Search pods are restarted automatically after the commands are executed.

4. Ensure all Elastic search pods are restarted.

   oc get po | grep fci-elasticsearch

5. Verify if the log4j fix is applied successfully. The JVM process starts with a new JVM argument -Dlog4j2.formatMsgNoLookups=true.

   oc exec fci-elasticsearch-data-0 -- ps aux
   oc exec fci-elasticsearch-master-0 -- ps aux

   

Steps for Hadoop:

To fix the log4j vulnerability in Hadoop for IBM Financial Crimes Insight for Claims Fraud, complete the following steps:

1. 1. Download the cloudera-scripts-for-log4j-main.zip file.
   2. Copy it to all the Hadoop nodes.
   3. Do the following steps for every Hadoop nodes:
      
      1. Copy the cloudera-scripts-for-log4j-main.zip file to the /root/.
      2. Run the below commands to extract the .zip file:

         cd /root
         unzip cloudera-scripts-for-log4j-main.zip

      3. Run the below command and note down the folder names, such as /usr, /fcigraph, and /grid.

         find / -name log4j*.jar > list_of_impacted_jars.txt

      4. Create a backup folder with the below command.

          mkdir /log4j_backup

      5. Run the following command for each folder found in the preceding step to apply the fix:

         ./run_log4j_patcher.sh hdp -t /usr/ -b /log4j_backup > patch.log 2>&1 &

         Note: In the above command, replace /usr/ with the folder names at the preceding step, such as /fcigraph/, /grid/, etc.

         This process may take 10 to 15 minutes.

   4. Run the following commans to verify:

      cd /log4j_backup
      find . -name *.backup

      Note: This lists all the impacted .jar files that are patched, and the list matches the list_of_impacted_jars.txt.

   5. Restart the impacted services from Ambari console (hive and oozie).