Hadoop integration using a standard Guardium S-TAP

Learn how to integrate Hadoop using a standard Guardium S-TAP for HDFS and MapReduce monitoring.

Hadoop deployments include two fundamental components:

Hadoop Distributed File System (HDFS), which is stores data
MapReduce or MapReduce 2), which provides a framework for accessing and analyzing data

Capturing activity on these two components covers basic auditing requirements because all data except management console traffic goes through HDFS.

Be aware that HDFS activity is not auditor-friendly, as it is somewhat like monitoring file access in a relational database. Consider monitoring activity from other components used in your environment, such as Hive, Big SQL, or Impala. These components support monitoring that more closely resembles database accesses.

Redaction and blocking policies

Guardium supports redaction using extrusion rules and blocking using S-GATE Terminate for Hive and Impala. Blocking for BigSQL was supported in V9.x when the S-TAP is used.

For detailed instructions on using redaction and blocking policies with Hadoop, see the IBM Security Guardium Deployment Guide for Hadoop Systems.

Kerberos

Guardium supports the use of Kerberos secure clusters with some restrictions. In order to decrypt Kerberos user IDs, Guardium requires that keytab files be generated and placed in a specific location. Detailed instructions are available in the IBM Security Guardium Deployment Guide for Hadoop Systems.

Attention: Kerberos configuration may be required only if you are using HBase or Hive.