Linux-UNIX: Hortonworks Ranger architecture and data flow

Learn how the monitoring and audit, and the blocking, are implemented.s

Monitor and Audit

The important difference with this architecture is that the S-TAP is not collecting audit data directly from the Hadoop component; instead the Ranger plugins write the audit messages to log4j, which forwards them to the S-TAP. The S-TAP then sends the messages to the Guardium collector for logging, alerting, reporting, and analytics.

The configuration is quite flexible in that you can install S-TAPs on more nodes. You can configure Ranger to send all component traffic to one S-TAP or you could specify, for example, that all HBase traffic goes to one S-TAP and Hive and HDFS goes to another.

Figure 1. Ranger plugin using log4j as an alternate logging source, forwarding audit data to Guardium
Ranger plugin using log4j

Blocking (Ranger Dynamic Policy integration)

Blocking is implemented by extending Ranger access control policies to honor blocking policy rules that are specified on the Guardium appliance. The actual implementation of blocking is performed as an access denial from Ranger. For more information about how blocking fits into the architecture and data flow and guidance for implementing blocking, see IBM Security Monitoring and Blocking for Hortonworks Hadoop Using Apache Ranger Integration.

For blocking, you need an additional component called the Guardium plug-in for Ranger. This plug-in is called guardium_evaluator.jar and resides alongside the Ranger plugin on the Hadoop component nodes. You need this on the data/slave nodes as well if you want to block HBase.

S-TAPs required: You do not need any additional S-TAPs than what is already required for monitoring/auditing. It makes sense to use the same collector/S-TAP combinations for blocking as you do for auditing.

Prerequisite (Step 0): The administrator sets up filtering conditions on a Ranger policy based on resource, user or group or other conditions allowed by Ranger. For simplicity, let's call this the “watch” criteria. For example, the Ranger policy might specify Scott’s activity against certain resources, because he’s a privileged user. The policy also includes a condition to call the Guardium evaluator plugin. For more information about creating Ranger policies, see the Hortonworks documentation and Ranger tutorials.

The administrator sets up S-TAP to enable integration with the dynamic policies and the firewall. The S-TAP does not have to be directly co-located the Ranger or Guardium plugins.

On the Guardium appliance, install a policy that includes the rule action of S-GATE Terminate for inappropriate access to Hadoop. This rule could include additional criteria such as client IP address or other runtime information.

Figure 2 illustrates the blocking flow, described by the following text.
Figure 2. Blocking flow with Guardium and Hortonworks Ranger
  1. User tries to access a resource that meets the “watch” criteria.
  2. Ranger plugin sends information about this access to the Guardium plugin.
  3. Guardium plugin sends message to S-TAP.
  4. S-TAP sends request to appliance about this access.
  5. If Guardium blocking policy rule conditions are met, the Guardium appliance sends “block” response to S-TAP
  6. S-TAP sends “block” to Guardium plugin
  7. Guardium plugin tells Ranger to not match the original watching rule. This means that if there is no other Ranger policy that allows access to the resource, then access will not be allowed to the resource.