Enabling Apache Ranger policy for resources

IBM® watsonx.data supports Apache Ranger policies for Presto and Hadoop SQL services to allow comprehensive data security on integrating with multiple governance tools and engines.

watsonx.data on IBM Software Hub

Before you begin

Ensure that you have the following details:

  • IBM watsonx.data.
  • Apache Ranger environment.
  • You can only integrate with one of the following policy engines starting with watsonx.data version 2.1.
    • Apache Ranger
    • IBM Knowledge Catalog
    To do that, the watsonx.data instance administrator must select either Apache Ranger or IKC from the watsonx.data UI after the upgrade.
  • The Presto (Java) JDBC URL and credentials in watsonx.data.
  • watsonx.data and Apache Ranger are integrated with LDAP to sync users or groups.

Creating policies for Presto and Spark in Ranger

IBM watsonx.data uses the policies defined under the following service types in Ranger to allow data security on catalogs(Iceberg, Hive and Hudi), buckets, schemas and tables.
  • Presto : Create resource policies in this Ranger service type to enforce security on catalogs(Iceberg, Hive and Hudi), storage, schemas and tables used by Presto engine in watsonx.data.
  • and Hadoop SQL : Create resource policies in this Ranger service type to enforce security on database(Iceberg, Hive and Hudi), schemas and tables used by Spark engine in watsonx.data.

Complete the following steps :

  1. Log in to Apache Ranger by using the username and password.
  2. The Service Manager page lists all the resources and available services under them. For more information about the different resources, see Service Manager.
  3. Click Resource tab and select one of the following resources depending on your use case.
    • PRESTO : Create policies for Presto engine in watsonx.data.
    • Hadoop SQL : Create policies for Spark engine in watsonx.data.
  4. Click the Add New Service (+) icon against the required service type and create a new service to define policies. For more information about the different resources, see Service Manager.
    Note:
    • You can also select an existing service to define policies.
    • To define Ranger policies for Presto, you must create a service under PRESTO section and to define Ranger policies for Spark, you must create a service under Hadoop SQL section.
  5. Create policy against the new (or existing) service. To do that, see Policy Manager.
  6. The service is successfully added in the respective resource list. Click the service name to verify that the default policies are added.
    Note: The testing might fail initially, you can re-test the connection after saving the details since the default policies will be automatically added after saving.

Associating Ranger policies for Presto and Spark in watsonx.data

Complete the following steps to enable and configure Apache Ranger in watsonx.data.

  1. Log in to watsonx.data console.
  2. From the navigation menu, select Access control.
  3. Click the Integrations tab.
  4. Click Integrate service. The Integrate service window opens.
  5. In the Integrate service window, provide the following details:
    Field Description
    Service Select Apache Ranger.
    URL The URL of Apache Ranger.
    Username The admin credentials.
    Password The admin credentials.
    Supported catalogs Select the applicable catalogs for associating Ranger policies.
    List resources Click the link to load the policies ( from both Presto and Hadoop SQL services) from the Apache Ranger server. All the Apache Ranger policies created in Creating policies section gets listed here.
    Resources Select the resource for which the Apache Ranger policy must be enabled.
    Note: To apply the policy for Presto, you must select the resource that is created for PRESTO and for Spark, you must select the resource created Hadoop SQL.
    Policy Cache Time Configuration The time taken to refresh the newly defined Ranger policies.
    Enable data policy within watsonx.data Select the checkbox to enable AMS data policy along with Apache Ranger policy.
  6. Click Integrate. The Apache Ranger policy is integrated and listed in the Access Control page. By default, the policy will be in Inactive state.
  7. Click the overflow menu against the policy and select Activate. On confirming the updates, the policy will be activated.

Verify the integration

Complete the following steps to verify access control :
  1. Log in to watsonx.data instance.
  2. From the navigation menu, click Query workspace. Execute a simple query. The access denied error appears as currently no policies are defined in the Ranger for the user.

Granting permission to users

Complete the following steps to grant permissions to the user:
  1. Log in to Apache Ranger.
  2. Grant the required permission to the test user.
  3. Scroll down to the bottom, click the Save button.
  4. Log in to watsonx.data instance and execute a query again. The access is allowed for the user after adding policies in the Ranger.

Limitations

  • In Apache Iceberg catalog, an error occurs if a policy is not defined for the snapshots views related to the tables in Ranger. You must manually define policies in Apache Ranger to eliminate the error.
  • watsonx.data supports access control for Apache Ranger integration.
  • When configuring Ranger integration using an HTTP URL, you may encounter an error stating SSL and certificate must be provided for Ranger even though certificates are not required for HTTP connections.