Example 4: Obfuscating data

This example shows how to use obfuscation functions.

About this task

It is an example with Operational Decision Manager, but it can be adapted for Decision Intelligence as well.

In this example, you customize the default Operational Decision Manager processing configuration to obfuscate values from time series. Obfuscate functions operate on string values.

The bai:obfuscate-sha256-regex(regex, obj) non-reversible obfuscation function correlates values.
The bai:obfuscate-stars-regex(regex, obj) non-reversible obfuscation function preserves the length of the value.
The bai:obfuscate-base64-regex(regex, obj) reversible obfuscation function operates base64 encoding.

Such customization involves the following steps.

Step 1: Create a dedicated transformation file.
Step 2: Obfuscate data.
Step 3: Use obfuscation schemes.

Procedure

Create a dedicated file for your custom transformation.

Copy the transformation.jslt file and rename it to custom-transformation.jslt.
Reference the new file in your processing-conf.json configuration file.

The definition of the OpenSearch egress now looks like this.

{
      "uid": "46D2CB58-2978-4DF5-909A-80583719A0A3",
      "type": "opensearch",
      "index": {
        "name": "icp4ba-bai-odm-timeseries",
        "docId": {
          "type": "jslt-inline",
          "expression": ".id"
        },
        "mapping": {
          "filename": "opensearch-mapping.json"
        }
      },
      "transformer": {
        "type": "jslt-file",
        "filename": "custom-transformation.jslt"
      }
    }

Obfuscate data.

The bai:obfuscate-base64-regex(regex, obj) function takes two arguments: the business data object and a regular expression to select one or multiple keys in the object that you want to obfuscate.

Edit the custom-transformation.jslt file.

Wrap the processing of the data object with a call to one of these obfuscation functions: bai:obfuscate-sha256-regex(regex, obj), bai:obfuscate-stars-regex(regex, obj), or bai:obfuscate-base64-regex(regex, obj).

For example, to encode values of following data

loanvalidation.loanvalidationrules.in.borrower.firstName
loanvalidation.loanvalidationrules.in.borrower.lastName
loanvalidation.loanvalidationrules.out.report.borrower.spouse.firstName

for this business data object of the time series,

{
  ...
  "data": {
     "loanvalidation.loanvalidationrules.in.borrower.firstName" : [ "John", "Emilien" ],
     "loanvalidation.loanvalidationrules.in.borrower.lastName" : "Doe",
     "loanvalidation.loanvalidationrules.in.borrower.yearlyIncome" : 100000,
     "loanvalidation.loanvalidationrules.out.report.borrower.spouse.firstName" : "Maria"
  }
  ...
}

define your custom-transformation.jslt transformer like this.

...
bai:optional-key-array("errors", .data.odm.errors) +
{
  ...
  "data":
    // encode data
    bai:obfuscate-base64-regex(
      "^loanvalidation.loanvalidationrules.in.borrower.(last|first)Name$|" +
      "^loanvalidation.loanvalidationrules.out.report.borrower.spouse.firstName$",

      // transform raw event
      bai:array-to-single-object(
        flatten([
          bai:flatten-object($sourceIdParts[1] + "." + $sourceIdParts[3] + ".in", .data.odm.input-parameters),
          bai:flatten-object($sourceIdParts[1] + "." + $sourceIdParts[3] + ".out", .data.odm.output-parameters)
        ])
      )
  )
  ...
}

Use multiple obfuscation schemes.

The provided example gathers all the transformation for a business data object of the time series that contains the following data,

{
  ...
  "data": {
     "loanvalidation.loanvalidationrules.in.borrower.firstName" : [ "John", "Emilien" ],
     "loanvalidation.loanvalidationrules.in.borrower.lastName" : "Doe",
     "loanvalidation.loanvalidationrules.in.borrower.yearlyIncome" : 100000
  }
  ...
}

and you have the following requirements.

To replace the borrower's first name and last name with stars.

loanvalidation.loanvalidationrules.in.borrower.lastName
loanvalidation.loanvalidationrules.out.report.borrower.spouse.firstName

To encode a serial number in base64.

loanvalidation.loanvalidationrules.out.report.borrower.ssn.serialNumber

To remove values from the array with the following key.

loanvalidation.loanvalidationrules.in.borrower.firstName

To remove the yearly income field.

loanvalidation.loanvalidationrules.in.borrower.yearlyIncome

Example

...
bai:optional-key-array("errors", .data.odm.errors) +
{
  ...
  "data":
    // replace with '*'
    bai:obfuscate-stars-regex(
      "^loanvalidation.loanvalidationrules.in.borrower.lastName$|" +
      "^loanvalidation.loanvalidationrules.out.report.borrower.spouse.firstName$",

      // encode in base64
      bai:obfuscate-base64-regex(
        "^loanvalidation.loanvalidationrules.out.report.borrower.ssn.serialNumber*$",

        // remove items from array
        bai:empty-array-values-regex(
          "^loanvalidation.loanvalidationrules.in.borrower.firstName$",

          // remove property
          bai:filter-key-regex(
            "^loanvalidation.loanvalidationrules.in.borrower.yearlyIncome$",

            // transform raw event
            bai:array-to-single-object(
              flatten([
                bai:flatten-object($sourceIdParts[1] + "." + $sourceIdParts[3] + ".in", .data.odm.input-parameters),
                bai:flatten-object($sourceIdParts[1] + "." + $sourceIdParts[3] + ".out", .data.odm.output-parameters)
              ])
            )
          )
        )
      )
    )
  ...
}

What to do next

To apply your changes, follow instructions in Updating the processing configuration.
You can then send events to the Kafka ingress topic, which by default is icp4ba-bai-odm-ingress.