Adding fields that are extracted by rule from existing fields

Create a new field whose value is extracted from an existing field.

With the AUTOTAG action you can create a policy that adds a field to a document, where the new field contains a value that is extracted by rule from an existing field in the same document. Two types of rule are available. With the first type of rule, you can extract a value from an existing field by parsing tokens in the field value that are separated by a specified delimiter. With the second type of rule, you can extract a value from an existing field by searching the field value with a regular expression.

Note: The new field can contain a maximum of 256 characters. If the extracted value contains more than 256 characters, it is trimmed to 256 characters before it is assigned to the new field.

In the following two examples, the documents contain a "filepathActiveUsers" field in which the parts of the path are separated by forward slashes (/) and in which the second part of the path contains the project name. For example:

"filepathActiveUsers":"'/Lab1/ProjectHA48/users/active'"

You want to add a field "projectname" that contains the project name, and you want to add the field to all documents that contain the field "user":"'research1'".

The first example shows how to extract the project name by parsing the path into tokens that are separated by a forward slash. Follow these steps:

Create a file autotag_rule_pol_split.dat that contains the parameters for the new policy. The following code block shows the contents of the file:

{						
    "action_id": "AUTOTAG",		
    "action_params": 
    {		
        "rule" : {			
            "name": "setFromExistingField",
            "action": "split",		
            "existingField": "filepathActiveUsers",		
            "delimiter": "/",					
            "fieldNo": 2,						
            "newField": "projectname"				
        }
    },
    "schedule": "NOW",				
    "pol_filter": "user='research1'
}

Create the policy and apply it:

curl -k -H 'Authorization: Bearer <token>' 
https://<spectrum_discover_host>/policyengine/v1/policies/autotagpol2 
-d @autotag_rule_pol_split.dat
-X POST 
-H "Content-Type: application/json"

The policy extracts the second token, the project name, from the path and assigns the value to the new field "projectname".

The second example shows how to extract the project name from the path with a regular expression. Follow these steps:

Create a file autotag_rule_pol_regex.dat that contains the parameters for the new policy. The following code block shows the contents of the file:

{
    "action_id": "AUTOTAG",
    "action_params": {
        "rule" : {
            "name": "setFromExistingField",
            "action": "regex",
            "existingField": "filepathActiveUsers",
            "regexPattern": "[^/]+",
            "matchNo": 3,
            "newField": "projectname"
        }
    },
    "schedule": "NOW",
    "pol_filter": "user='research1'"
}

Create the policy and apply it:

curl -k -H 'Authorization: Bearer <token>' 
https://<spectrum_discover_host>/policyengine/v1/policies/autotagpol2 
-d @autotag_rule_pol_regex.dat
-X POST 
-H "Content-Type: application/json"

The policy extracts the third match to the regex pattern, the project name, from the path and assigns the value to the new field "projectname".