White Papers
Abstract
This tutorial series explains how to configure and use DataPower GitOps template objects with the GitOps object. GitOps template objects are used to transform the configuration on both GitOps read and GitOps write directions. The transformation of configuration occurs by using the DataPower implementation of JSONata. In the first part, you learned how to configure GitOps and perform basic read and write operations. In Part 2, you learn how to develop and test GitOps templates as well as push them to source and attach them to GitOps read and write operations.
Content
Introduction
GitOps Template is a new Configuration object on DataPower, which consists of an array of template entries. Each template entry represents a modification of configuration to occur on a GitOps Read and Write.
In version 10.6.0, two types of templates exist for configuration: Change and custom.
Change templates
These templates define a modification of an existing and not null key within the JSON configuration. The key is captured for transformation by constructing a JSONata capture by using the ClassName, Name, and Field defined within the template entry. On GitOps Write, the template transforms the captured keys value to a generated template name by using the ClassName, Name, and Field. On GitOps Read, the template transforms the captured keys value from generated template name to the configured value defined in the template entry.
Custom templates
These templates are for advanced use where extra logic or modification of an existing configuration is required. With custom templates, the key is also captured for transformation by constructing a JSONata capture by using the ClassName, Name, and Field. However, the Field can be extended to contain selection sets for filtering. The user is also responsible for writing the Write Transform (value field), Read transform (value-inverse field), and completing the Query (value-validate field).
Simple change example
Given a configuration that contains an HTTPService named DummyHttpServ. A template can be created to modify the LocalPort from its existing value to a new value of 8080.
In CLI, the template configuration looks as follows.
template change HTTPService DummyHttpServ LocalPort 8080
From a JSON configuration perspective, the template would display as follows through each step of the Write Read lifecycle.
| On DataPower, before any git-operation | On Git, after the template write transformation (GitOps Write) | On DataPower, after the read transformation (GitOps Read) |
|
|
|
Development and writing of templates
Before we move on to more complex examples of templating this section describes useful commands settings and tips to develop, debug, test, and push GitOps templates.
- Dry-run: GitOps read and write can be used in "dry-run" mode. Dry-run mode produces a file in temporary:/// which can be used for verification of template executions. A dry-run flag during GitOps Read, produces a de-templated cfg.json file. The config source is either from source at the current configured commit hash, or from the file that is specified in the location argument. A dry-run flag during GitOps Write, produces a templated cfg.json file. The config source is either from persisted config on the domain or from the file that is specified in the location argument.
- GitOps configuration as JSON: The
gitops-write-memory-jsoncommand can be used to create a JSON representation of the domains persisted config. This configuration omits any GitOps or GitOps template objects. The JSON produced represents what configuration would be written to it on a GitOps write assuming no templating occurs. This command can serve as a document to identify names, JSON structures, and match patterns when a user is developing templates. - JSONata expressions: On commit of a GitOps template, each template entry produces 3 JSONata expressions. A capture + write transform, capture + read transform, and template query. For reference, these expressions are logged as
[gitops][debug]level messages. When a user develops complex or custom templates, these raw expressions serve as a diagnostic source of truth. Given the prior template example titled Simple Change Example.template change HTTPService DummyHttpServ LocalPort 8080On template commit, the following JSONata expressions are logged.
Template: change_HTTPService_DummyHttpServ_LocalPort created expression: configuration.HTTPService[$match(`@name`, /^DummyHttpServ$/) and $exists(LocalPort) and $not($type(LocalPort) = "null")].LocalPort := "{{ change_HTTPService_DummyHttpServ_LocalPort }}" Template: change_HTTPService_DummyHttpServ_LocalPort created expression: configuration.HTTPService[$match(`@name`, /^DummyHttpServ$/) and $exists(LocalPort) and $not($type(LocalPort) = "null")].LocalPort := "8080" Template: change_HTTPService_DummyHttpServ_LocalPort created expression: $Query := $distinct(configuration.HTTPService[$match(`@name`, /^DummyHttpServ$/)].LocalPort) = "{{ change_HTTPService_DummyHttpServ_LocalPort }}" - GitOps write template: When tested for functionality of write, read, and configuration runtime, a GitOps template can be written to the GitOps configured Git repository by using the GitOps write template action. This action writes or overwrites the
/datapower/templates/<template_name>.cfg.jsonfile within the configured repository. After the template is written, any GitOps object can configure this template for operations under its template profile vector. - GitOps remove template: When no longer needed, a GitOps template can be removed from the GitOps configured Git repository by using the GitOps remove template action. This action removes the
/datapower/templates/<template_name>.cfg.jsonfile from the configured repository. After the template is removed, any existing GitOps template objects remain locally configured on the domain. However, the removed template can no longer be configured in any GitOps template profile vector.
Complex Example
Given a configuration that contains a set of HTTPSourceProtocolHandler objects named HTTP_90, HTTP_443, and HTTP_8080, a template can be created to turn "on", POST allowance for each handler. The name field of a template is wrapped in a regular expression and thus can be used to pattern-match. The POST field as shown in the following JSON snippets is within the AllowedFeatures object and thus dot notation can be used to capture to the specific field to replace.
In CLI, the template configuration looks as follows.
template change HTTPSourceProtocolHandler "HTTP_.*" "AllowedFeatures.POST" on
From a JSON configuration perspective, the template would modify the HTTPSourceProtocolHandler config as follows through each step of the write and read lifecycle.
| On DataPower, before any git-operation | On Git, after the template write transformation (GitOps write) | On DataPower, after the read transformation (GitOps read) |
|
|
|
Object replacement example
Given a configuration, which contains a single object of type SSLClientProfile named Sample_TLS, a template can be created to enforce only TLS V1.3 on the profile. The Protocols field is of type Object and contains multiple fields. By capturing to the Protocols field itself, the value input to the template can represent the new value of the entire Protocols field.
In CLI, the template configuration looks as follows.
template change SSLClientProfile Sample_TLS Protocols "{\"SSLv3\":\"off\",\"TLSv1d0\":\"off\",\"TLSv1d1\":\"off\",\"TLSv1d2\":\"off\",\"TLSv1d3\":\"on\"}"
From a JSON configuration perspective, the template would display as follows through each step of the Write Read lifecycle.
| On DataPower, before any git-operation | On Git, after the template write transformation (GitOps write) | On DataPower, after the read transformation (GitOps read) |
|
|
|
Custom template example 1 (Logic-based transform)
Expanding upon change templates, custom templates allow a user to perform advanced replacement or logical checks. This extended functionality is achieved by manually configuring the final part of the capture, the transform, and the template query for validation. In this example, we have a set of XMLFirewallService objects named Sample_XMLFW_1, Sample_XMLFW_2, and Sample_XMLFW_3. A custom template can be created here to update the HTTPTimeout value, only if it meets the specified criteria of being larger than 120 seconds. For this template example, we use the template editor view entered by using the template keyword while in a GitOps template object. Specifying the template as follows.
idg[testgit](config gitops-template template logic)# show
type custom
class XMLFirewallService
name Sample_XMLFW_.*
field "HTTPTimeout[$ > 120]"
value 120
value-inverse 120
value-validate ""
Notes:
- The
fieldfield is matching a property as well as using a selection set to compare the value within the property against our logical value of 120. - The
value-validatefield is left blank. Because the logic relies on the number comparison and performs identical transformation of value to 120 seconds in each direction, a decision to not perform validation is made for this custom template.
On template commit, the following JSONata expressions are logged.
Template: custom_XMLFirewallService_Sample_XMLFW_.*_HTTPTimeout[$ > 120] created expression:
configuration.XMLFirewallService[$match(`@name`, /^Sample_XMLFW_.*$/)].HTTPTimeout[$ > 120] := 120
Template: custom_XMLFirewallService_Sample_XMLFW_.*_HTTPTimeout[$ > 120] created expression:
configuration.XMLFirewallService[$match(`@name`, /^Sample_XMLFW_.*$/)].HTTPTimeout[$ > 120] := 120
Template: custom_XMLFirewallService_Sample_XMLFW_.*_HTTPTimeout[$ > 120] created expression:
$Query :=
From a JSON configuration perspective, the template would display as follows through each step of the Write Read lifecycle.
| On DataPower, before any git-operation | On Git, after the template write transformation (GitOps write) | On DataPower, after the read transformation (GitOps read) |
|
|
|
Note the value and value-inverse both set to 120 seconds based on the field logic. The result is that HTTPTimeout on Git after the template transformation (GitOps write) and DP after the read transformation (GitOps read) have the same replaced values.
Custom Template Example 2 (Array modification)
In this custom template example, an existing array is modified while we retain existing values. Given a single SSHClientProfile object named Sample_SSHProfile, The Ciphers field is modified to remove any algorithm that contains "CHACHA". For this template example, we use the template editor view entered by using the template keyword while in a GitOps template object. Specifying the template as follows.
idg[testgit](config gitops-template template modifyArray)# show
type custom
class SSHClientProfile
name Sample_SSHProfile
field Ciphers
value "$append($,"{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}")"
value-inverse "$[$match($, /^(?!.*CHACHA.*|{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}).*/)]"
value-validate ".Ciphers[$contains($,"{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}")]"
Note:
- The
valuefield appends a self-created template value for validation on the value-validation step by using the JSONata$appendfunction. - The
value-inversefield performs a match for values NOT containing"CHACHA"<or our self-created template value by using the JSONata$matchfunction. - The
value-validateuses the JSONata$containsfunction to verify whether the self-created template value is contained within theCiphersarray. Thevalue-validateexpects to returntrueon a GitOps write and expects to returnfalseon a GitOps read.
On template commit, the following JSONata expressions are logged.
Template: custom_SSHClientProfile_Sample_SSHProfile_Ciphers created expression:
configuration.SSHClientProfile[$match(`@name`, /^Sample_SSHProfile$/)].Ciphers := $append($,"{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}")
Template: custom_SSHClientProfile_Sample_SSHProfile_Ciphers created expression:
configuration.SSHClientProfile[$match(`@name`, /^Sample_SSHProfile$/)].Ciphers := $[$match($, /^(?!.*CHACHA.*|{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}).*/)]
Template: custom_SSHClientProfile_Sample_SSHProfile_Ciphers created expression:
$Query := configuration.SSHClientProfile[$match(`@name`, /^Sample_SSHProfile$/)].Ciphers[$contains($,"{{ custom_SSHClientProfile_Sample_SSHProfile_Ciphers }}")]
From a JSON configuration perspective, the template would display as follows through each step of the Write Read lifecycle.
| On DataPower, before any git-operation | On Git, after the template write transformation (GitOps write) | On DataPower, after the read transformation (GitOps read) |
|
|
|
Note about idempotency
Custom templates allow the ability to produce configuration changes that are not idempotent through a GitOps write and read cycle. It is important to keep the behavior in mind that when a user executes multiple writes and reads on a domain the template changes configuration on this domain. The changes are written to the Git source on subsequent writes. To use the example previously titled: Custom Template Example 1 (Logic-based transform). After the initial GitOps write then read, subsequent writes would coalesce HTTPTimeout values to no greater than 120 seconds and write them to source. It is recommended to use nonidempotent templates in a single direction such as read-only mode.
Was this topic helpful?
Document Information
Modified date:
13 June 2024
UID
ibm17085892