HTTP transformations

Lua is a lightweight, high-level, multi-paradigm programming language designed primarily for embedded use in applications.

HTTP transformation rules that are used to control the processing of a request by IBM® Application Gateway (IAG) can be easily written as a Lua script.

Detailed information on Lua scripting can be found in the Lua reference manual: Lua 5.4 Reference Manual. Because Lua is a widely used scripting language, many resources are available on the web to help you learn Lua. You can start learning Lua with Programming in Lua. It is available in paperback and as an e-book. The first edition is available for free at Programming in Lua (first edition).

IAG provides custom Lua modules to control the processing of the request. See Custom Lua modules.

Since Lua is a lightweight and performant scripting language, and the custom Lua modules operate on live data, the performance impact of calling simple Lua transformation rules is negligible. This means that you can invoke Lua transformation rules on every request to handle tasks such as inserting custom HTTP headers into responses.

Configuring HTTP transformation rules

Transformation rules can be applied to HTTP requests or responses by the application gateway. Rules themselves can be attached based on a combination of host, paths and methods. See HTTP Transformations Request , HTTP Transformations Preazn , HTTP Transformations Postazn , HTTP Transformations Postauthn , and HTTP Transformations Response for complete details of the matching properties.

Different triggers are used to indicate that the transformation should take place at different stages during processing:

Trigger Description
request The rule is triggered as soon as the request is received by IAG.
preazn The rule is triggered immediately before the authorization decision has been made. This option should be used if you wish to provide additional information to the authorization engine.
postazn The rule is triggered immediately after the authorization decision has been made. This option should be used if you wish to use credential or session information in your rule.
postauthn The rule is triggered immediately after the authentication decision has been made. This option should be used to add additional information to the generated user session.
response The rule is triggered after the response has been received from the resource server.
policies:
  http_transformations:
    request:
      - name: insert-custom-request-identifier-header
        host: www.test.com
        paths:
          - "/reports/*"
          - "/myapp/*"
        method: GET
        rule: "@req_insert_header.lua"
    preazn:
      - name: insert-request-attributes
        host: www.test.com
        paths:
          - "/myapp/*"
        method: POST
        rule: "@req_insert_attributes.lua"
    postazn:
      - name: insert-credential-attribute-header
        host: www.test.com
        paths:
          - "/reports/generate"
          - "/reports/export"
        method: POST
        rule: "@req_insert_header.lua"
    postauthn:
      - name: insert-session-information
        host: www.test.com
        authentication_mechanisms:
          - "oidc"
        rule: "@req_insert_session_attributes.lua"
    response:
      - name: apply-content-security-policy
        host: www.test.com
        paths:
          - "*"
        method: GET
        rule: "@resp_csp.lua"
Tip: IAG also supports legacy XSLT HTTP Transformation Rules from IBM Application Gateway. To configure an XSLT rule, just provide the XSL rule in place of the Lua rule. Lua transformation rules execute significantly faster than the legacy XSL engine rules and are much simpler to author. It is strongly recommended to consider rewriting any legacy XSL rules in Lua.

Utility Lua scripts

Utility Lua scripts let you define reusable Lua modules that can be shared across multiple HTTP transformation rules. This capability allows you to centralize common logic and functions, improving maintainability, and reducing duplication.

Utility scripts are defined in the utilities section under http_transformations and can be loaded from any transformation rule by using Lua’s standard require() function.

Configuring utility scripts
Each utility script requires two fields:
  • name: The name of the utility module (used when requiring it in transformation rules)

  • rule: The Lua script content (can be inline or referenced from a file using the @ prefix)

policies:
  http_transformations:
    utilities:
      - name: LoggingUtils
        rule: |
          local M = {}
          function M.log_info(message)
            Control.trace(5, string.format("[INFO] %s", message))
          end
          function M.log_error(message)
            Control.trace(1, string.format("[ERROR] %s", message))
          end
          return M
      - name: JWTHelper
        rule: "@jwt_utils.lua"
    request:
      - name: process-request-with-utilities
        paths:
          - "/api/*"
        method: GET
        rule: |
          local logger = require 'LoggingUtils'
          local jwt = require 'JWTHelper'
          
          logger.log_info("Processing API request")
          
          local token = HTTPRequest.getHeader("Authorization")
          if token then
            local claims = jwt.decode(token)
            logger.log_info("User: " .. claims.sub)
          end

Installed Lua modules

LuaRocks is a package manager for Lua scripts. It allows you to create and install Lua modules as self-contained packages called rocks.

It is not possible to install additional rocks into the container, but several rocks have been preinstalled. The following table lists the preinstalled rocks that are available to the transformation rule scripting:

Rock Description
basexx A Lua library which provides base2(bitfield), base16(hex), base32(crockford/rfc), base64(rfc/url), base85(z85) decoding and encoding.
binaryheap Binary heaps are an efficient sorting algorithm. This module implements a plain binary heap (without reverse lookup) and a 'unique' binary heap (with unique payloads and reverse lookup).
cqueues cqueues is a type of event loop for Lua, except it's not a classic event loop. It does not use callbacks—neither as part of the API nor internally—but instead you communicate with an event controller by the yielding and resumption of Lua coroutines using objects that adhere to a simple interface.
fifo A Lua library/'class' that implements a FIFO. Objects in the FIFO can be of any type, including nil.
http A HTTP library for Lua.
lpeg LPeg is a new pattern-matching library for Lua, based on Parsing Expression Grammars (PEGs). The nice thing about PEGs is that it has a formal basis (instead of being an ad-hoc set of features), allows an efficient and simple implementation, and does most things we expect from a pattern-matching library (and more, as we can define entire grammars).
lpeg_patterns A collection of LPEG patterns.
luaossl

luaossl is a comprehensive binding to OpenSSL for Lua. It includes support for certificate and key management, key generation, signature verification, and deep bindings to the distinguished name, alternative name, and X.509v3 extension interfaces. It also binds OpenSSL's bignum, message digest, HMAC, cipher, and CSPRNG interfaces.

IBM Application Gateway includes an extended version of this module along with following capabilities:
  • ECDH key derivation: Supports key derivation for EC256 keys.
  • Additional Authentication Data (AAD): Enables AAD support for AEAD cipher modes such as AES-GCM and AES-CCM.
  • JWE support in Lua: Allows the construction of JSON Web Encryption (JWE) objects using pure Lua.

For more information on the extended version, see Comprehensive OpenSSL Module for Lua.

luasocket LuaSocket is a Lua extension library that provides support for the TCP and UDP transport layers, commonly needed by applications that deal with the Internet.
lua-cjson Lua CJSON provides fast UTF-8 JSON parsing/encoding support for Lua.
redis-lua A Lua client library for the redis key value storage system.
urlencode A URL encoder/decoder with native extension.

Custom Lua modules

IAG provides a number of custom modules which can be used to control the processing of the request.

Detailed documentation of the custom Lua modules can be found on the Lua Module Documentation page.

Module Description
Authentication This module allows you to provide authentication information to the EAI authentication module to generate an authenticated session. It can be used, in conjunction with EAI, to produce a Lua scripted authentication mechanism. The functions within this object are only valid for postazn rules.
Authorization This module allows you to set the results of a custom authorization decision. The functions within this object are only valid for preazn rules.
Client This module allows you to retrieve information associated with the client of the request.
Control This module allows you to control the processing of the request.
HTTPRequest This module allows you to retrieve details of the current HTTP request, and modify the current HTTP request. The request information, with the exception of the body, is available on all triggers, but the request can only be modified on a request or postazn trigger.
HTTPResponse This module allows you to retrieve details of the current HTTP response, and modify the current HTTP response.
Session This module allows you to access details of the current user session. The user session information is not available for request triggers.
Session This module allows you to access details of the current user session. The user session information is available for response triggers and is not available for request triggers.

Debugging

There is a command line tool which allows Lua transformation rules to be developed and tested offline, iag_lua_transformation_test. This command is available in the IAG container image.

The input to this command is a YAML representation of the context and the Lua transformation rule script. The context of the request includes the HTTP request, HTTP response and session information.

The output from the binary is a description of the transformation actions which would be performed by IAG.

Example usage of the iag_lua_transformation_test tool

The following scenario tests a simple transformation rule using the command line tool.

The example YAML representation of the context, named context.yaml:

http:
  request:
    method: "GET"
    uri:    "/test.html"
    headers:
      - name:  "User-Agent"
        value: "curl"
      - name:  "Host"
        value: "www.ibm.com"
  response:
    status: 201
    headers:
      - name:  "Origin"
        value: "https://www.ibm.com"
session:
    id:       "<Session ID>"
    username: "testuser"
    credential:
        attributes:
          - name:  "AZN_CRED_PRINCIPAL_NAME"
            value: "testuser"

The example transformation rule, named rule.lua:

if Session.containsCredentialAttribute("AZN_CRED_PRINCIPAL_NAME") then
    HTTPRequest.setHeader("IV-USER", Session.getCredentialAttribute("AZN_CRED_PRINCIPAL_NAME"))
end

if HTTPResponse.containsHeader("origin") then
  if not (HTTPResponse.containsHeader("access-control-allow-origin")) then
    HTTPResponse.setHeader("access-control-allow-origin", HTTPResponse.getHeader("origin"))
  end

  if not (HTTPResponse.containsHeader("access-control-allow-methods")) then
    HTTPResponse.setHeader("access-control-allow-methods", "GET,POST,OPTIONS")
  end

  if not (HTTPResponse.containsHeader("access-control-max-age")) then
    HTTPResponse.setHeader("access-control-max-age", "86400")
  end
end

In this docker example, the context and rule are present in the current directory, which is mounted into a temporary container at /lua_test. Passing in the context and rule produces output which describes the changes IAG would make when applying this rule in this context:

~/test$ ls
context.yaml  rule.lua
~/test$ docker run --rm -v "${PWD}":/lua_test \
  --entrypoint iag_lua_transformation_test \
  icr.io/ibmappgateway/ibm-application-gateway:26.06.0 \
  -c /lua_test/context.yaml -s /lua_test/rule.lua
Request header: set (IV-USER,testuser)
Response header: set (access-control-allow-origin,https://www.ibm.com)
Response header: set (access-control-allow-methods,GET,POST,OPTIONS)
Response header: set (access-control-max-age,86400)

Generating sample context YAML

To assist with the generation of the YAML representation of the context of the request, the iag_lua_transformation_test command can be started with the -t option to produce a template context YAML file. For example:

docker run --rm \
  --entrypoint iag_lua_transformation_test \
  icr.io/ibmappgateway/ibm-application-gateway:26.06.0 \
  -t

The produced template YAML context:

http:
  request:
    method:   <method>
    url:      <url>
    version:  <version>
    protocol: <protocol>
    body:     <body>
    headers:
      - name:  <hdr-name>
        value: <hdr-value>
    cookies:
      - name:  <cookie-name>
        value: <cookie-value>
  response:
    version:  <version>
    body:     <body>
    status:
      code:    <code>
      message: <message>
    headers:
      - name:  <hdr-name>
        value: <hdr-value>
    cookies:
      - name:  <cookie-name>
        value: <cookie-value>
session:
  id:       <id>
  username: <username>
  credential:
    attributes:
      - name:  <attribute-name>
        value: <attribute-value>

Capturing context YAML

The dumpContext function within the Control Lua module can be called from within a running IAG instance to capture a YAML representation of the current request context. For example:

print(Control.dumpContext())

Tracing

Print and trace statements can be used in in Lua transformation rules scripts.

Printing to container standard out

The output from any print statements will be sent to the container standard out. For example:

print(string.format("Host header: %s",
                    HTTPRequest.getHeader("host")))

Tracing using the tracing framework

The Control custom module contains a trace function which can be used to trace a string using the tracing framework. Use the pdweb.http.transformation tracing component to activate the tracing.

For example:

  1. Configure pdweb.http.transformation in the IAG YAML:

  2. Use the Control.trace function to write to the tracing framework.

logging:
  tracing:
    - file_name: /var/iag/lua_trace.log
      component: pdweb.http.transformation
      level: 5
Control.trace(5, string.format("Host header: %s",
                   HTTPRequest.getHeader("host")))

Example transformation rules