Enabling Knowledge Transformer

Enable Knowledge Transformer to transform documents into AI-optimized taxonomies.

About this task

Knowledge Transformer is disabled by default. When enabled, it deploys a REST API service that processes documents and generates AI-optimized taxonomies. The service integrates with your existing watsonx/LLM configuration.

Before you begin

The transform-secret must be created, see Knowledge Transformer Secret.
The wxa4z-ifm-credentials secret must be configured with valid LLM credentials.

Procedure

Add the transform section to your deploy-zad.yaml

Add the following configuration at the same level as other sections such as opensearch and wrapper

transform:
    enabled: true
    secretName: transform-secret
    createRoute: true
    route:
      annotations:
        haproxy.router.openshift.io/timeout: 30m
    env:
      MAX_ARCHIVE_FILES: "10000"
      LLM_TLS_VERIFY: "false"
    resources:
      requests:
        cpu: 500m
        memory: 512Mi
      limits:
        cpu: 2
        memory: 1Gi
        ephemeral-storage: "8Gi"

Update the relevant configuration fields within the transform section
Required fields:

enabled

Enables or disables Knowledge Transformer. This is disabled by default. Set to true to deploy Knowledge Transformer.

secretName

Name of the secret containing TRANSFORM_API_KEY. The default is transform-secret. The secret must exist before deployment.

createRoute

Set to true to create an OpenShift route. createRoute is true by default. This is required for external access.

route.annotations

Route-level annotations. The default timeout value for route.annotations.haproxy.router.openshift.io/timeout is 30m timeout. Increase this value for very large files.

Optional fields:

env.MAX_ARCHIVE_FILES

The maximum number of files allowed in an archive. The default value is "10000". Adjust this value for your use case.

env.LLM_TLS_VERIFY

Enable or disable TLS verification for LLM connections. The default is "false". Set to "true" for production environments.

env.LOG_LEVEL

Logging level. The default value is INFO, and the available options are INFO, DEBUG, and WARN. For debug logging, set env.LOG_LEVEL to DEBUG.

resources.limits.cpu

Maximum CPU. The default value is 2. Increase this value for heavy workloads.

resources.limits.memory

Maximum memory. The default value is 1Gi. Increase this value for large files.

resources.limits.ephemeral-storage

Temporary storage. The default value is 8Gi. This field is required for file processing.
Optional: Add any advanced configuration where appropriate
For custom CA certificates (when LLM_TLS_VERIFY is "true"), add fields for SSL_CERT_FILE and REQUESTS_CA_BUNDLE with the filepath to your CA certificates:
```
transform:
  enabled: true
  env:
    LLM_TLS_VERIFY: "true"
    SSL_CERT_FILE: "/etc/ssl/llm/llm-cert.pem"
    REQUESTS_CA_BUNDLE: "/etc/ssl/llm/llm-cert.pem"
```

Results

The deployment creates:

A wxa4z-transform deployment with 1 replica
A wxa4z-transform service on port 8080
A wxa4z-transform route (if createRoute: true)
A wxa4z-transform-tls-secret for TLS communication

The deployment may take 2-5 minutes to complete. Monitor progress with:

oc get pods -n wxa4z-zad -w | grep transform