IBM Support

Apply hot fix for IBM watsonx Orchestrate

Fix Readme


Abstract

To resolve specific issues, install the latest hotfix available for your version of watsonx Orchestrate.

Content

Hotfixes are cumulative. Each new hotfix includes all fixes from previous ones.

You can apply the latest hotfix:;
   • After completing the watsonx Orchestrate installation
   • After upgrading from a previous version
   • When a new hot fix is released for your version 

Note: Installing the latest hot fix automatically includes all prior fixes for that release. 
Important: Hot fixes cannot be downgraded or rolled back once installed. Attempting to downgrade may cause issues with the watsonx Orchestrate service and is not supported.

Before you begin
   Before you apply a hotfix, ensure that:
   • You have verified the current version of watsonx Orchestrate before applying the corresponding hotfix.
   • You have administrator privileges and access to the IBM Software Hub environment where watsonx Orchestrate is deployed.

Who can apply hotfixes

A Cluster-admin can apply or install hotfixes.

Hotfixes by releases:

IBM watsonx Orchestrate release 5.3.1 Patch-5 (5.3.3)

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 0Latest hotfix
Fixed
  • The issue where the UI Proxy pod crashed after upgrading to 5.3.1 Patch 5 due to conflicting read-only mount configurations between parent and child directories was resolved.
  • The issue where connecting to a remote MCP server failed with a 401 authentication error, preventing MCP tools from being added to agents, was resolved.

IBM watsonx Orchestrate release 5.3.1 Patch-2 (5.3.2)

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 0Latest hotfix
Fixed
  • Security issue on password are visible in clear text in logs for TRM from Connections.
  • TRM leaked credentials in logs.

IBM watsonx Orchestrate release 5.3.1

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 4Latest hotfix
Features
  • RTL support for agent responses in chat (CP4D) was implemented.
  • Orchestrate tool-calling (cold start) performance was improved.
  • Support for SSO OBO and Token Exchange in on-prem scenarios was implemented for Connections.
  • SSO OBO, Token Exchange, and Direct Access Token support for MS Teams in on-prem scenarios was ensured.
Fixed
  • watsonx Orchestrate timeout issue when connecting to external agents deployed as watsonx.ai runtime was fixed.
  • Issue where embedded chat loaded successfully with an expired SSO token in the user payload was resolved.
  • Air-gapped agent timeout issue was addressed.
  • Runs API failure in wxo-server (embedded chat setup) during SSO token retrieval for Teams/Entra users was fixed.
  • Handling of tool name hallucination was improved.
  • watsonx Orchestrate release 5.3.1 instance accessibility issue (502 Bad Gateway error) was resolved.
  • Issue of the same SSO token being cached for all users in channels was fixed.
  • LLM response processing error (TypeError due to missing completion_token from Google/LLM) was resolved.
  • Cluster instability caused by executor pod timeout during asynchronous handling, despite successful tool execution, was addressed.
Hotfix 3Included hotfix4
Features
  • As a Builder, you can configure the public APIs that use embed application token. That enables the end user's SSO token. When builders integrate watsonx Orchestrate into their existing chatbot they can use user's SSO token for invoking tools that are configured for SSO.
  • ILMT package is updated for 5.3 with existing PID, new name, RU and Install, VPC non chargeable annotations.
Fixed
  • Error - "Can't connect until invalid transaction is rolled back" was in the chat when running certain tools - Langflow-based remote MCP servers.
  • Customer content PII log issue in an AI Agent is fixed. Now metadata (timestamps, identity, agent name, function calls, return codes, document title) is logged instead of full content, aligning with data privacy policies.
  • wo-ui-proxy was failing on s390x for WxO-5.3.1 with error message.
  • Fixed the STT (Speech-to-Text) inconsistency. The results were returned only in some sessions, causing unreliable voice recognition functionality. Now the system correctly identifies and attaches to the WebSocket connection carrying actual audio data, preventing attachment to the silent connection and ensuring consistent STT results across all sessions.
  • Agents and tools were getting disappeared most of time after applying 5.3.1 Hotfix 3.
Hotfix 2Included in hotfix3
Features
  • You can measure the consumption of watsonx Orchestrate at a service-instance level, for example, tenant-specific consumption.
  • As Agent Builder, you can connect to virtual models exposed
    via authentication methods like OAuth Client credentials.
Fixed
  • The installation issue was fixed, which was stuck in progress state with the error
"sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection
to server.
  • The wo-voice controller failed to start in CPD 5.3.1 due to a missing module dependency. The application attempted to import deque from the pyparsing package at line 18 of wo_archer/voice_runtime/observer/observer.py, but this module was not available in the runtime environment.
Hotfix 1Included in hotfix 2
Fixed

  • Fixed the Upload files feature in Knowledge source that was not working.

  • Provided hotfix/Patch to enable HPA for Agentic task manager #61822

  • Fixed Genesys Bot Connector that was failing on CPD cluster upgraded to 5.3.1HF0.

  • Resolved Tool Calling Inconsistency.

  • Fixed watsonx Orchestrate Dashboard that was failing to load initially in cluster. Previously required removal and re-addition in Genesys dashboard.

  • Fixed Genesys Audio Connector Phone Config upgrade issue from 5.2.2 Hotfix6 to 5.3.1 Hotfix0.

  • Fixed Summary and NBA sections that were not loading in Genesys dashboard once chat was transferred to Human agent in cluster.

  • Fixed watsonx Orchestrate window UI that was getting shrunk in Genesys dashboard in cluster.

  • Fixed Generate summary feature that was not working in watsonx Orchestrate section in Genesys dashboard in cluster.

  • Fixed Pre-Call summary in summary section that was not generating in Genesys in cluster.

  • Fixed Traces API "traces/{{ _.trace_id}}/spans" that were failing to execute in On-prem cluster. 

  • Fixed Enable Observability that was failing in ontap-nas storage enabled environment.

  • Fixed Agent analytics data that was not displaying in Fusion cluster with observability enabled.

  • Fixed Agent analytics data that was not loading in Airgap cluster with HTTP proxy.

  • Fixed hard coded values in deploy-observability script - Documentation Blocker.

Hotfix 0Included in hotfix 1
Fixed

  • Uploaded files in Chat With Documents did not expire after 4 hours, causing user confusion.

  • Catalog search was non‑functional in the 5.3.1 GPU cluster, blocking item discovery.

  • Tool flows that worked in 5.3.0 failed after upgrading to 5.3.1.

  • Domain‑specific agents and tools did not appear in the 5.3.1 listing interface.

  • Operations returned 504 Gateway Timeout errors, causing request failures.

  • Streaming runs failed without a thread_id, forcing unnecessary thread creation.

  • Lists were incorrectly wrapped in a root object in on‑premise deployments.

  • The collaborator pattern failed with React intrinsic agent style.

  • Watson Orchestrate instances could not be created in 5.3.1 Multi region active deployment clusters.

  • GPU‑based tools failed to execute in air‑gapped clusters with HTTP proxy.

  • PII was logged in container and pod logs in on‑premise deployments.

  • OpenAPI tools failed with Gemini virtual models in air‑gapped proxy environments.

  • OpenAPI tools failed with both GPU and virtual models in air‑gapped proxy setups.

  • Slack integration setup failed during installation in on‑premise deployments.

  • Voice chat failed immediately on initiation in CPD 5.3.1.

  • WhatsApp and SMS messages received no responses in CPD 5.3.1.

  • llm_config applied only to OOTB models and not to virtual models.

  • Upgrade from 5.2.2 to 5.3.1 failed in the wo‑uiproxy component.

  • The watsonx Orchestrate code block editor failed to load.

  • Summary and NBA sections failed to load after chat transfer in Genesys.

  • The watsonx Orchestrate UI appeared shrunk and unusable in the Genesys dashboard

IBM watsonx Orchestrate release 5.3.0

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 6Latest hotfix
Features
  • As a Builder, you can configure the public APIs that use embed application token. That enables the end user's SSO token. When builders integrate watsonx Orchestrate into their existing chatbot they can use user's SSO token for invoking tools that are configured for SSO.
Hotfix 5Included in hotfix 6
Fixed
  • The flow runtime LLM invocation was failing intermittently because the LLM provider doesn't always reply with a usage field.
  • AI Gateway supported the OAuth Client Credentials. As Agent Builder, you can connect to virtual models exposed via authentication methods like OAuth Client credentials.
Hotfix 4Included in hotfix 5
Fixed
  • Set in the config provided RCA for error - ATM OOM killed due to the pyodide runtime memory requirements exceed the upper limit.
  • By downgrade to ADK 1.15 you can run the flow.
  • User utterance is not displayed in conversation controller pod logs.
  • Sync API having doc processing flow is executed successfully.
Hotfix 3Included in hotfix 4
Fixed
  • wo-ads-runtime-service pods were showing error: Pod ephemeral local storage usage exceeds the total limit of containers 1824Mi.
  • Conversational controller pod was restarting if user starts a multi-turn conversation in chat UI.
  • Orchestrate was not supported large size so that Assistant could run in large size.
  • OBC resource was not getting bound in recent cluster builds, which was blocking Orchestrate installation in both install modes.
  • Existing Genesys Audio Connector configurations does not work after the upgrade. You must delete the existing configuration and create a new one for the connector to function.
  • The token URL appears to be account or deployment-specific. Derive it from the API URL so that the integration works correctly across different environments.
Hotfix 2Included in hotfix 3 
Fixed
  • This issue is fixed on the appropriate Cross-Origin Resource Sharing (CORS) (as a browser security feature) headers in the zenExtension where the Assistant integration is exposed.
Hotfix 1Included in hotfix 2
Features
  • Features and supports in multi region active deployment architecture:
    • Support for ADK, Knowledge, Catalog, Flows and Flow initial Analysis for implementing, Foundation, Create Agent and Environment with same UUID in clusters, Runtime/Tools, Connections, IDP.
    • Environment variables setting on wxo-microservices pods.
    • The instance name to instance ID mapping is not working in Orchestrate assistant builder.
  • In Runtime is enhanced to support HA set up.
Fixed
  • Granite 3.3 routing accuracy at Orchestrate node was not at an expected level.
  • wo-agentic-task-manager was failing on s390x.
Hotfix 0Included in hotfix 1
Features
  • With Agent Assist, when a human agent receives a call transferred from an AI agent, they can immediately view a summarized transcript of the conversation that occurred prior to the transfer.
  • A pro-code extension is provided to the Agent to do specific access control.
Fixed
  • File upload under Knowledge section was failing in LITE cluster.
  • Chat with Doc was not providing any response in LITE cluster.
  • Assistant builder was not display in Agentic + Assistant cluster.
  • In Manage  Phone menu option was not visible.
  • Most of the times "Timeout was reached" error was displayed for Sync API.
  • Async OpenAPI Tool Execution Stalls was with "Tool is Processing" Message and no further response.
  • Context variables did not pass to agent in embedded chat.
  • Getting error when creating Assistant in Assistant + Agentic cluster.
  • Assistant bots were not deleted in Agentic + Assistant cluster.
  • Unable to invoke Flow or run sync endpoint in CP4D 5.3.0.
  • Sync API was not working for flow having attachment as input.
  • Flow sync point was not working.

IBM watsonx Orchestrate release 5.2.2

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 6Latest hotfix 
Fixed
  • Importing or exporting tools, KB doc uploads was failing due to s3 route config issue.
  • Certificates are added for the language's cert store for TRM pods on start-up.
  • Supporting VLLM model gpt-oss-120b with wxo 5.2.2.
  • Fixed Error: 'NoneType' object has no attribute 'get'.
Hotfix 5Included in hotfix 6
Fixed
  • After applying 5.2.2 hotfix4, the conversational controller pod was crash looping with error.
  • After enabled the embedded chat security the HTML-based embedded chat (the one not passing any JWT token) continued to work, which should not work.
Hotfix 4Included in hotfix 5
Features
  • Following styles are updates or added: React intrinsic is added, Pro code changes in intrinsic style, Collaborator support for intrinsic style.
  • CPD UI proxy logs now include upstream time logs, enabling you to identify performance bottlenecks in upstream services.
  • The runtime is improved to reduce cold‑start issues and prevent workers from generating unnecessary restart requests.
Fixed
  • Environments networking issue.
  • Granite 3.3 routing accuracy at orchestrator node was not at an expected level.
  • ReACT style Granite Agent was unable to access local variable 'correlation_id' error.
  • Collaborator agents were failing to invoke with Llama LLM.
  • Environments problem was that both the wo-connection-manager and wo-connection-manager-service pods in crashloopbackoff.
  • Agent Runtime was losing context when calling the live version of an Agent (Whatsapp, API).
Hotfix 3Included in hotfix 4
Fixed
  • Granite 3.3 Optimisation was not triggered when model was imported via model gateway.
  • Orchestrate lite deleted the OOTB models that was not started by Orchestrate and not enabling models correctly.
Hotfix 2Included in hotfix 3
Fixed
  • In Agents, Pro-code Extensions for Agent Access Control for policies, guardrails, security scope.
  • In Voice, User silence management (API) - The agent should prompt users to make sure they're still there or ask if they need additional time.
  • Orchestrate chat UI timing out even though stream is started.
Hotfix 1 Included in hotfix 2
Features
  • Performance enhancement:
    • Docker Images were updated.
    • Resource configuration Changes for Executor Deployment.
    • HPA configuration for executor deployment.
    • Log configuration changes in ui-proxy, Log configuration changes in ibm-nginx.
  • Granite model produces better results with temperature 0 instead of the default value 1.
  • Optimized prompt handling for react-type agents on WXO using the Granite model.
  • Z agents on Spyre ReACT Mode Context Management with Granite.
Fixed
  • File upload under knowledge was failing in LITE cluster.
  • Embedded model was not listing under Milvus/Astra DB connection in LITE cluster without IFM.
  • Increased the Copy on Write POOL TTL to 8 hours.
  • The new TTL for the memory POOLs was too low still for multiple use cases with heavier dependencies.
Hotfix 0Included in hotfix 1
Features
  • Document Processing supports Agentic.
Fixed
  • Chat transfer was not working.
  • In Agent Assist incorrect message was asking irrelevant questions to agent.
  • Transfer from self-service to human agent did not produce the right summary.
  • Voice over browser chat was not accepting user input.
  • In Agent Assist, Redis Stream expired unexpectedly due to TTL.
  • In Agent Assist, Chat transfer generates initial NBA record was generated but no subsequent NBA updates happened. • File Download should not work in Forms, showed error “Unsupported field schema for Field”.
  • In Agent Assist, Phone Call Summary does not contain customer or session IDs.
  • File Download did not work in User activity. • Unresolved was showing for Display message response.
  • CPU needed increase the limit for skill-server.

IBM watsonx Orchestrate release 5.2.1

Hotfix no.StatusFeatures and fixes  
(Expand the rows for details)
Hotfix 3Latest hotfix
Fixed
  • Watson Orchestrate lite mode was deleting the OOTB models could not start.
Hotfix 2Included in hotfix 3
Fixed
  • Not able to add user groups (500 error) to Orchestrate (Agentic Only) instance in CPD 5.2.1.
Hotfix 1Included in hotfix 2
Features
  • In Foundation, porting completed for Milvus in WxO/CP4D on S390.
  • The tool sub-process timeout is updated to 120 seconds to accommodate for longer running tools.
Fixed
  • High EOF errors were in Agentic tool executions on CPD Lite 5.2.1 with 1 RPS load compared to previous release.
  • skill-server component was failing on s390x cluster with the latest WxO-5.2.1 case-bundle.
  • Tool flows with Domain agent tools were failing.
  • Watson Orchestrate 5.2.1 was failing because of mfe-landing component on s390x.
  • Watson Orchestrate 5.2.1 was failing due to platform-ui component.
  • Watson Orchestrate 5.2.0.1 was failing to deploy on the s390x architecture due to an issue with the Digital Employee Operator.
  • In Watson Orchestrate 5.2.0.1, Tools were failing to invoke with External Agents using self-signed certificates. • In UAB internal server error was occurring while creating GenAI project.
  • Conversation controller pod crashed while running 3 Request per sec load tests in CPD lite cluster.
  • Third party models were not working on 5.2.1 for flow runtime, if it is configured via connection manager.
  • Python tools with connections were failing to execute in tool flows.

IBM watsonx Orchestrate release 5.2.0

Hotfix no.StatusFeatures and fixes
Hotfix 1Latest hot fix. 

[{"Type":"MASTER","Line of Business":{"code":"LOB76","label":"Data Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSVAUS","label":"IBM watsonx Orchestrate Cartridge for IBM Cloud Pak for Data"},"ARM Category":[{"code":"a8mKe0000008OVAIA2","label":"Operate-\u003EInstall"}],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":""}]

Document Information

Modified date:
18 June 2026

UID

ibm17247038

Page Feedback