Troubleshooting
Problem
When navigating to Order Hub to a screen where the application triggers a getOrganizationHierarchy API call or internally via the multiApi endpoint. In environments with a large number of organizations, this results in a 504 Gateway Timeout error. After 2–3 repeated attempts, the SMCFS application server becomes unresponsive and goes down.
Symptom
• HTTP 504 Gateway Timeout at the getOrganizationHierarchy endpoint.
• SMCFS app server crash after multiple failed requests.
• Issue is reproducible when the organization count exceeds is huge.
Cause
The root cause is an N+1 recursive query problem in the getOrganizationHierarchy API. When triggered, the API performs a recursive traversal of the entire organization hierarchy. In environments with tens of thousands of organizations, this generates an extremely high number of database queries in a very short time.
Technical details:
• The N+1 pattern generates sequential SELECT queries against YFS_ORGANIZATION and YFS_INBOUND_COMPLIANCE tables — one per organization.
• The inclusion of the SubOrganization element in the API template drives the recursive traversal of all sub-organizations for every org in the hierarchy.
Diagnosing The Problem
Use the following steps to confirm this issue is the root cause:
- Check the HTTP error at the getOrganizationHierarchy or multiApi endpoint
• Capture a HAR log.
• Look for a 504 Gateway Timeout. Confirm that the failing internal API call is getOrganizationHierarchy. - Verify organization count
• Query the YFS_ORGANIZATION table to confirm the total number of organizations. If the count exceeds approximately 30,000, this issue is highly likely to occur. Monitor database query volume
• Coordinate with the DBA team to enable monitoring of long-running queries on the YFS_ORGANIZATION and YFS_INBOUND_COMPLIANCE tables.
• A spike of thousands of sequential SELECT statements within seconds of triggering the Nodes page confirms the N+1 query pattern.
Resolving The Problem
Two resolution paths are available depending on the upgrade readiness of the environment:
Option 1: Upgrade to the latest fix pack (Recommended)
This issue has been fixed and planned in the April 2026 release (26.2.1.0). The fix removes the unnecessary getOrganizationHierarchy calls from the Nodes search form and the Place Order Holds modal within Order Hub.
Option 2: OrderHub customization via multiApi (Workaround for older versions)
If there is no plan to upgrade, the following customization approach can be used to prevent the recursive hierarchy traversal
- Locate the multiApi call that triggers getOrganizationHierarchy.
- Create a custom service to be invoked in place of the default getOrganizationHierarchy call within multiApi.
- Within the custom service, call the getOrganizationHierarchy API using a modified template that removes the
element. This prevents the recursive traversal of all sub-organizations.
Reference documentation: https://www.ibm.com/docs/en/order-management?topic=hub-making-api-calls-in-order
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
31 March 2026
UID
ibm17267159