LI82183: 504 GATEWAY TIME-OUT ERROR DURING UI AND REST API OPERATIONS DESPITE INGRESS TIMEOUT INCREASE

APAR status

• Closed as program error.

Error description

• '504 Gateway Time-out' errors returned for slow operations after
  240s despite having increased the proxy_send_timeout and
  proxy_read_timeout in the ingress-nginx ingress controller
  config.map.
  

Local fix

• You can use the following 2 workarounds that you can use to
  change the default timeout for the juhu component:
  
  Workaround 1:  exec into the juhu pod and replace references to
  '240' with new timeout value such as '600' :
  - kubectl get po
  - kubectl exec -it {management-cluster-juhu-pod-name} -- bash
  - sed -e "s|240|600|g"
  /etc/api-gateway/generated-conf.d/juhu.conf > /tmp/juhu.conf
  - cp /tmp/juhu.conf /etc/api-gateway/generated-conf.d/juhu.conf
  - /usr/local/openresty/bin/openresty -c
  /etc/api-gateway/api-gateway.conf -s reload
  
  Note: This change will be lost if the juhu pod restarts
  (deleted/recreated)
  
  
  Workaround 2: Edit the juhu configmap and add the timeout
  settings
  - kubectl get cm
  - kubectl edit cm {<span style="background-color:rgb(255, 255,
  255);color:rgb(8, 7, 7);font-size:13px;font-family:salesforce
  sans,arial,sans-serif">management-cluster-name-juhu-route-config
  }</span>
  - insert text '<span style="background-color:rgb(255, 255,
  255)">  </span>proxy_send_timeout 600;\n<span
  style="background-color:rgb(255, 255, 255)">
  </span>proxy_read_timeout
  600;\n' right after server '{\n listen 2007 ssl;\n\n'' exactly
  where shown below:
  ---
  
  # Please edit the object below. Lines beginning with a '#' will
  be ignored,
  # and an empty file will abort the edit. If an error occurs
  while saving this file will be
  # reopened with the relevant failures.
  #
  apiVersion: v1
  data: route.conf: "\n# APIM UI\n# ap.....\n\n# Combined
  UI\nserver {\n   listen 2007 ssl;\n\n<span
  style="background-color:rgb(255, 255, 255)">
  </span>proxy_send_timeout 600;\n
  proxy_read_timeout 600;\n   add_header
  'Access-Control-Allow-Origin'\"  ...
  ...
  ---- - restart juhu pod by deleting it.
  
  Note: This change will be kept if the juhu pod restarts, but not
  if an operator restarts.
  

Problem summary

• juhu hard-code 240s timeout for proxying the request. The
  workaround is to increase it to 600s.
  

Problem conclusion

• juhu has been updated to use 600s as the default timeout in
  10.0.3.0/10.0.1.5.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  API CONNECT ENT

• Fixed component ID

  5725Z2201

Applicable component levels