Builds of managed entities fail
Builds of managed entities might fail with a Failed to save
error.
Symptom
The builds of managed entities might fail with the following error:
Unable to create orchestration process: Failed to save process
Cause
This problem occurs if the content of the request is large.
Solution
To resolve this problem, complete the following steps:
- Log in to the Vault UI.
- To get the Vault route, run the following command:
NAMESPACE NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD lifecycle-manager cp4na-o-vault cp4na-o-vault-lifecycle-manager.apps.devmgmtcluster.example.com cp4na-o-vault 8200 passthrough/Redirect None
The Vault route is displayed in the HOST/PORT column of the output.
- Append /ui/vault/auth to the Vault route, then open the URL in a browser.
For example, open a URL similar to the following URL:
https://cp4na-o-vault-lifecycle-manager.apps.devmgmtcluster.example.com/ui/vault/auth
- To get the Vault key, run the following command:
oc get secret cp4na-o-vault-keys -o jsonpath --template '{.data.alm_token}' | base64 -d
- Log in to Vault UI with the Vault key.
- To get the Vault route, run the following command:
- Add a maximum request size value for all applications, then the Galileo pod.
- In the Vault UI, click Secrets in the menu bar, and then click lm/1.0.0.
- Click application,custom.
- Switch the view to JSON in edit mode, then add the following value to the application secret:
"spring.kafka.producer.properties.max.request.size": 4194304
- Click galileo,custom.
- Switch the view to JSON in edit mode, then add the following value to the Galileo secret:
"alm.galileo.ldu.streams.max.request.size": 4194304
- Restart the pods for the Talladega, Galileo, and Daytona microservices:
- Update the StatefulSets and Deployments and complete a rolling restart of the affected pods:
oc patch orchestration <instance name> --patch '{"spec":{"advanced": {"synchronizeVaultChanges": true}}}' --type=merge
- When the pods are restarted, reset the
synchronizeVaultChanges
flag:oc patch orchestration <instance name> --patch '{"spec":{"advanced": {"synchronizeVaultChanges": false}}}' --type=merge
When all the pods have a status of Running, the new configuration is in effect.
- Update the StatefulSets and Deployments and complete a rolling restart of the affected pods:
- Increase the maximum bytes value for the Kafka configuration map.
- Edit the Kafka
instance:
oc edit kafka cp4na-o-events
- Increase the value of the
message.max.bytes
parameter to 4194304, as shown in the following sample excerpt from a configuration map:kafka: authorization: superUsers: - ANONYMOUS type: simple config: message.max.bytes: 4194304 offsets.topic.replication.factor: 1
- Restart the Kafka pod by running the following command:
oc delete pod cp4na-o-events-kafka-0
- Edit the Kafka
instance:
- Increase the maximum bytes value for the Ansible®
lifecycle driver.
- To open the Ansible lifecycle driver configuration
map, run the following command:
oc edit configmap ansible-lifecycle-driver
- Locate the
messaging
section in theald_config.yml: |
section in the configmap. - Add the
max_request_size
parameter with a value of 4194304 in theconfig
list, as shown in the following sample excerpt from a configuration map:ald_config.yml: | ansible: max_unreachable_retries: 60 output_prop_prefix: output__ unreachable_sleep_seconds: 5 messaging: api_version_auto_timeout_ms: 5000 connection_address: cp4na-o-events-kafka-bootstrap:9092 config: max_request_size: 4194304
- Restart the Ansible lifecycle driver pod by running
the following command:
oc delete pod ansible-lifecycle-driver
- To open the Ansible lifecycle driver configuration
map, run the following command:
- If you are using other drivers, you must increase the maximum bytes value for those drivers in
the same way that you did for the Ansible lifecycle
driver. For example, if you are using the OpenStack driver, add the
max_request_size
parameter in theconfig
list in themessaging
section in theovd_config.yml: |
section in the configmap. Similarly, you can addmax_request_size
in thekubedriver_config.yml: |
section in the Kubernetes driver configmap.