Troubleshooting
Problem
Availability zones related to Amazon Web Service (AWS) are not being retrieved using the IBM Cloud Orchestrator (ICO) integration.
Symptom
The IBM Cloud Orchestrator integration with Amazon Web Service fails even though the access key and secret key work from Amazon Web Service using the command line.
Availability zones related to Amazon Web Service are not being retrieved.
The pcg.log shows the following errors:
[2015-12-15 06:23:42,453] com.ibm.ccs.publiccloud.service.artifacts.PublicCloudEndpoint [ERROR] Validation of the connection information failed, check cloud username and password.
[2015-12-15 06:23:42,453] com.ibm.ccs.publiccloud.service.actions.ConnectEndpointAction [ERROR] connect.call: FAILED
[2015-12-15 06:23:42,454] com.ibm.ccs.publiccloud.model.CloudResourceUtil [ERROR] Unable to connect to public cloud : null
[2015-12-15 06:23:42,457] com.ibm.ccs.publiccloud.model.AvailabilityZoneList [INFO] ### setAvailabilityZones ###[]
Cause
This issue is related to time on the IBM Cloud Orchestrator and IBM Cloud Manager with OpenStack master controller servers being off by six to seven minutes.
Diagnosing The Problem
To debug the problem, enable DEBUG in log4j.properties as follows:
- Edit log4j.properties in /opt/ibm/ico/pcg/etc directory on the ICO server
- Uncomment DEBUG line and comment WARN line as follows:
log4j.logger.org.apache.http=DEBUG
#log4j.logger.org.apache.http=WARN
- Restart pcg service.
After you enable debug, recreate the problem and you should see the following messages in the pcg log:
[2015-12-15 06:23:42,184] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] >> Connection: Keep-Alive
[2015-12-15 06:23:42,185] org.apache.http.impl.conn.Wire [DEBUG] >> "Action=DescribeAvailabilityZones&Version=2014-10-01"
[2015-12-15 06:23:42,382] org.apache.http.impl.conn.Wire [DEBUG] << "HTTP/1.1 401 Unauthorized[\r][\n]"
[2015-12-15 06:23:42,383] org.apache.http.impl.conn.Wire [DEBUG] << "Transfer-Encoding: chunked[\r][\n]"
[2015-12-15 06:23:42,384] org.apache.http.impl.conn.Wire [DEBUG] << "Date: Tue, 15 Dec 2015 12:28:58 GMT[\r][\n]"
[2015-12-15 06:23:42,384] org.apache.http.impl.conn.Wire [DEBUG] << "Server: AmazonEC2[\r][\n]"
[2015-12-15 06:23:42,385] org.apache.http.impl.conn.Wire [DEBUG] << "[\r][\n]"
[2015-12-15 06:23:42,386] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] Receiving response: HTTP/1.1 401 Unauthorized
[2015-12-15 06:23:42,386] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] << HTTP/1.1 401 Unauthorized
[2015-12-15 06:23:42,387] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] << Transfer-Encoding: chunked
[2015-12-15 06:23:42,387] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] << Date: Tue, 15 Dec 2015 12:28:58 GMT
[2015-12-15 06:23:42,387] org.apache.http.impl.conn.DefaultClientConnection [DEBUG] << Server: AmazonEC2
[2015-12-15 06:23:42,390] org.apache.http.impl.client.DefaultRequestDirector [DEBUG] Connection can be kept alive indefinitely
[2015-12-15 06:23:42,390] org.apache.http.impl.auth.HttpAuthenticator [DEBUG] Authentication required
[2015-12-15 06:23:42,391] org.apache.http.impl.auth.HttpAuthenticator [DEBUG] ec2.eu-central-1.amazonaws.com:443 requested authentication
[2015-12-15 06:23:42,391] org.apache.http.impl.auth.HttpAuthenticator [DEBUG] Response contains no authentication challenges
[2015-12-15 06:23:42,397] org.apache.http.impl.conn.Wire [DEBUG] << "fe[\r][\n]"
[2015-12-15 06:23:42,398] org.apache.http.impl.conn.Wire [DEBUG] << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
[2015-12-15 06:23:42,398] org.apache.http.impl.conn.Wire [DEBUG] << "<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>9667663d-7b5a-4dcf-99aa-faff70d2a68e</RequestID></Response>"
[2015-12-15 06:23:42,399] org.apache.http.impl.conn.Wire [DEBUG] << "[\r][\n]"
[2015-12-15 06:23:42,400] org.apache.http.impl.conn.Wire [DEBUG] << "0[\r][\n]"
[2015-12-15 06:23:42,400] org.apache.http.impl.conn.Wire [DEBUG] << "[\r][\n]"
[2015-12-15 06:23:42,401] org.apache.http.impl.conn.PoolingClientConnectionManager [DEBUG] Connection [id: 0][route: {s}->https://ec2.eu-central-1.amazonaws.com] can be kept alive indefinitely
[2015-12-15 06:23:42,401] org.apache.http.impl.conn.PoolingClientConnectionManager [DEBUG] Connection released: [id: 0][route: {s}->https://ec2.eu-central-1.amazonaws.com][total kept alive: 1; route allocated: 1 of 50; total allocated: 1 of 50]
[2015-12-15 06:23:42,404] com.amazonaws.util.XpathUtils [DEBUG] Ingore failure in speeding up DTMManager
java.lang.NoClassDefFoundError: com.sun.org.apache.xpath.internal.XPathContext
at com.amazonaws.util.XpathUtils.speedUpDTMManager(XpathUtils.java:65)
Resolving The Problem
To resolve the issue, correct the operating system time differences on the IBM Cloud Orchestrator and IBM Cloud Manager with OpenStack master controller servers.
Note: To prevent this issue from happening, synchronize the system clock on all servers with a public ntp server (or make the time difference less than 5 minutes).
Related Information
[{"Product":{"code":"SS4KMC","label":"IBM SmartCloud Orchestrator"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"General Information","Platform":[{"code":"PF016","label":"Linux"}],"Version":"2.5","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]
Was this topic helpful?
Document Information
More support for:
IBM SmartCloud Orchestrator
Software version:
2.5
Operating system(s):
Linux
Document number:
619237
Modified date:
17 June 2018
UID
swg2C1000027