Agent is running in safe mode
Use this information to troubleshoot an agent that is running
in safe mode. The VM Status for a virtual machine
where the agent is running in safe mode is SAFERUNNING.
Problem
If an error occurs during VM bootstrap, it tries to go to safe mode. When the agent is running in safe mode, the VM Status for the virtual machine isSAFERUNNING.
You can see this status in the VM Status column
of the Virtual Machine perspective section
for a virtual application. In safe mode, only the basic artifacts
are downloaded and installed, such as agent, ssh, firewall,
and deployment inlet. Other artifacts, such as WAS and DB2 parts,
are ignored. This mode is used so that the agent can still be up and
running even when something is wrong. You can look at the logs from
the Instance Console for
problem determination. - Click Patterns and select Virtual Application Instances or Virtual System Instances ( or if in 2.3.3.3 or later).
- Select a virtual application deployment and click Manage.
- Click Logging.
Debug
Look for this message in 0config.log when 5_exec_vm_tmpl.py is
initiated: Try to start agent in safe mode and see
what caused error. For example:
[Mon 03 Jun 2013 07:09:26 PM UTC] maestro/registry.py getRegistry: newer client lifecycle script call ('ITCAMWAS', '1.0')
[Mon 03 Jun 2013 07:09:26 PM UTC] maestro Downloading https://IBMWorkloadDeployer:9443/services/registry/ITCAMWAS/1.0 to /0config/nodepkgs/common/python/maestro/ITCAMWAS.json
[Mon 03 Jun 2013 07:09:26 PM UTC] maestro Invoking curl -g -H X-IWD-Authorization : ..... --cacert /0config/cert.tmp/cert_1.pem -f -o
/0config/nodepkgs/common/python/maestro/ITCAMWAS.json -D - -H StorehouseCommand: unsecuredDownload
https://IBMWorkloadDeployer:9443/services/registry/ITCAMWAS/1.0 --write-out \nSTATUS_CODE: %{http_code}\n
[Mon 03 Jun 2013 07:10:29 PM UTC] maestro % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:00:04 --:--:-- 0
....
....
0 0 0 0 0 0 0 0 --:--:-- 0:01:00 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:01:01 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:01:02 --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- 0:01:02 --:--:-- 0
curl: (7) couldn't connect to host
[Mon 03 Jun 2013 07:10:29 PM UTC] invoker An error occurred; see trace.log for details.
[Mon 03 Jun 2013 07:10:29 PM UTC] invoker Traceback (most recent call last):
File "/0config/nodepkgs/common/python/invoker.py", line 223, in execute
execfile(pyfile, _pyglobals)
File "7_install_itcamwas.py", line 15, in <module>
itcam.was.install(isUpdate=maestro.isUpdate(), isRevert=maestro.isRevert())
File "/0config/nodepkgs/common/python/itcam/was.py", line 89, in install
itcamSS = maestro.registry.getRegistry(was.AGENT_NAME.upper(),"1.0")
File "/0config/nodepkgs/common/python/maestro/registry.py", line 47, in getRegistry
maestro.download(registry_url, registry_file)
File "/0config/nodepkgs/common/python/maestro/__init__.py", line 299, in download
return download_if_changed(url, f, None, user_token)[1]
File "/0config/nodepkgs/common/python/maestro/__init__.py", line 319, in download_if_changed
if rc != 0: raise Exception('Failed to download. curl rc == %d, status code == %s' % (rc, sc))
Exception: Failed to download. curl rc == 7, status code == None
current locale: ('en_US', 'UTF8')
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr Traceback (most recent call last):
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/invoker.py", line 263, in <module>
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr execute()
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/invoker.py", line 223, in execute
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr execfile(pyfile, _pyglobals)
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "7_install_itcamwas.py", line 15, in <module>
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr itcam.was.install(isUpdate=maestro.isUpdate(), isRevert=maestro.isRevert())
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/itcam/was.py", line 89, in install
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr itcamSS = maestro.registry.getRegistry(was.AGENT_NAME.upper(),"1.0")
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/maestro/registry.py", line 47, in getRegistry
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr maestro.download(registry_url, registry_file)
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/maestro/__init__.py", line 299, in download
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr return download_if_changed(url, f, None, user_token)[1]
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr File "/0config/nodepkgs/common/python/maestro/__init__.py", line 319, in download_if_changed
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr if rc != 0: raise Exception('Failed to download. curl rc == %d, status code == %s' % (rc, sc))
[Mon 03 Jun 2013 07:10:29 PM UTC] stderr Exception: Failed to download. curl rc == 7, status code == None
ERROR # 1 : Executing install script 7_install_itcamwas.py
update status to ERROR
CURL_SSL_OPTION = --cacert /0config/cert.tmp/cert_1.pem
updateURL: inputURL = https://[fd8c:215d:178e:710:200:c972:fac2:9314]:9443/services/deployments/d-dce1c09e-6d7a-4400-b583-8f17d7c3c6ae/command
updateURL: newURL = https://IBMWorkloadDeployer:9443/services/deployments/d-dce1c09e-6d7a-4400-b583-8f17d7c3c6ae/command
newURL = https://IBMWorkloadDeployer:9443/services/deployments/d-dce1c09e-6d7a-4400-b583-8f17d7c3c6ae/command
X-IWD-Authorization : { "userdata" : { "tokens" : [ "...." ], "date" : "Mon, 03 Jun 2013 19:10:29 GMT" }, "signature" : "...." }
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 99 0 0 100 99 0 399 --:--:-- --:--:-- --:--:-- 402
Try to start agent in safe mode
Stop node packages firstly
....