Troubleshooting
Problem
You receive a "Invalid image status in OpenStack: DELETED" message in the IBM SmartCloud Orchestrator/IBM Workload Deployer user interface for a virtual system instance for a deployment request.
Symptom
The deployment request fails.
The IBM Workload Deployer trace log file on Central Server 3, which is the IBM Workload Deployer server, shows the following error:
[2015-04-01 14:32:39:287 CDT] 00000027 MessageLocali 2 com.ibm.websphere.rainmaker.license.report.AuditReportCreator reportInstanceVmsLicenseUsage VMs for instance 18985: [[diskmode=ENABLE_LINK_CLONE,memory=12288,config=null,ipaddressid=null,name=ibm06005,partkey=rhel6template_ibm_04B_sco23_,virtualservergroupid=17785,displayname=ibm06005,partid=164,user_provided_ip=1,cpureserved=F,cloudid=612,elastic=T,templateid=162,virtualsystemid=1/42099407-290b-322b-cdf9-402f0db28805#1,storageid=IBM02_IBMCLU004-B@999.9.99.999-443,pcpucount=null,id=17606,flavor=ec.c4-m12,parentpartkey=null,storagereserved=102400,priority=4,cpucount=4,groupidentifier=null,created=1427247711589,identifier=rhel6template_ibm_04B_sco23.11427247666494,vmnamesettingid=17928,currentstatus=RM01013,hypervisormachineid=IBM02_IBMCLU004-B@999.9.99.999-443,currentmessage=Invalid image status in OpenStack: DELETED,instanceid=18985,startuporder=0,hypervisorid=612,ip_group_id=null,runtimeid=null,location=IBM02_IBMCLU004-B@999.9.99.999-443,poolid=1256,grouptype=null,type=OpenStack,updated=1427263617976,memreserved=null]]
You might also see the following exceptions in IBM Workload Deployer log file:
[2015-04-01 14:32:39:286 CDT] 00045577 RegisterVmTas E com.ibm.ccs.openstack.shim.task.RegisterVmTask register Error creating OpenStack instance
java.lang.Exception: Image 42136952-abe1-ff0b-75cf-540f881d8488 status is not ACTIVE but DELETED
at com.ibm.ccs.openstack.shim.task.RegisterVmTask.register(RegisterVmTask.java:289)
at com.ibm.ccs.openstack.shim.task.RegisterVmTask.doCall(RegisterVmTask.java:107)
at com.ibm.ccs.openstack.shim.task.RegisterVmTask.doCall(RegisterVmTask.java:53)
at com.ibm.ccs.openstack.shim.task.BaseTask.call(BaseTask.java:105)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314)
at java.util.concurrent.FutureTask.run(FutureTask.java:149)
at com.ibm.vespa.util.task.VSPLocalTaskHandle.run(VSPLocalTaskHandle.java:75)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:761)
Cause
The image or template UUID that is being used for the deployment is marked as deleted in OpenStack.
Environment
This issue pertains to IBM SmartCloud Orchestrator 2.3 Fix Pack 1 Interim Fix 6 and IBM SmartCloud Orchestrator Limited Availability Fixes 17, 20, 21, 23, 24, 26, and 28.
Diagnosing The Problem
If you see these errors or exceptions, log into Central Server 3 and complete the following steps:
- Find the UUID that IBM Workload Deployer is using:
- cd into the /drouter/ramdisk2/mnt/raid-volume/raid0/templates/rainmaker-templates/ directory.
- Run find . -name "*.xml" | xargs grep -i "<image you are having a problem with>"
You should see results that are similar to the following text:
./162/osNodeC.xml: label="rhel6template_ibm_04B_sco23" resID="1" key="rhel6template_ibm_04B_sco23"
./162/osNode.xml: description="Red Hat Enterprise Linux 6 (64-bit)" label="rhel6template_ibm_04B_sco23"
./162/osNode.xml: resID="1" elastic="true" key="rhel6template_ibm_04B_sco23">
./162/osNode.xml: <rmpibm:virtualImageEditionRef resID="2" key="name=rhel6template_ibm_04B_sco23version=1.0,build=1,serviceLevel=0"
- cd into directory that were listed in the results for step 1.b. For example, ./162
- Run the ls -ltr command.
You should see results that are similar to the following text:
-rw------- 1 root root 9985 May 22 2014 osNode.xml
-rw------- 1 root root 9838 May 22 2014 osNodeC.xml
-rw------- 1 root root 18425 May 22 2014 42099407-290b-322b-cdf9-402f0db28805.ovf
The file list contains an .ovf file, which is the UUID that you need.
- Copy this UUID. For example: 42099407-290b-322b-cdf9-402f0db28805
- Log into the region server where this image is defined and used.
- Type the following commands:
source /root/keystonerc
glance image-show <image uuid>
The following example shows the glance command follower by the image UUID that was determined in step 3:
glance image-show 42099407-290b-322b-cdf9-402f0db28805
The following information is an example of the results from the command:
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| container_format | bare |
| created_at | 2014-05-22T09:52:38.888688 |
| deleted | True |
| deleted_at | 2015-03-24T18:45:37.424591 |
| disk_format | vmdk |
| id | 42099407-290b-322b-cdf9-402f0db28805 |
| is_public | True |
| min_disk | 0 |
| min_ram | 0 |
| name | rhel6template_km_04B_sco23 |
| owner | 476107bff7cf4bf6a6939f474a96ce24 |
| protected | False |
| size | 0 |
| status | deleted |
| updated_at | 2015-03-24T18:45:37.426931 |
+------------------+--------------------------------------+
Note that the image is deleted.
Resolving The Problem
If the image is marked as deleted, a new UUID for the image has been created. On the region server, you can run the glance image-list command to verify. This situation can occur if the template is converted to a virtual machine, changes are made, and then it is converted back to a template.
To resolve this issue, create a new image using the following general steps:
- Delete the original template from IBM SmartCloud Orchestrator and delete the image from the pattern in which it is used.
- Register the new template.
- Insert the new image into the pattern.
- Deploy the pattern
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
swg21702146