Activation Engine Troubleshooting Guide

This page has not been liked. Updated 1/4/16, 3:59 PM by SamMatzekTags: None

NOTE:  VSAE is deprecated as of PowerVC 1.2.3 .  The PowerVC 1.3.0 Knowledge Center contains this statement:  "VSAE is deprecated and will be withdrawn from support in the next release. It is strongly recommended that new images are constructed with cloud-init. Cloud-init is the strategic image activation technology of IBM. It offers a rich set of system initialization features and a high degree of interoperability."

This is a troubleshooting guide to use when virtual machines deployed by PowerVC do not ping or do not boot.  It also contains troubleshooting information for the pre-capture preparation steps.  The application of the user input IP configuration is handled by an activation engine and its product activations. For an overview of the PowerVC provided activation engine, see Activation Engine Overview.

When a deployed virtual machine does not ping, you must first determine whether it is able to boot into its operating system. The easiest way to determine this is to open a partition console, also known as a console terminal, and try to start the virtual machine.

The console terminal will show the messages and the status of the boot. If the messages eventually show a login prompt, then the virtual machine is booting into its operating system.

There are several ways to open a console terminal. For more information, refer to the following topics:

For HMC: Remote console terminal - HMCconsole, mkvterm and vtmenu

For IVM: Opening a terminal session to your virtual machine

If the partition boots into its operating system continue to "1. The partition boots".

If the partition does not boot into its operating system continue to "2. The partition does not boot." 

If the error is occurring while enabling the activation engine before the capture operating continue to "3. Enabling the activation engine fails or was not run".

If the you are having difficultly with your passed in script not running on AIX, continue to 4. Debugging User Scripts

1. The partition (virtual machine) boots

Determine whether the IP configuration specified during the deploy is configured.

Run the following commands to determine whether the network configuration has been applied:

ifconfig -a  and  netstat -nr

The ifconfig command will show you the network interfaces and their IP addresses. Note if the IP address is the one specified during the deploy or if the interface is missing an IP address.

The netstat -nr command will show the routing table. Check the routing table to see if the default IPv4 gateway that was specified on the deploy is set. Note that the default gateway will be listed with either a destination of "default" or 0.0.0.0.

Here are two examples:

Destination    Gateway        Genmask        Flags    MSS Window    irtt Iface

0.0.0.0        10.32.42.1    0.0.0.0        UG    0 0        0 eth0



Routing tables

Destination    Gateway        Flags    Refs    Use    If    Exp    Groups



Route Tree for Protocol Family 2 (Internet):

default        10.32.42.1    UG    4    1016792    en2    -    -

If the IP address and gateway are both set, continue to "1.1 IP address and gateway are both set."

If either the IP address or gateway is not set, or you see two default gateways, continue to "1.2 IP address and gateway are not set."

If either the IP address or gateway is set, but the values are not the values that were specified on the deploy, continue to "1.2 IP address and gateway are not set."

1.1 IP address and gateway are both set

Run the ping command on the gateway, like this: ping 10.32.42.1

If the ping is successful, there is likely a problem with the network configuration outside of your virtual machine and operating system that is not allowing the virtual machine to be pinged.

If the ping is unsuccessful, there might be a configuration problem with the network or the Shared Ethernet Adapter that is preventing the operating system from reaching its gateway.

1.2 IP address and gateway are not set

Run the following command and note the files that are listed:

ls -l /opt/ibm/ae/AR

If this directory does not exist, it is likely that the activation engine used by PowerVC 1.2 and later, is not installed. Follow the prerequisite capture steps described in the product documentation for PowerVC Express or PowerVC Standard.

On a successful deploy, the list of files should look similar to this:

-rw-r--r-- 1 root root    0 Sep 13 10:27 com.ibm.ovf.vmcontrol.adapter.networking.1.err

-rw-r--r-- 1 root root  194 Sep 13 10:27 com.ibm.ovf.vmcontrol.adapter.networking.1.out

-rw-r--r-- 1 root root    0 Sep 13 10:27 com.ibm.ovf.vmcontrol.system.err

-rw-r--r-- 1 root root    0 Sep 13 10:27 com.ibm.ovf.vmcontrol.system.networking.err

-rw-r--r-- 1 root root  426 Sep 13 10:28 com.ibm.ovf.vmcontrol.system.networking.out

-rw-r--r-- 1 root root  608 Sep 13 10:27 com.ibm.ovf.vmcontrol.system.out

-rw-r--r-- 1 root root 1346 Sep 13 10:28 ovf-env.ar

-rw-r--r-- 1 root root    0 Sep 13 10:28 ovf-env.done

There might be more com.ibm.ovf.vmcontrol.adapter.networking.* files than are listed above. 

If some of the files are missing, the system activation might still be running. Go to "1.2.1 Determining if activation is in progress."

If the list of files looks similar to the list above, continue to "1.2.2 Determining activation status."

If the list of files looks like this, continue to "1.2.3 Capture / restore activations ran."

-rw-r--r-- 1 root root  134 Aug 31 15:09 com.ibm.ovf.vmcontrol.restore.network.err

-rw-r--r-- 1 root root  462 Aug 31 15:09 com.ibm.ovf.vmcontrol.restore.network.out

-rw-r--r-- 1 root root 1200 Aug 31 15:09 ovf-env.ar

If only ovf-env.* files exist like this:

-rw-------    1 root     system         1055 Oct 15 15:07 ovf-env.ar

-rw-------    1 root     system            0 Oct 15 15:07 ovf-env.done



continue to "3. Enabling the activation engine fails or was not run."

If there are no files listed, continue to "1.2.4 No activation ran" and "1.2.5 Activation takes a long time to run".

1.2.1 Determining if activation is in progress

Run the following command: cat /opt/ibm/ae/AR/ovf-env.ar

This will show the overall results file and the status of the various pieces of activation.

<ProductActivation class="com.ibm.ovf.vmcontrol.system.networking">

                        <Execution kind="self" status="Successful"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.out"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.err"/>

                        <Properties>

                                <Property key="dnsIPaddresses" value="192.168.182.4"/>

                                <Property key="hostname" value="myhostname"/>

                                <Property key="ipv4defaultgateway" value="192.168.182.1"/>

                                <Property key="domainname" value="privatenetwork.com"/>

                        </Properties>

                </ProductActivation>

If any of the lines show "In Progress" as a status, the activation is currently running. The activation should be monitored by checking this results file.  If activation takes a long time to complete (over 20 minutes), go to section "1.2.5 Activation takes a long time to run".

If none of the lines show "In Progress" then there are one of two possibilities for activation state:

  1. The results files was viewed between the successful completion of one of the activations and the start of the next activation.
  2. If all the expected activations have not run, the /opt/ibm/ae/AR file list does not match the list from section 1.2, and activations do not after waiting more time, the likely cause is that the VM has taken a very long time to boot and the CD drive has been removed from the virtual machine before all the activations have had a chance to run.  The CD drive is removed from the VM on PowerVM servers by a periodic task.  The minimum amount of time between the start of the deploy and the removal of the CD drive is one hour.  If the VM does not boot and complete activation in this time it can be left with only some of the activations run.

1.2.2 Determining activation status

Run the date command and verify that the time on the operating system is accurate. Compare the timestamps on the log files in /opt/ibm/ae/AR with the date in the operating system and the time the virtual machine was deployed.

If the log files were written before the virtual machine was deployed, then the prerequisite steps were not taken or the the activation engine reset failed. Continue to "3. Enabling the activation engine fails or was not run."

If the timestamp of the log files correlates with the time of the deploy, then one of the product activations might have failed. Check the status of the product activations by running the following command: cat /opt/ibm/ae/AR/ovf-env/ar

For a successful deploy, the file will look like the following example:

NOTES:

  • Depending on the version of the activation engine being used, the input properties might or might not be displayed. They are not displayed in the log file below.
  • "Skipped" is the correct status for "com.ibm.ovf.vmcontrol.restore.network" on a successful deploy.

<?xml version="1.0" ?>

<ar:Activation status="Failed" xmlns:ar="http://www.ibm.com/xmlns/ovf/activation/ar/2">

        <VirtualSystem id="vs0" status="Failed" status-detail="No coordinator">

                <ProductActivation class="com.ibm.ovf.vmcontrol.adapter.networking" instance="1">

                        <Execution kind="self" status="Successful"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.adapter.networking.1.out"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.adapter.networking.1.err"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.restore.network">

                        <Execution kind="self" status="Skipped" status-detail="No properties"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.system">

                        <Execution kind="self" status="Successful"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.out"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.err"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.system.networking">

                        <Execution kind="self" status="Successful"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.out"/>

                        <Log file="/opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.err"/>

                        <Properties/>

                </ProductActivation>

        </VirtualSystem>

</ar:Activation>

If any of the product activations failed, the .out and .err log files might contain error messages that explain why they failed.

If product activations other than 'com.ibm.ovf.vmcontrol.restore.network' have a status of 'skipped' the activation may have run very slowly and the virtual optical may have been removed before activation completed.  See section "1.2.5 Activation takes a long time to run".

1.2.3 Capture / restore activations ran

Run the date command and verify that the time on the operating system is accurate. Compare the timestamps on the log files in /opt/ibm/ae/AR with the date in the operating system and the time the virtual machine was deployed.

If the log files were written before the virtual machine was deployed, then the prerequisite steps were not taken or the the activation engine reset failed. In this case, it is likely that the prerequisite steps were taken but the virtual machine was restarted and shut down before it was captured. Continue to "3. Enabling the activation engine fails or was not run."  Ensure that the prerequisite steps are performed again if the virtual machine was restarted before it was captured.

If the timestamp of the log files correlate with the time of the deploy, then the capture and restore activations ran on the deployed virtual machine. In general, this will occur on a deploy if the activation engine cannot find its input file, ovf-env.xml, on a virtual CD drive that PowerVC creates during the deploy process.  There are a few known causes for this:

  •  if the VM that was captured to produce the image did not have device drivers installed for CD/DVD devices
  • A volume was imported into PowerVC and cloud-init was specified as the activation style. Note that this is the PowerVC default for Linux.  If "Activation engine / OVF" is not specified the image will be missing special metadata that helps provide the data to the activation engine.  One way to verify if this is the case is to re-deploy the image and once the VM is deployed check the contents of if CD drive.  If this is the root cause, the CD will have data but will be missing the ovf-env.xml file in the root directory of the drive.
  • You have an image that has incomplete or damaged metadata.  This can result in the ovf-env.xml not being placed in the CD drive of the VM.

1.2.4 No activation ran

If the /opt/ibm/ae/AP/noap file exists, the activation engine will not run during the boot of the virtual machine. If this file exists and there are no logs in the /opt/ibm/ae/AR directory, it is possible that the logs were removed from the captured virtual machine and the prerequisite capture steps were not taken.

You must complete the prerequisite steps on the virtual machine before it is captured. For details about the prerequisite steps, see the product documentation for PowerVC Express or PowerVC Standard.

In particular, you must run the following commands before capturing the virtual machine:

rm /opt/ibm/ae/AP/*

cp /opt/ibm/ae/AS/vmc-network-restore/resetenv /opt/ibm/ae/AP/ovf-env.xml

/opt/ibm/ae/AE.sh -R

If the prerequisite steps were not performed, you must complete the steps on the original virtual machine and re-capture it.

If the VM is running AIX another reason activations did not run during boot is because the activation engine was unable to find Java in the path during the boot sequence.  To fix this, ensure that the PATH environment variable in /etc/environment contains the path to either the Java 5 or Java 6 executables.

1.2.5 Activation takes a long time to run

If activation takes too long to run, the virtual optical can be removed before it completes.  This will result in an incomplete activation.

Issues that can cause activation and the system in general to run slowly include:

1. Specifying DNS IP addresses in the /etc/resolv.conf of your image or PowerVC network which are not pingable in the environment you are deploying into.  This will result in a generally slow system.  To address this situation it's recommended that you remove /etc/resolv.conf before capturing images that will be deployed into environments where this is a problem.  You can also set 'hosts=local,bind4' in /etc/netsvc.conf.

2. Specifying a gateway that is not pingable in the PowerVC network which you are deploying into.

3. Running AIX in a partition that has less than the operating system minimum recommended memory.

4. Using IBM Systems Director Common Agent when it is not needed.  If you are not using IBM Systems Director or any other software that needs the common agent it could be uninstalled.  During activation the activation engine unmanages the common agent and resets common agent unique keys.  This requires several restarts of the common agent which can take a significant amount of time in a memory constrained operating system.  Uninstall instructions can be found here: https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W3e8d1c956c32_416f_a604_4633cd375569/page/Uninstall%20IBM%20Systems%20Director%20on%20AIX

2. The partition (virtual machine) does not boot

There can be several reasons a virtual machine does not boot into its operating system. For details, read the following information.

2.1. SUSE Linux mounts disks by ID by default

A default installation of SUSE Linux will mount disks by ID. This is not supported while doing disk cloning as PowerVC does.

If the SUSE Linux image is deployed, the virtual machine does not start and the partition console shows a message like this:

Waiting for device /dev/disk/by-id/scsi-3600a0b800032e8000000c5be4f8d35bf-part3 to appear:

In this case, the operating system has not been changed from its ID mounting.

See the product documentation for PowerVC Express or PowerVC Standard for information about changing devices to be mounted by device name or UUID.

2.2. NPIV deploys and the operating system lacks the drivers

If you are using NPIV for your disk attachment, the operating system might fail to load because it is not configured to boot from NPIV.

For example, on SLES 11 SP3, the /etc/sysconfig/kernel file must contain ibmvfc in its INITRD_MODULES line, like this:

INITRD_MODULES="ibmvscsic ibmvfc"

On SLES 11 SP3, if you add this driver you must run mkinitrd and lilo and then re-capture the virtual machine to enable it to boot with NPIV.

Other Linux distributions might have similar requirements.

2.3. Operating system level is not supported on the virtual machine

Some operating system levels are not supported on certain levels of server hardware. For example, AIX 6100-01 will not run on Power 7 servers.

2.4 The deployed virtual machine does not have a bootable disk

It is possible to capture virtual machines that do not have bootable disk images by deselecting the boot disk when you capture the virtual machine. If such a virtual machine is deployed, it will not boot.

3. Enabling the activation engine fails or was not run.

The prerequisite steps must be taken on the virtual machine before it is captured. For details about the prerequisite steps, see the product documentation for PowerVC Express or PowerVC Standard.

In particular, you must run the following commands before capturing the virtual machine:

rm /opt/ibm/ae/AP/*

cp /opt/ibm/ae/AS/vmc-network-restore/resetenv /opt/ibm/ae/AP/ovf-env.xml

/opt/ibm/ae/AE.sh -R

If the prerequisite steps were not taken, you must complete the steps on the original virtual machine and re-capture it.

On AIX you must have Java in the PATH in root's /.profile file.  It must also be in the PATH in the system's /etc/environment file so activations run during deploy.

It is possible that the AE.sh -reset command was run. This is not the same as the AE.sh -R or AE.sh --reset commands. In this case, the /opt/ibm/ae/AR/ovf-env/ar file will look like this:

<?xml version="1.0" ?>

<ar:Activation status="Successful" xmlns:ar="http://www.ibm.com/xmlns/ovf/activation/ar/2">

        <VirtualSystem id="vs0" status="Successful" status-detail="No coordinator">

                <ProductActivation class="com.ibm.ovf.vmcontrol.adapter.networking">

                        <Execution kind="self" status="Skipped" status-detail="No properties"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.restore.network">

                        <Execution kind="self" status="Skipped" status-detail="No properties"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.system">

                        <Execution kind="self" status="Skipped" status-detail="No properties"/>

                        <Properties/>

                </ProductActivation>

                <ProductActivation class="com.ibm.ovf.vmcontrol.system.networking">

                        <Execution kind="self" status="Skipped" status-detail="No properties"/>

                        <Properties/>

                </ProductActivation>

        </VirtualSystem>

</ar:Activation>

It is possible that the AE.sh -R failed.

Add the -n parameter to the AE.sh -R command line to prevent it from shutting down the virtual machine. Then, check the output of the reset call. A successful AE.sh -R -n output looks like this on Linux:

[2012-09-18 15:17:52,361] INFO: Looking for platform initialization commands

[2012-09-18 15:17:52,361] INFO: OS: redhat Version: 5.8 Architecture: Power

[2012-09-18 15:17:52,371] INFO: No initialization commands found....continuing

[2012-09-18 15:17:52,371] INFO: CLI parameters are ' --reset -n'

[2012-09-18 15:17:52,371] INFO: AE base directory is /opt/ibm/ae/

[2012-09-18 15:17:52,373] INFO: Resetting system. AP file: None. Interactive: False

[2012-09-18 15:17:52,390] INFO: In reset

[2012-09-18 15:17:52,390] INFO: Resetting products

[2012-09-18 15:17:52,391] INFO: Start to reset com.ibm.ovf.vmcontrol.adapter.networking

mount: /dev/sda3 already mounted or /tmp/activation-engine-0gQtiI busy

hfs: unable to find HFS+ superblock

mount: you must specify the filesystem type

[2012-09-18 15:17:52,472] INFO: No reset program for com.ibm.ovf.vmcontrol.adapter.networking, failed

[2012-09-18 15:17:52,479] INFO: Start to reset com.ibm.ovf.vmcontrol.system.networking

[2012-09-18 15:17:52,500] INFO: Reset: about to execute path /opt/ibm/ae/AS/vmc-sys-net/reset.py

[2012-09-18 15:17:52,569] INFO: [com.ibm.ovf.vmcontrol.system.networking] reset Failed

[2012-09-18 15:17:52,570] INFO: [5561:com.ibm.ovf.vmcontrol.system.networking@1347995872.57] current results: ActivationReport(271652744) < status: Not Activated, virtual_systems:

        >

[2012-09-18 15:17:52,570] INFO: [5561@1347995872.57] OVERALL STATUS For 'vs0': Not Activated

[2012-09-18 15:17:52,572] INFO: Start to reset com.ibm.ovf.vmcontrol.restore.network

[2012-09-18 15:17:52,593] INFO: Reset: about to execute path /opt/ibm/ae/AS/vmc-network-restore/reset.py

[2012-09-18 15:17:52,670] INFO: [com.ibm.ovf.vmcontrol.restore.network] reset Not Activated

[2012-09-18 15:17:52,671] INFO: [5561:com.ibm.ovf.vmcontrol.restore.network@1347995872.67] current results: ActivationReport(271688088) < status: Not Connected to VSAE, virtual_systems:

        vsId: vs0, overall status: Not Activated

                class: com.ibm.ovf.vmcontrol.system.networking, instance: None, status: Not Activated>

[2012-09-18 15:17:52,672] INFO: [5561@1347995872.67] OVERALL STATUS For 'vs0': Not Activated

[2012-09-18 15:17:52,689] INFO: Resetting operating system

[2012-09-18 15:17:52,690] INFO: Cleaning AR and AP directories

A successful AE.sh -R -n output looks like this on AIX:

[2014-03-05 13:11:11,756] INFO: Looking for platform initialization commands

[2014-03-05 13:11:11,767] INFO: OS: AIX Version: 7.1

[2014-03-05 13:11:12,369] INFO: No initialization commands found....continuing

[2014-03-05 13:11:12,375] INFO: Base PA: /opt/ibm/ae/ovf-env-base.xml

[2014-03-05 13:11:12,378] INFO: VSAE Encryption Level:  Disabled

[2014-03-05 13:11:12,381] INFO: CLI parameters are '['AE/ae.py', '-R', '-n']'

[2014-03-05 13:11:12,382] INFO: AE base directory is /opt/ibm/ae/

[2014-03-05 13:11:12,411] INFO: Resetting system. AP file: None. Interactive: False

[2014-03-05 13:11:12,660] INFO: In reset

[2014-03-05 13:11:12,661] INFO: Resetting products

[2014-03-05 13:11:12,663] INFO: Start to reset com.ibm.ovf.vmcontrol.system

0821-515 ifconfig: error loading /usr/lib/drivers/if_eth: A file or directory in the path name does not exist.

[2014-03-05 13:11:13,035] INFO: Start to reset com.ibm.ovf.vmcontrol.restore.network

0821-515 ifconfig: error loading /usr/lib/drivers/if_eth: A file or directory in the path name does not exist.

[2014-03-05 13:11:16,763] INFO: Resetting the operating system

[2014-03-05 13:11:16,813] INFO: Cleaning AR and AP directories


 

Shut down the operating system manually by running the following command: shutdown now -h

The AE.sh -R command will fail with this output when Java is not in root's PATH:

821-515 ifconfig: error loading /usr/lib/drivers/if_eth: A file or directory in the path name does not exist.

  File "/opt/ibm/ae/AE/invoker.py", line 114, in invokescript

    p = subprocess.Popen(javaargs + [prog] + args, 4096, None, None, stdout, stderr, None, False, False, None, env, False, None, 0)

  File "/opt/ibm/ae/lib/jython/Lib/subprocess.py", line 755, in __init__

    self._execute_child(args, executable, preexec_fn, close_fds,

  File "/opt/ibm/ae/lib/jython/Lib/subprocess.py", line 1269, in _execute_child

    raise OSError(e.getMessage() or e)

Traceback (most recent call last):

  File "AE/ae.py", line 2079, in <module>

    aeng.main(argv)

  File "AE/ae.py", line 1981, in main

    self.reset(classname, None, False, instance, options.values.vs_id,shutdown)

  File "AE/ae.py", line 1981, in main

    self.reset(classname, None, False, instance, options.values.vs_id,shutdown)

  File "AE/ae.py", line 409, in reset

    self.__reset(service, inst, interactive, vid, shutdown)

  File "AE/ae.py", line 705, in _ae__reset

    self.__doreset(interactive, prod, vsys.id)

  File "AE/ae.py", line 959, in _ae__doreset

    self._do_single_reset(prod.classname, prod, None, params, vs_id)

  File "AE/ae.py", line 1264, in _do_single_reset

    rc = p.wait()

AttributeError: 'NoneType' object has no attribute 'wait'


 

If the AE.sh -R command fails with output like the following, run "rm /opt/ibm/ae/AR/*" and then re-run the activation engine enablement commands:

[2014-03-05 12:48:14,737] INFO: Start to reset com.ibm.ovf.vmcontrol.system

Traceback (most recent call last):

  File "AE/ae.py", line 2079, in <module>

    aeng.main(argv)

  File "AE/ae.py", line 1981, in main

    self.reset(classname, None, False, instance, options.values.vs_id,shutdown)

  File "AE/ae.py", line 1981, in main

    self.reset(classname, None, False, instance, options.values.vs_id,shutdown)

  File "AE/ae.py", line 409, in reset

    self.__reset(service, inst, interactive, vid, shutdown)

  File "AE/ae.py", line 705, in _ae__reset

    self.__doreset(interactive, prod, vsys.id)

  File "AE/ae.py", line 848, in _ae__doreset

    currentar = ar.getresult(ARFILE)

  File "/opt/ibm/ae/AE/parser/ar.py", line 61, in getresult

    root = minidom.parse(arfile).documentElement

  File "/opt/ibm/ae/lib/jython/Lib/xml/dom/minidom.py", line 1923, in parse

    return _do_pulldom_parse(pulldom.parse, (file,),

  File "/opt/ibm/ae/lib/jython/Lib/xml/dom/minidom.py", line 1908, in _do_pulldom_parse

    toktype, rootNode = events.getEvent()

  File "/opt/ibm/ae/lib/jython/Lib/xml/dom/pulldom.py", line 275, in _slurp

    self.parser.parse(self.stream)

  File "/opt/ibm/ae/lib/jython/Lib/xml/sax/drivers2/drv_javasax.py", line 141, in parse

    self._parser.parse(JyInputSourceWrapper(source))

  File "/opt/ibm/ae/lib/jython/Lib/xml/sax/drivers2/drv_javasax.py", line 58, in fatalError

    self._err_handler.fatalError(_wrap_sax_exception(exc))

  File "/opt/ibm/ae/lib/jython/Lib/xml/sax/handler.py", line 38, in fatalError

    raise exception

xml.sax._exceptions.SAXParseException: <unknown>:-1:-1: Premature end of file.

4. Debugging user scripts

The activation engine shipped with PowerVC 1.2.2 and later can run user scripts on AIX.  This section contains debugging information for issues with user scripts.

4.1 Known issues

The activation engine expects to mount the CD drive to a temp directory.  If the operating is auto-mounting the CD drive using an /etc/filesystems entry or the CD daemon, the activation engine may configure networking but not run the user script.  If the /opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.out log contains the message 'The CD mount directroy was not found.' the activation engine was not able to find the CD drive mounted in the expected location.

4.2 Logs, script location on CD drive

The user provided script sent to the virtual machine as the /openstack/latest/user_data file on the CD drive.  If the script you provide on deploy is not running or there are issues with the script, immediately after the deploy, mount the CD drive and check the user_data file.  You may wan to copy it to a different directory, set execute permissions and attempt running it.  For example:

/usr/sbin/mount -rv cdrfs /dev/cd0 /cdrom

cat /cdrom/openstack/latest/user_data

 

When the script runs its standard out and standard error is written to the /opt/ibm/ae/AR/com.ibm.ovf.vmcontrol.system.networking.out log file.