ksysvmmgr command

Purpose

The ksysvmmgr command provides a consistent interface for the VM agent to manage the virtual machine monitor (VMM) and the applications that are running in the virtual machine.

Syntax

The ksysvmmgr command uses the following basic format:

ksysvmmgr [flags] ACTION CLASS [NAME] [ATTRIBUTES...]

Description

The VM agent consists of the following subsystems:

VM monitor: This subsystem tracks the health of the VM by communicating periodically with the host monitor in the VIOS.
Application monitoring framework: This subsystem is an application monitoring framework that registers, starts, stops, and monitors the applications.

You can access the VM monitor and the application monitoring framework only through the command line by using the ksysvmmgr command. Graphical user interface is not available for such operations.

Log file

All ksysvmmgr command operations are logged in the /var/ksys/log/ksysvmmgr.log file, including the name of the command that was executed, the start and stop time of the command, and the user name of the user who initiated the command. You can use the -l flag to change the amount of information that is written to the log files. The ksysvmmgr command sends user messages that are transformed by using the catalog messages.

Flags

You can use the following flags with the ksysvmmgr command:

ACTION

Describes the action to be performed.

The ACTION flags are not case-sensitive. All ACTION flags provide synonyms, and each alias and synonym have shorter alias. For example, remove is an alias for delete, and remove can be abbreviated with rm. Aliases are provided for convenience from the command line and must not be used in scripts. The following ACTION flags are available:

help (alias: h*)
add (alias: a*, register, reg*)
query (aliases: q*, list, get, li*, g*)
modify (aliases: mod*, change, set, ch* se*)
delete (aliases: unregister, remove, de*, unr* re*, rm)
sync (alias: syn*)
start (alias: on, enable, star*, en*)
stop (alias: off, disable, sto*, di*)
backup (alias: bac*)
restore (alias: rest*)
suspend (alias: unmonitor, sus*, unm*)
resume (alias: monitor, resu*, mon*)
snap (alias: snap, sna*)
refresh (alias: refresh, ref*)

CLASS

Specifies the type of object on which the ACTION is performed. The following CLASS objects are supported:

vmm (alias: v*): The vmm CLASS is used by default. If you do not specify any CLASS, the action is performed on the VM monitor by default.
app (alias: a*): The app CLASS is used to perform ACTION on applications. You must specify a NAME attribute to apply the ACTION on a specific application.
dependency (alias:dep*): The dependency CLASS establishes a dependency relationship between the applications. You must specify the list of applications and dependency type to apply a dependency between applications.
process (alias:none): The process class is used to perform ACTION on processes. You must specify a NAME attribute to apply the ACTION on a specific process.

NAME

Specifies the particular object, of type CLASS, on which the ACTION must be performed. The NAME flags are case-sensitive. You can use this flag only for the app CLASS.

ATTRIBUTES

Specifies an optional flag that has attribute pairs and value pairs that are specific to the ACTION and CLASS combination. Use these pairs to specify configuration settings or to run particular operations.

Both ATTRIBUTES and VALUE flags are case-sensitive. You cannot use the asterisk (*) character in the ACTION and CLASS names.

-a

Displays only the specified attributes. This flag is valid only with the query ACTION. Attribute names are not case-sensitive.

-f

Overrides any interactive prompts, forcing the current operation to be attempted, if allowed.

-h/-?

Displays help information.

-l 0|1|2|3

Activates the following trace log values for serviceability:

0: Updates the log file when an error is detected. This level of trace logging is the default value.
1: Logs warning messages also.
2: Logs basic information messages also that can be used for demonstrations.
3: Performs high tracing by logging the details about the routine function and the utility function. Traces the entry and exits for various functions.

All trace data is written into the vmmgr.log file. This flag is used for troubleshooting problems.

-s

Synchronizes the VM monitor daemon immediately by sending a notification to the daemon. The VM monitor daemon reloads the XML configuration when it receives this notification.

This flag is valid only with the add, modify, delete, resume, and suspend ACTIONS. By default, no notification is sent. To send a notification, use the ksysvmmgr sync command.

Attributes

The ksysvmmgr command configures the following classes and attributes:

vmm

When you start the VM monitor, the VM monitor daemon sends heartbeats to the host monitor, when requested by the host monitor, so that the KSYS subsystem can monitor the virtual machines. The VM monitor can have the following attributes:

version

Specifies the version of XML. This mandatory attribute is set to 1.0 for the current version of VM monitor and cannot be modified.

log

Specifies the log level of the VM monitor daemon. This attribute can have the following values:

0: Only errors are logged. It is the default value.
1: Warnings are also logged.
2: Informational messages are also logged.
3: Details of the operation are logged. This information is used for debugging.

period

Specifies the time duration in seconds between two consecutive occurrences of checks that are performed by the Application Management Engine (AME). By default, the value of this attribute is 1 second. The value of this attribute must be in the range 0 - 6. For best monitoring performance, do not modify the default value.

Application (app)

The application class contains the following mandatory attributes:

monitor_script

A mandatory script that is used by the VM agent to verify application health. This script is run regularly (based on the monitor_period attribute value) and the result is checked for the following values:

0: Application is working correctly.
Any value other than 0: Application is not working correctly or has failed.

After several successive failures (based on the monitor_failure_threshold attribute value), the application is declared as failed. Based on the specified policies, the KSYS subsystem determines whether to restart the virtual machine.

stop_script

A mandatory script that is used by the VM agent to stop the application if the application must be restarted. The application can be restarted by successively calling the stop_script and start_script scripts.

start_script

A mandatory script that is used by the VM agent to start the application if the application must be restarted.

Exception: These scripts are not mandatory for the following application types: ORACLE, DB2, SAPHANA, and SAPNW_*.

The application class contains the following optional attributes:

monitored

Specifies whether the application is monitored by the KSYS subsystem. This attribute can have the following values:

1 (default): The application monitoring is active.
0: The application monitoring is suspended.

monitor_period

Specifies the time in seconds after which the application monitoring must occur. The default value of 30 seconds specifies that the monitor_script script is run by the VM agent every 30 seconds.

monitor_timeout

Specifies the waiting time in seconds to receive a response from the monitor_script script. The default value is 10 seconds, which means that the VM monitor waits for 10 seconds to receive a response from the monitor_script script after which the script is considered as failed.

monitor_failure_threshold

Specifies the number of successive failures of the monitor_script script that is necessary before the VM monitor restarts the application. A restart operation is performed by successively calling the stop_script and start_script scripts.

stop_stabilization_time

Specifies the waiting time in seconds to receive a response from the stop_script script. The default value is 25 seconds, which means that the VM monitor waits for 25 seconds to receive a response from the stop_script script after which the script is considered as failed.

stop_max_failures

Specifies the number of successive failures of the stop_script script that is necessary before the VM monitor considers that it cannot stop the application. The default value is set to 3.

start_stabilization_time

Specifies the waiting time in seconds to receive a response from the start_script script. The default value is 25 seconds, which means the VM monitor waits for 25 seconds to receive a response from the start_script script after which the script is considered as failed.

start_max_failures

Specifies the number of successive failures of the start_script script that is necessary before the VM monitor considers that it cannot start the application. The default value is set to 3.

max_restart

Specifies the number of cycles of successive VM restart operations that result in a monitoring failure before the daemon pronounces that restarting at VM level is insufficient. By default, this attribute is set to 3.

Note: The value of the max_restart attribute can be set to 1 or any other positive value, to set this value to 0, run the modify command with the -f option.

ksysvmmgr -f modify app <NAME> max_restart=0

status

Specifies the dynamic status of application that is returned by Application Management Engine (AME). This attribute cannot be modified.

The color codes signifies the dynamic status of an application.

Table 1. Color code for dynamic application status
Color code	Application status
Red	Permanent application failure
Orange	Initial application state
Yellow	Application is in normal state but failed more than two times within last 24 hours
Green	Application is in normal state
Gray	Application is not monitored by the VMM daemon
Blue	Application is in other intermittent states. For example, starting, stopping, or failing state.

version

Specifies the application version. This attribute does not have a default value.

critical

Marks the application as critical. The valid values are Yes and No (default). If you mark an application as critical, failure of the application may lead the VM to be rebooted or relocated by the KSYS subsystem.

type

Specifies the type of application. By default, the type attribute value is CUSTOM that indicates general applications. Other supported values are ORACLE, DB2, POSTGRES , SAPHANA, and SAPNW_*. This attribute is case-sensitive and you must use uppercase characters. For these types of applications, if you do not specify start, stop, and monitor scripts, the internal scripts of the VM monitor are used.

instancename

Specifies the instance name for applications. This attribute is applicable only for agent applications, which are internally supported by the VM agent. The supported agent applications are: oracle, DB2, SAPHANA and POSTGRES. For example,

If the application type is ORACLE, the instancename attribute must be specified with the Oracle user name.
If the application type is DB2, the instancename attribute must be specified with the DB2® instance owner.
If the application type is SAPHANA, the instancename attribute must be specified with the SAPHANA system id.
If the application type is SAPNW_*, the instancename attribute must be specified with the SAPNW system id.
If the application type is POSTGRES, the instancename attribute must be specified with the POSTGRES instance id.

database

Specifies the database that the applications must use. This attribute is applicable only for agent applications, which are internally supported by the VM agent. The supported agent applications are: oracle, DB2, SAPHANA and POSTGRES.For example,

If the application type is ORACLE, the database attribute must be specified with the Oracle system identifier (SID).
If the application type is DB2, the database attribute is not required.
If the application type is SAPHANA, the database attribute must be specified with the SAP HANA database.
If the application type is SAPNW_*, the database attribute must be specified with the SAPNW database.
If the application type is POSTGRES, the database attribute can be specified with the database name. If the database name is not specified, the script monitors all database of the POSTGRES instance.

appstarttype

Specifies the method in which the applications must be started. This attribute can have the following values:

VMM: Specifies that the VM agent must start and monitor the applications.
OS: Specifies that the application must be started by the operating system or by a user.
KSYS: Specifies that the application must be started or stopped by the KSYS subsystem. After the application is started by the KSYS subsystem, and eventually the application crashes, the VMM must restart the application.

From the ksysvmmgr command-line-interface (CLI), you can modify the appstarttype attribute of an application.

Further, if a VM daemon reboots, the VMM daemon starts all the VMM controlled applications, but the VMM daemon cannot start the KSYS controlled applications. Instead, the VMM daemon sends the status of the application to the KSYS subsystem. The KSYS subsystem determines whether to start or stop the KSYS controlled applications. The KSYS subsystem has privilege to modify the value of the appstarttype attribute of an application from KSYS to VMM, or vice versa.

The default value of the appstarttype attribute of an application is VMM.

Note: If the appstarttype attribute is modified from KSYS to VMM, then you must manually delete all the related application dependencies in the KSYS subsystem.

groupname

Specifies the groupname to which the application belongs. The default value of the groupname attribute is NULL. After a user creates a new group, the groupname of each application in the group is updated.

configfile

This file contains the application configurations settings for the supported agent applications . This attribute is used only by SAP HANA, SAP NetWeaver, and POSTGRES agent applications. This attribute is blank for other agent applications.

Application dependency (dependency)

The dependency class contains the following mandatory attributes:

dependency_type

Specifies the type of dependency between applications. This attribute can have the following values:

start_sequence: Specifies the order in which the applications must be started as mentioned in the dependency_list attribute. The dependency_list attribute must have more than one application for this dependency type.
stop_sequence: Specifies the order in which the applications must be stopped as mentioned in the dependency_list attribute. The dependency_list attribute must have more than one application for this dependency type.
parent_child: Specifies the parent-child relationship of the two specified applications in which one application is parent and the other is child. The parent application must start first and then the child application starts. You must stop the child application first and then stop the parent application. If the parent application fails, the child application also stops automatically. If the parent application recovers and starts, the child application started automatically.

dependency_list

Specifies the list of applications that have a dependency between them.

The dependency class also contains the following optional attributes:

strict: Specifies whether to continue the script or command if the dependency policy cannot be followed. If the strict attribute is set to Yes, the next application is not started until the previous application starts and is in the normal state. If the strict attribute is set to No, the next application is started immediately after the first application is started irrespective of the state of the first application. This attribute is applicable only for the start_sequence dependency.

Process

The process class can have the following attributes:

start_script: This script is used by the process monitor to restart a process in a virtual machine. The restart operation is performed by successively calling the stop_script and the start_script.
stop_script: This script is used by the process monitor to stop an application.
monitor_period: Duration between two successive process monitor operations. This duration is displayed in seconds. The default value is 30 seconds.
stop_stabilization_time: The period of time before which no answer from the stop_script script is considered as timeout. The default value is 25 seconds.
stop_max_failures: The number of successive failures of the stop_script script after which it is considered that the process cannot be stopped. The default value is 3 successive failures.
start_stabilization_time: The duration of time before which no answer from the start_script script is considered as timeout. This duration is displayed in seconds. The default value is 25 seconds.
start_max_failures: The number of successive failures of the start_script script after which it is considered that the process cannot be started. The default value is 3 successive failures.
max_restart: The maximum number of restart cycles, after which the process monitor is considered as failed. The default value is 3.

Comprehensive list of ksysvmmgr commands

Use the following information to find the syntax for all possible ksysvmmgr operations:

VM monitor operations

To display help information about the vmm class, run the following command:
```
ksysvmmgr -h vmm
```

To start the VM monitor daemon, run the following command:

ksysvmmgr start [vmm] [<ATTR#1>=<VALUE#1>][,<ATTR#n>=<VALUE#n>]

To stop the VM monitor daemon, run the following command:
```
ksysvmmgr stop [vmm]
```
To query the details about the VM monitor daemon, run the following command:
```
ksysvmmgr [-l {0|1|2|3}] [-a <ATTR#1>[,<ATTR#2>,...]] query [vmm]
```
To modify the VM monitor daemon attributes, run the following command:
```
ksysvmmgr [-s] modify [vmm] <ATTR#1>=<VALUE#1>[,<ATTR#n>=<VALUE#n>]
```
To back up the VM monitor configuration settings, run the following command:
```
ksysvmmgr backup vmm [<ATTR#1>=<VALUE#1>]
```
To restore the VM monitor configuration settings, run the following command:
```
ksysvmmgr restore vmm <ATTR#1>=<VALUE#1>
```
To notify the VM monitor daemon to synchronize with the contents of the XML configuration file, run the following command:
```
ksysvmmgr sync [vmm] 
```
To compress all log files, run the following command:
```
ksysvmmgr snap [vmm]
```
This command creates a *.pax.gz file in the /var/ksys/log/snap directory. To read the contents of the file, you can unzip the file by using the following commands:
```
unzip *.pax.gz
pax -r -f *.pax
```
To view the status of communication from VM monitor to host monitor, run the following command:
```
ksysvmmgr query vmm
```
This command displays the VM monitor attributes along with the host monitor details such as UUID of the host monitor and the Heartbeat status of the host monitor.

Application operations

To display help information about the app class, run the following command:
```
ksysvmmgr -h app
```

To add an application that must be monitored, run the following command:

ksysvmmgr [-s] [-l {0|1|2|3}] add app <NAME> <ATTR#1>=<VALUE#1>[,<ATTR#n>=<VALUE#n>]

To add a critical application that must be monitored, run the following command:
```
ksysvmmgr [-s] [-l {0|1|2|3}] add app <NAME> critical=yes
```

To modify the attributes of a registered application, run the following command:

ksysvmmgr [-s] [-l {0|1|2|3}] modify app <NAME> <ATTR#1>=<VALUE#1>[,<ATTR#n>=<VALUE#n>]

To query the details about a registered application, run the following command:
```
ksysvmmgr [-l {0|1|2|3}] [-a <ATTR#1>[,<ATTR#2>,...]] query app <NAME>
```
To delete specific or all applications from the VM agent configuration settings, run the following command:
```
ksysvmmgr [-s] [-l {0|1|2|3}] delete app <NAME>|<NAME1,NAME2,NAME3...>|<ALL>
```
To suspend the monitoring of an application or all applications, run the following command:
```
ksysvmmgr [-s] [-l {0|1|2|3}] suspend app [<NAME>]
```
To resume the monitoring of an application or all applications, run the following command:
```
ksysvmmgr [-s] [-l {0|1|2|3}] resume app [<NAME>]
```
To start an application, run the following command:
```
ksysvmmgr start app <NAME>
```
To stop an application, run the following command:
```
ksysvmmgr stop app <NAME>
```
To group the applications, run the following command:
```
ksysvmmgr [-s] add group <NAME> <ATTR#1>=<VALUE#1>
```
To modify the applist in a group, run the following command:
```
ksysvmmgr [-s] modify group <NAME> applist=<VALUE#1>
```
To ungroup (delete a group), run the following command:
```
ksysvmmgr [-s] delete group <NAME>
```
To refresh the configuration of VM agent applications, run the following command:
```
ksysvmmgr refresh app <NAME>
```

Dependency operations

To display help information about the dependency class, run the following command:
```
ksysvmmgr -h dep
```

To add a dependency relationship between applications, run the following command:

ksysvmmgr [-s] [-l {0|1|2|3}] add dependency <ATTR#1>=<VALUE#1>[,<ATTR#n>=<VALUE#n>]

To query the details about an existing dependency between applications, run the following command:
```
ksysvmmgr [-l {0|1|2|3}] [-a <ATTR#1>[,<ATTR#2>,...]] query dep<DEPUUID>
```

To modify a specific dependency relationship between applications, run the following command:

ksysvmmgr [-s] [-l {0|1|2|3}] modify dependency <NAME> <ATTR#1>=<VALUE#1>[,<ATTR#n>=<VALUE#n>]

To delete a specific dependency relationship, run the following command:
```
ksysvmmgr [-s] [-l {0|1|2|3}] delete dep DEPUUID
```

Process operations

To add a process for monitoring, run the following command:

ksysvmmgr [-s] add process process_name=<NAME> <ATTR#1>=<VALUE#1> [,<ATTR#n>=<VALUE#n>]

To query a process, run the following command:
```
ksysvmmgr query process [process_name=<NAME>]
```

To modify a process attribute, run the following command:

Ksysvmmgr [-s] modify process process_name=<NAME> <ATTR#1>=<VALUE#1> [,<ATTR#n>=<VALUE#n>]

To suspend a process, run the following command:
```
ksysvmmgr suspend process process_name=<NAME>
```
To resume monitoring a process, run the following command:
```
ksysvmmgr resume process process_name=<NAME>
```
To delete a process (which is monitored), run the following command:
```
ksysvmmgr [-s] delete process process_name=<NAME>
```

An example scenario

The following examples show a scenario in which you start the VM monitor daemon, configure the VM agent by adding applications and dependencies:

To display the help information about the vmm class, run one of the following command:
```
1. ksysvmmgr -h vmm
2. ksysvmmgr help vmm
```
Start the VM monitor daemon by running the following command:
```
ksysvmmgr start
```
To stop the VM monitor daemon, run the following command:
```
ksysvmmgr stop
```
Modify the VM monitor daemon attributes by running the following command:
```
ksysvmmgr modify log=2
```
Back up the VM monitor configuration settings by running the following command:
```
ksysvmmgr backup vmm filepath=/var/ksys/config
```
Note: Create the backup HAMonitoring 2019-08-30 00:11:08.xml in the /var/ksys/config path.
To restore the saved configuration settings, run the following command:
```
ksysvmmgr restore vmm
   filepath=/var/ksys/config/HAMonitoring_2019-08-30_00:11:08.xml
```
Note: Stores the current configuration settings to the /var/ksys/config/snapshot/HAMonitoring_backup.xml file and the current configuration file is updated with the specified configuration file path.
To collect the snapshot, run the following command:
```
ksysvmmgr snap
```
Note: Install the pax rpm package before using the ksysvmmgr snap command on your virtual machine.
To notify the VM monitor daemon to synchronize with the contents of the XML configuration file, run the following command:
```
ksysvmmgr sync vmm
```
To query vmm parameters, run the following command:
```
ksysvmmgr query vmm
```

To add an application (say app1), run the following command:

ksysvmmgr -s add app app1 monitor_script=/tmp/monitor1.sh  
        start_script=/tmp/start1.sh stop_script=/tmp/stop1.sh

To delete an application (say app1), run the following command:
```
ksysvmmgr -s delete app app1
```
To suspend an application (say app1), run the following command:
```
ksysvmmgr suspend app app1
```
To resume an application (say app1), run the following command:
```
ksysvmmgr resume app app1
```
To start an application (say app1), run the following command:
```
ksysvmmgr start app app1
```
To stop an application (say app1), run the following command:
```
ksysvmmgr stop app app1
```
To query all applications, run the following command:
```
ksysvmmgr query app
```
To query application (say app1), run the following command:
```
ksysvmmgr query app app1
```
To get the status of all applications, run the following command:
```
ksysvmmgr status app
```
To get the status of an application (say app1), run the following command:
```
ksysvmmgr status app app1
```
To group the applications, run the following command:
```
ksysvmmgr -s add group grp1 applist=app1,app2,app3
```
To modify the application list in the group, run the following command:
```
ksysvmmgr -s modify group grp1 applist=app2,app5
```
To ungroup (delete the group), run the following command:
```
ksysvmmgr -s delete group grp1
```
To display help information about the dependency class, run one of the following command:
```
ksysvmmgr -h dep
ksysvmmgr help dep
```
Notify the VM monitor daemon to synchronize with the contents of the XML configuration file, run the following command:
```
ksysvmmgr sync vmm
```

Add an application app1 by running the following command:

ksysvmmgr add app app1 monitor_script=/tmp/monitor1.sh 
     start_script=/tmp/start1.sh stop_script=/tmp/stop1.sh

Modify the application attributes by running the following commands:

 ksysvmmgr modify app app1 new_name=newApp critical=yes
ksysvmmgr modify app app1 monitor_script=/apps/new_monitor_script
ksysvmmgr modify app app1 monitor_period=10

Create a start_sequence dependency with 3 applications app1, app2, app3 in the dependency list by running the following command:
```
ksysvmmgr -s add dependency dependency_list=app1,app2,app3 
dependency_type=start_sequence
```
To create a stop_sequence dependency with 3 applications app4, app5, app6 in the dependency list, run the following command:
```
ksysvmmgr -s add dependency dependency_list=app4,app5,app6 dependency_type=stop_sequence 
```
Create a parent_child dependency with applications app1 and app2 in the dependency list by running the following command:
```
ksysvmmgr -s add dependency dependency_list=app1,app2 dependency_type=parent_child
```
Display the dependency UUIDs by running the following command:
```
ksysvmmgr query dependency
```

To modify the details of the dependency that has UUID 1531835289870752764, run the following command:

ksysvmmgr -s modify dependency 1531835289870752764 dependency_list=app3,app1,app2 dependency_type=start_sequence

Display the details of the dependency that has UUID 1531835289870752764 by running the following command:
```
ksysvmmgr query dep 1531835289870752764
```
To delete the dependency that has UUID 1531835289870752764, run the following command:
```
ksysvmmgr -s delete dep 1531835289870752764
```
To display help information about the process class, run one of the following commands:
```
ksysvmmgr -h process
ksysvmmgr help process
```

To add a process to monitor, run the following command:

ksysvmmgr -s add process process_name=proc1 start_script=/tmp/startproc1.sh stop_script=/tmp/stopproc1.sh

To query process proc1, run the following command:
```
ksysvmmgr query process process_name=proc1
```

To modify process attributes, run the following command:

ksysvmmgr -s modify process process_name=proc1 start_script=/tmp/new_startproc1.sh

To suspend a process, run the following command:
```
ksysvmmgr suspend process process_name=proc1
```
To resume a process, run the following command:
```
ksysvmmgr resume process process_name=proc1
```
To delete a process, run the following command:
```
ksysvmmgr -s delete process process_name=proc1
```
To refresh application APP1:
```
ksysvmmgr refresh app APP1
```