ksysmgr command

Purpose

The ksysmgr command provides a consistent interface to configure the controller system (KSYS) and to perform IBM® VM Recovery Manager DR for Power Systems operations by using a terminal or script.

Syntax

ksysmgr [-v] [-f] [-l {low|max}] [-i] 
[-a {<ATTR#1>,<ATTR#2>,...}] <ACTION> <CLASS> [<NAME>]
[-h | <ATTR#1>=<VALUE#1> <ATTR#2>=<VALUE#2> ...]

ksysmgr [-v] [-f] [-l {low|max}] 
[-a {<ATTR#1>,<ATTR#2>,...}] <ACTION> <CLASS> [<NAME>]
<ATTR#1>=<VALUE#1> <ATTR#2>=<VALUE#2> ...]

         ACTION={add|modify|delete|query|manage|unmanage|...}\n\
         CLASS={ksyscluster|site|hmc|host|...}\n\

ksysmgr {-h|-?} [-v] [<ACTION> [<CLASS>]]

ksysmgr [-v] help

The basic format for using the ksysmgr command is as follows:

ksysmgr ACTION CLASS [NAME] [ATTRIBUTES...]

Notes:

You must have root authority to run the ksysmgr command.
Help information is available for the ksysmgr command from the command line. For example, when you run the ksysmgr command without any flags or parameters a list of the available ACTIONs is displayed.
If you enter ksysmgr <ACTION> in the command line without specifying any CLASS, the command results in a list of all the available CLASSes for the specified ACTION.
Entering ksysmgr <ACTION> <CLASS> without specifying any NAME or ATTRIBUTES parameters is different because some ACTION and CLASS combinations do not require any additional parameters. To display help information in this scenario, you can view the help information by appending the -h flag to the ksysmgr <ACTION> <CLASS> command.
You cannot display help information from the command line for each of the ksysmgr command's individual ATTRIBUTES.

Description

All ksysmgr command operations are logged in the /var/ksys/log/ksysmgr.oplog file, which includes the name of the command that was executed, start time, process ID for the ksysmgr operation, the command with arguments, and overall return code.

Flags

ACTION

Describes the action to be performed.

Note: The ACTION flags are not case-sensitive. All ACTION flags provide a shorter alias. For example, rm is an alias for delete. Aliases are provided for convenience from the command line and must not be used in scripts.

The following ACTION flags are available:

Note: The asterisk (*) in the aliases signify wildcard characters. For example, for the modify ACTION, the alias value is mod*. If you type modd, the command still works.

query (alias: q*, ls, get, sh*)
add (alias: ad*, cr*, make, mk)
delete (alias: de*, remov*, rm, er*)
modify (alias: mod*, ch*, set)
verify (alias: ver*, validate)
sync (alias: syn*, pr*)
restore (alias: resto*)
manage (alias: man*, mg)
unmanage (alias: unman*, umg)
discover (alias: di*)
help (alias: hel*, ?)
move (alias: mov*, mv, sw*)
recover (alias: rec*)
pair (alias: map)
cleanup (alias: clean*)
resync (alias: resy*)
report (alias: rep*)
refresh (alias: ref*)
start
stop
trace
reset

CLASS

Specifies the type of object on which the ACTION is performed. The CLASS flags are not case-sensitive.

The following CLASS objects are supported:

ksyscluster (alias: clu*, ksyscl*)
site (alias: sit*)
hmc (alias: hmcs, hmces)
host (alias: serv*, mach*, cec*)
host_group (alias: hg, host_g*)
vios (alias: vios*)
vm (alias: lp*, vm*)
disk_pair (alias: dp, disk_p*)
disk_group (alias: dg, disk_g*)
storage_agent (alias: storage*, sta)
version (alias: vers*)
notify (alias: rn, remote_not*, noti*)
snapshot (alias: snap*)
script (alias: scr*)
workgroup (alias: workg*, work_g*, wg)
disk
event (alias: ev*)
system (alias: sys*)
log (alias: logs)
diskview (alias: diskv*)
inventory (alias: inv*)
storage_clone

NAME

Specifies the particular object, of type CLASS, on which the ACTION must be performed. The NAME flags are case-sensitive.

ATTR=VALUE

Specifies an optional flag that has attribute pairs and value pairs that are specific to the ACTION and CLASS combination. Use these pairs to specify configuration settings or to run particular operations. Both ATTR and VALUE flags are case-sensitive.

-a {<ATTR#1>,<ATTR#2>,...}

Displays only the specified attributes. This flag must be used with the query ACTION flag. For example:

ksysmgr -a name,sitetype query site

-f

Overrides any interactive prompts and forces the current operation to be run.

-h

Displays help information.

-i

Indicates that the ksysmgr command must prompt for your confirmation for important operations such as move operation across sites.

-l low|max

Activates the following trace logging values for troubleshooting purposes:

low (default): Logs basic information for every ksysmgr operation.
max: Performs high tracing operations such as adding the routine function and the utility function. Adds transaction ID to the function's entry messages.

Note: All trace data is written into the ksysmgr.log file. This flag is ideal for troubleshooting problems.

-v

Displays maximum verbosity in the output.

Exit status

RC_UNKNOWN (Exit value = -1): Result is not known. This value is used as an initializer.
RC_SUCCESS (Exit value = 0): No errors are detected. The operation is successful.
RC_ERROR (Exit value = 1): An error occurred.
RC_NOT_FOUND (Exit value = 2): The specified resource does not exist or cannot be found.
RC_MISSING_INPUT (Exit value = 3): Required input is missing.
RC_INCORRECT_INPUT (Exit value = 4): Detected input is incorrect.

Examples

To get help information about the ksyscluster class, enter the following command:

ksysmgr help ksyscluster

An output that is similar to the following example is displayed:

Available actions for ksyscluster:
         add
         delete
         modify
         query
         sync
         verify

Cluster configuration examples

To add a KSYS cluster:

ksysmgr add ksyscluster
      [<ksysclustername>]
      [type=<HA|DR|HADR|HADRHA>]
      ksysnodes=<ksysnode1[,ksysnode2,...]>
      [sync=<yes|no>]
    add => ad*, cr*, make, mk
    ksyscluster => ksysclu*, clu*

Note: By default, the value of type variable is DR cluster type. To configure the cluster type to HA, enter the following command:

ksysmgr add ksyscluster clusterName 
         ksysnodes=node1 type=HA

For information about high availability features, see IBM VM Recovery Manager HA for Power Systems documentation.

To query the KSYS cluster:
```
ksysmgr query cluster
```
To remove a KSYS cluster:
```
ksysmgr delete ksyscluster clustername
```
When you delete a KSYS cluster, the ksysmgr command prompts for your confirmation. The ksysmgr command also recommends to create a snapshot at this point. You can override these prompts by using the -f flag. For example,
```
ksysmgr -f delete ksyscluster SampleClusterName
```
Note: If the remove cluster command fails, you must use the force option with the remove cluster command. After using the force option with the remove cluster command, ensure that:
- The shared storage pool (SSP) clusters are removed.
- The adapters with VLAN ID 101 and 102 on the Virtual I/O Server are removed.
- The host monitor daemon on the Virtual I/O Server is inactive.

To modify a KSYS cluster:

ksysmgr modify ksyscluster <ksysclustername> <add|remove>
      [ksysnodes=<ksysnode1[,ksysnode2,...]>
    ksyscluster => ksysclu*, clu*

To check the version of the KSYS software:
```
ksysmgr query version
```

Site configuration examples

To create a site in the KSYS subsystem:
```
ksysmgr add site sitename sitetype=home|backup
```
No output is displayed. The command returns with the exit value of 0.
To query the details about all existing sites:
```
ksysmgr query site
```
An output that is similar to the following example is displayed:
```
Name:               SiteA
Sitetype:           HOME
Active Host_groups: SampleHG
Active Workgroups:  None

Name:               SiteB
Sitetype:           BACKUP
Active Host_groups: None
Active Workgroups:  None
```
Note: Active means which site the VMs in host_group/workgroup are running, HG/WG are only active on one site at a time
By default, the replication type of the site is async.

To query the details about a specific site:

ksysmgr -v query site sitename

An output that is similar to the following example is displayed:

Name:               SiteA
Sitetype:           HOME
Active Host_groups: None
Active Workgroups:  None

System-level Attributes
Memory_capacity:    Priority Based Settings
                    high:100
                    medium:100
                    low:100
CPU_capacity:       Priority Based Settings
                    high:100
                    medium:100
                    low:100
skip_resource_check:No
skip_power_on:      No
sriov_override:     No
vswitchmap:         Not currently set
drvswitchmap:       Not currently set
vlanmap:            Not currently set
drvlanmap:          Not currently set

To discover a site:
```
ksysmgr discover site sitename
```
The KSYS subsystem discovers all the hosts and virtual machines from all the host groups across both the sites. The discovery operation might take a few minutes to a few hours to complete depending on the scale of the environment. By using the -t flag with the discover, verify, or move command, you can get timestamps for every step of progress.

To modify the configuration information of a site:

ksysmgr modify site <sitename> | file=<filepath>
[memory_capacity=<(1-1000) | minimum | current_desired | none | default>
priority=<low | medium | high>]
[cpu_capacity=<(1-1000) | minimum | current_desired | none | default>
priority=<low | medium | high>]
[skip_resource_check=<yes|no>]
[skip_power_on=<yes|no>]
[sriov_override=<yes|no>]
[network=<vlanmap | vswitchmap> backupsite=siteB
sitename=<#[,#,...] | all> siteB=<#[,#,...] | all> [dr_test=<yes|no>]
[policy=delete]]

To change the replication type of a site from the default async value to the sync value:
```
ksysmgr modify system replication_type=sync sites=Site1,Site2
```
To delete a site:
```
ksysmgr delete site <sitename>
```

HMC port number

To query a HMC port number:

ksysmgr q system

An output that is similar to the following example is displayed:

auto_discovery_time:     00:00 hours
notification_level:      low
dup_event_processing:    yes
hmc_port_number:         443

To get help message for HMC port number:
```
ksysmgr modify system -h
```
A help message that is similar to the following example is displayed:
```
[hmc_port_number=<val>] 
  val - 443 or 12443
```
To modify the HMC port number:
```
ksysmgr modify system hmc_port_number=12443
```
Note: By default the HMC port number value is 443. You can modify the HMC port number value to 12443.

HMC configuration examples

To add an HMC to the KSYS configuration:

ksysmgr add hmc hmcname
      login=username
      [ password=password ]
      hostname|ip=hostname|ip
      site=site_name

The following output is displayed:

HMC HMCNAME added successfully

You can also run the command without specifying the password in the command line. In this case, you can enter the password as hidden characters when the command prompts for the password. For example,

ksysmgr add hmc PrimaryHmcName login=hscroot ip=86.xx.xx.xx site=SiteA
Enter Password for hmc: ***********
Re-Enter Password: ************

To query the details about all HMCs:

ksysmgr query hmc

An output that is similar to the following example is displayed:

Name:            Site2_HMC1
Site:            Site2
Ip:              9.xx.yy.zz
Login:           hscroot

                 Managed Host List:

Host Name                               Uuid
=========                               ====
Host1_Site2                             82e8fe16-5a9f-3e32-8eac-1ab6cdcd5bcf
Host2_Site2                             74931f30-e852-3d47-b564-bd263b68f1b1
Host3_Site2                             c15e9b0c-c822-398a-b0a1-6180872c8518
Host4_Site2                             f6cbcbda-8fec-3b6e-a487-160ca75b2b84
Host5_Site2                             4ce17206-4fde-3d5a-a955-dbf222865a77
Host6_Site2                             346f184d-bace-36f5-97b5-3955c62a6929
Host7_Site2                             a977d52e-fd3a-325d-bd02-124663066cac
Host8_Site2                             ae115482-3a50-32f3-935a-7ff9be433e33
Host9_Site2                             b3880199-3b8b-3ade-b360-f76146c2d7f3
Host10_Site2                           26c7c48e-3b81-363b-85d0-e110ebc43b15
============================================================================


Name:            Site1_HMC1
Site:            Site1
Ip:              9.xx.yy.zz
Login:           hscroot


                        Managed Host List:

Host Name                               Uuid
=========                               ====
Host1_Site1                             caffee0a-4206-3ee7-bfc2-f9d2bd3e866f
Host2_Site1                             6ce366c5-f05d-3a12-94f8-94a3fdfc1319
Host3_Site1                             67ff62ec-ecb5-3ad4-9b35-0a2c75bb7fe4

To query the details about a specific HMC:
```
ksysmgr query hmc hmcname
```

To modify the details of a specific HMC:

ksysmgr modify hmc hmcname
      [login=new_username]
      [password=new_password]
      [hostname|ip=hostname|ip]

For example, to modify the login and password details of the PrimaryHmcName HMC, enter the following command:

ksysmgr modify hmc PrimaryHmcName login=scott password=tiger

To remove an HMC from the KSYS configuration:
```
ksysmgr delete hmc hmcname
```
To sync updated HMC information, including latest details on newly added LPARs:
```
ksysmgr refresh hmc <hmcname>|<ALL>
```

Host configuration examples

To add a host to the KSYS resource manager:

ksysmgr add host <CEC name on HMC>
      <site=sitename>
      [uuid=<uuid>]
      [hostname|ip=<hostname|ip>]

For example,

ksysmgr add host Site1_host1 site=Site1

To list details about all hosts:

ksysmgr query host

An output that is similar to the following example is displayed:

Name:            <Site name_Host name>
UUID:            <Host_UUID> 
FspIp:           <FSP_IP> 
Host_group:      <HG1>
Pair:            <Pair>
Vswitchmap:      Not currently set
Vlanmap:         Not currently set
DrVswitchmap:    Not currently set
DrVlanmap:       Not currently set
Skip_power_on:   No
Site:            Site
VIOS:            hkv2
                 hkv1
HMCs:            vmhmc
MachineSerial:   <Machine_serial>

To list details about a specific host:
```
ksysmgr query host hostname
```
To pair two hosts across sites, use the following command syntax:
```
ksysmgr pair host hostname pair=hostname
```
To unpair two hosts across sites, use the following command syntax:
```
ksysmgr pair host hostname pair=none
```
To remove a host from the KSYS resource manager, use the following command syntax:
```
ksysmgr delete host hostname
```
To sync updated host information, including latest details on newly added LPARs:
```
ksysmgr refresh host <hostname>|ALL
```

To modify the details of a host:

ksysmgr modify host <hostname1[,hostname2,...]> | file=<filepath>
[uuid=<uuid>] [skip_power_on=<yes|no>] [network=<vlanmap | vswitchmap> sites=<siteA,siteB> siteA=<#[,#,...] | all> siteB=<#[,#,...] | all> [dr_test=<yes|no>] [policy=<add|delete>]]

Host group configuration examples

To create a host group, add the existing hosts to the new host group:

For EMC storage system:

ksysmgr add host_group hg_name [site=<sitename>] hosts=host1,[host2,...] |
file=filepath [mxn_pairing=<yes>] [workgroup_support=<no>][customcgname=<CgName>] [bsitecgname=<CgName>]

For storage systems other than EMC storage systems:

ksysmgr add host_group hg_name [site=<sitename>] hosts=host1,[host2,...] | 
file=filepath [mxn_pairing=<yes>] [workgroup_support=<no>][customcgname=<CgName>]

where, the file parameter is an XML file that contains a list of hosts.

To add or remove hosts from the existing host groups:

ksysmgr modify host_group hg_name add | remove
hosts=host1,[host2,...] | file=filepath

To modify the capacity-related attributes for all the hosts in a host group:

ksysmgr modify host_group hg_name options
      [memory_capacity=<(1-100) | minimum | current_desired | none | default>
        priority=<low | medium | high>]
      [cpu_capacity=<(1-100) | minimum | current_desired | none | default>
        priority=<low | medium | high>]
      [skip_power_on=<yes|no>]
      [sriov_override=<yes|no>]
      [network=<vlanmap | vswitchmap>  sites=<siteA,siteB>
        siteA=<#[,#,...] || all> siteB=<#[,#,...] || all> [dr_test=<yes|no>]
        [policy=delete]]
      [customcgname=<CgName>]
      [bsitecgname=<CgName>]
    modify => mod*, ch*, set
    host_group => hg, host_g*

Note: bsitecgname is applicable only for EMC storage
      customcgname, bsitecgname setting can only be modified before first discovery

To create a network mapping policy of VLAN ID or virtual switches for all the hosts in a host group, across sites:

ksysmgr modify host_group hg_name options
      network=vlanmap | vswitchmap  sites=siteA,siteB
       siteA=<#,[#,...]> siteB=<#,[#,...]>

To discover and verify all the hosts in a specific host group:
```
ksysmgr discover host_group hg_name verify=true
```
To delete a host group:
```
ksysmgr delete host_group hg_name
```
To query the details about all host groups or a specific host group:
```
ksysmgr query host_group [hg_name]
```

VIOS configuration example

To query the detail about a VIOS, run the following command:

ksysmgr query vios [<viosname>]

An output that is similar to the following example is displayed:

Name:                VIOS1
Site:                Site1
UUID:                UUID1
Host:                Host1
Version:             VIOS 3.1.1.22
State:               MANAGED

Name:                VIOS2
Site:                Site1
UUID:                UUID2
Host:                Host1
Version:             VIOS 3.1.1.22
State:               MANAGED

To unmanage or manage a VIOS, run the following command:
```
ksysmgr unmanage|manage vios <viosname>
```

Workgroup configuration example

- To create a workgroup using the EMC storage systems, run the following command from the command-line interface (CLI) of the KSYS node:
```
ksysmgr add workgroup <name> vm=<vmname1[,vmname2...]> [customcgname=<CgName>] [bsitecgname=<CgName>]
```
- To create a workgroup using the storage systems other than the EMC storage systems, run the following command from the command-line interface (CLI) of the KSYS node:
```
ksysmgr add workgroup <name> vm=<vmname1[,vmname2...]> [customcgname=<CgName>]
```
To add or remove the virtual machines from an existing workgroup, run the following command:
```
ksysmgr modify workgroup workgroup_name policy=<add | delete> vm=<vmname1[,vmname2...]>
```

To move a workgroup from one site to another site, run the following command:

ksysmgr [-f] move workgroup <name>
to=<site_name>
[force=<true|false>]
[lose_vios_redundancy=<yes|no>]
[dr_type=<planned|unplanned>]
[cleanup=<yes|no>]
[dr_test=<yes|no>]
[skip_shutdown=<yes|no>]
move => mov*, mv, swi*
workgroup => workg*, work_g*, wg

To query the details about a specific work group:

ksysmgr query wg

An output that is similar to the following example is displayed:

         Name:                           WG1
         VMs:                            VM1
                                         VM2
         Host_group:                     HG1
         Site:                           Site1
         Custom CG Name:
         Backup Site CG Name:

         Name:                           WG2
         VMs:                            VM3
                                         VM4
         Host_group:                     HG1
         Site:                           Site1
         Custom CG Name:
         Backup Site CG Name:

To add a new workgroup:

ksysmgr add workgroup <name> vm=<vmname1,vmname2...>

To delete an existing workgroup:
```
ksysmgr delete workgroup <name> 
```

To discover and verify all VMs in a specific work group:

ksysmgr discover workgroup <name> verify=yes [dr_test=<yes|no>]

To modify a workgroup:

ksysmgr modify workgroup <name> policy=<add | delete> vm=<vmname1[,vmname2...]>

To recover a workgroup:
```
ksysmgr recover workgroup name
```
To clean up a workgroup:
```
ksysmgr cleanup workgroup name
```
To resync a workgroup:
```
ksysmgr [-f] resync workgroup name
```

LPAR configuration examples

To include or exclude a specific virtual machine from the KSYS configuration:
```
ksysmgr unmanage vm
name=<vmname> host=<hostname> | uuid=<lparuuid> | ALL host=<hostname> | ALL
host_group=<host_group_name>
```
The ksysmgr unmanage ALL host=<hostname> command will unmanage all virtual machines in the host <hostname>. The ksysmgr unmanage ALL host_group=HG1 command will unmanage all virtual machines in the host group HG1.
```
ksysmgr unmanage vm <vmname|lparuuid,....>
unmanage => unman*, umg
vm => lp*, vm
```
The preceding command syntax can be used for targeted VM management. The excluded virtual machine is not moved to the backup site when a site-switch operation is initiated.
Notes:
- After including or excluding a virtual machine, you must run the discovery and verification commands to rediscover the resources and validate the modified configuration setting.
- If you use SAN Volume Controller storage system, before you include a specific virtual machine to the KSYS subsystem, you must ensure that the new virtual machine is associated with the storage volumes that have the same relationship type as the other existing virtual machines. For more information about this restriction, see SAN Volume Controller system and Storwize system.

To update the priority of virtual machines:

ksysmgr modify vm <vmname[,vmname2,...]> | file=<filepath> [uuid=<uuid>] [host=<hostname>] [homehost=<hostname>] [DrTargetHost=<hostname | none>] [priority=<Low|Medium|High>] [skip_power_on=<yes|no>] [mappings=<poolmap> sites=<siteA,siteB> siteA=<#[,#,...] siteB=<#[,#,...] policy=<add|delete>] modify => mod*, ch*, set vm => lp*, vm

where, the file parameter is an XML file that contains a list of virtual machine references.

Note:

The DrTargetHost attribute is valid only for an asymmetric type of host group.
The UUID attribute is used if more than one virtual machines exist with the same name.

To add a shared processor pool mapping for a virtual machine:

ksysmgr modify vm vm1
mappings=poolmap sites=Site1,Site2 Site1=Site1_Pool
Site2=Site2_Pool policy=add

To manage or unmanage discovered VMs automatically by using the vm_auto_discovery attribute:
```
ksysmgr modify system vm_auto_discovery=enable|disable
```
When enabled, the discovered VMs are automatically managed. In the disabled mode, the discovered VMs are unmanaged by default.
To display only minimum information about VMs before performing the first discovery operation, run the following command:
```
ksysmgr query vm display_all=yes
```
Note: The display_all option can be used before performing the first discovery operation to display only minimum information about the VMs that are added to the KSYS configuration. To display the current information about a VM, use the ksysmgr query vm command without the display_all option.

Storage configuration examples

To add a storage agent to a site:

ksysmgr add storage_agent StorageNamePrimary 
site=Site1
serialnumber=000196xxx
storagetype=emc 
ip=1.2.xx.xx

Note:

The login or password credentials might be needed for some storage devices.
The DrInstance instance number is required only for Hitachi storage if you want to perform the DR test.
Only the XIV storage systems supports up to three IP addresses.

SVC storage systems:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname | ip> site=<sitename> storagetype=svc clusterid=<number> login=<username>

EMC or SRDF storage systems:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname | ip> site=<sitename> storagetype=srdf serialnumber=<number>

DS8000 storage system:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname | ip> site=<sitename> storagetype=ds8k login=<username> password=<password>

Hitachi storage systems:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname | ip> site=<sitename> storagetype=hitachi serialnumber=<number> instance=<instance_number> [drinstance=<drinstance_number>] login=<username> password=<password>

XIV storage systems:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname[,hostname2,hostname3] | ip[,ip2,ip3]> site=<sitename> storagetype=xiv serialnumber=<number> login=<username> [password=<password>]

EMC Unity storage:

ksysmgr add storage_agent <storage_agent_name> hostname | ip =<hostname | ip> site=<sitename> storagetype=unity serialnumber=<number> login=<username> [password=<password>]

To list storage agent details:

ksysmgr query storage_agent storage_agent_name

An output that is similar to the following example is displayed:

Name:             StorageNamePrimary
Serialnumber:     00019xxxx
Storagetype:      SVC
Site:             Site1
Ip:               1.2.xx.xx
Login:            admin

To remove a storage agent from a site:

ksysmgr delete storage_agent storage_agent_name

To resynchronize the storage data after an unplanned recovery from the active site to the backup site:
```
ksysmgr [-f] resync site active_site_name
ksysmgr [-f] resync host_group active_hg_name
ksysmgr [-f] resync workgroup <workgroup_name>
```
If the unplanned move operation was performed at the site level, you must run the ksysmgr [-f] resync command at site level. Similarly, if a virtual machine was moved to the backup site during an unplanned move operation at host group level, you must run the ksysmgr [-f] resync command at host group level.

To sync updated storage agent information:

ksysmgr refresh storage_agent <storage_agent_name>|ALL

Discovery and verification examples

To discover the resources in a site:

ksysmgr discover site sitename

An output that is similar to the following example is displayed:

Running discovery on entire site, this may take few minutes...
        Storage state synchronization has started for Host_group HG1
        Storage state synchronization has completed for Host_group HG1
        Discovery has started for VM rtXYZXY
        Configuration information retrieval started for VM rtXYZXY
        Configuration information retrieval completed for VM rtXYZXY
        Storage information retrieval from VIOS started for VM rtXYZXY
        Storage information retrieval from VIOS completed for VM rtXYZXY
        Discovery for VM rtXYZXY is complete
        Disk Group creation on storage subsystem started for Host_group HG1
        Disk Group creation on storage subsystem completed for Host_group HG1
Discovery has finished for Site1
1 out of 1 managed VMs have been successfully discovered(0)

To discover resources and to verify the KSYS configuration:

ksysmgr discover site sitename verify=true

An output that is similar to the following example is displayed:

Running discovery on entire site, this may take few minutes...
        Storage state synchronization has started for Host_group HG1
        Storage state synchronization has completed for Host_group HG1
        Discovery has started for VM rtXYZXY
        Configuration information retrieval started for VM rtXYZXY
        Configuration information retrieval completed for VM rtXYZXY
        Storage information retrieval from VIOS started for VM rtXYZXY
        Storage information retrieval from VIOS completed for VM rtXYZXY
        Discovery for VM rtXYZXY is complete
        Disk Group creation on storage subsystem started for Host_group HG1
        Disk Group creation on storage subsystem completed for Host_group HG1
Discovery has finished for Site1
1 out of 1 managed VMs have been successfully discovered
        Site verification started for Site1
        Default_HG verification has started
        HG1 verification has started
        Default_HG verification has completed
        rtXY-XYZXYZ-XYA-XYZXECV verification has started
        rtXY-XYZXYZ-XYA-XYZXECV verification has completed
        rtXYZXY DR verification has started
        rtXYZXY DR verification has completed
        Disk Group verification on storage subsystem started for Host_group HG1
        Disk Group verification on storage subsystem completed for Host_group HG1
        HG1 verification has completed
Verification has finished for Site1
1 out of 1 VMs have been successfully verified(0)

Script configuration examples

To add a script for automatic execution before or after the discovery and verification operations:
```
ksysmgr add script entity=site|host_group pre_verify|post_verify=script_path
```
Note: The pre_verify and post_verify scripts can be run only at site or host group level.

To add a script for automatic execution before or after the disaster recovery move operation:

ksysmgr add script entity=site|host_group|vm 
    pre_offline|post_offline|pre_online|post_online=script_path

Events query examples

To query the events of a specific type:

ksysmgr query event [type=<error|warning|info>]

To query the events of all types, run the following command:

ksysmgr query event

An output that is similar to the following example is displayed:

Current Notification Level: Low
Event Name                                        Event Type     Description
--------------------------------------------------------------------------------

HMC_UNREACHABLE                                    error         HMC is down or not reachable
HMC_REACHABLE                                      info          HMC has recovered and is now reachable
VIOS_RMC_STATE_DOWN                                error         HMC to VIOS RMC connectivity/network seems to be having problems
VIOS_FAILURE                                       error         VIOS seems to have failed
VM_CONFIG_COLLECTION_FAILURE                       error         Configuration data collection failed for the VM
DAILY_VERIFY_FAILED                                error         Daily verification checks have failed
HOST_FAILURE                                       error         Host failure has occurred
FILESYSTEM_SPACE_WARNING                           warning       Filesystem is reaching full condition
VM_MOVE                                            info          VM has moved from one host to another
VM_STORAGE_COLLECTION_FAILURE                      error         VM disk information retrieval from VIOS has failed
HMC_LOGIN_FAILURE                                  error         HMC login failed. Check username and password
VIOS_DELETED                                       warning       VIOS deletion has been detected
VM_NOT_ACTIVE                                      info          VM does not seem to be active
VM_DISCOVERED_ON_HOST                              info          VM has been detected on host
VM_DELETED_FROM_HOST                               warning       VM has been deleted from host
VM_AUTO_START_ENABLED                              error         VM auto_start is enabled for LPAR.
HA_SWITCH_CREATE_FAILED                            error         HA related switch creation failed for host
HA_SWITCH_DELETE_FAILED                            error         HA related switch deletion failed for host
HA_TRUNK_ADAPTER_CREATE_FAILED                     error         Trunk adapter creation failed for VIOS
HA_TRUNK_ADAPTER_DELETE_FAILED                     error         Trunk adapter deletion failed for VIOS
HA_NW_ADAPTER_FAILED_FOR_VM                        error         Client ethernet adapter creation failed
CREATE_SSP_CLUSTER_FAIL                            error         SSP cluster creation failed
HA_INIT_OF_VIOS_IN_HOST_FAILED                     error         HA initialization of VIOS in a Host has failed
HA_INIT_OF_VM_IN_VIOS_FAILED                       error         Initialization of VM HA management inside VIOS failed
VM_START_HB_FAILED                                 error         Start HB failed
VM_STOP_HB_FAILED                                  error         Stop of VM for HB management failed
UNSUPPORTED_VERSION                                error         Version is not supported
NO_TARGET_HOST_IDENTIFIED_FOR_LPM_OF_VM            error         No Target host could be identified that can host the VM (capacity is insufficient, etc)
LPM_VALIDATION_FAILED_FOR_VM                       error         LPM validation failed in verify phase
LPM_FAILED error LPM operation failed
HOST_CONFIG_REDISCOVERY_AFTER_LPM_FAILED           error         Host rediscovery failed after LPM
VM_RESTART_FAILED                                  error         Restart of VM failed on target host
NO_TARGET_HOST_IDENTIFIED_FOR_RESTART_OF_VM        error         No Target host could be identified that can host the VM (capacity is insufficient, etc)
VM_CLEANUP_ON_HMC_FAILED                           error         VM cleanup operation on HMC failed
VM_VERIFY_FAILED                                   error         VM verification failed
VERIFY_STARTED                                     info          Host group verification Started.
VERIFY_FAILED                                      error         Host group verification failed
VM_REBOOT_FOR_APPLICATION_FAILURE                  info          VM Reboot initiated on the Host due to critical application failure
VM_RESTART_FOR_APPLICATION_FAILURE                 info          VM Restart/relocation initiated on another Host due to critical application failure
ALL_VIOS_UNREACHABLE                               error         Could not reach all of VIOSes in the Host group
VIOS_CAA_STATE_DOWN_DETECTED                       info          CAA state DOWN
VIOS_CAA_STATE_UP_DETECTED                         info          CAA state UP
VM_FAILURE_DETECTED                                error         VM failure detected
HOST_FAILURE_DETECTED                              error         HOST failure detected
VM_FDE_DISCOVER                                    info          Failure detection process doing discovery
VM_CLEANUP_PENDING                                 info          VM cleanup is pending
SCRIPT_SUCCESS_EVENT                               info          Script successfully ran
REPLICATION_LINK_FAILURE                           error         Storage is reporting replication failure
VM_ON_MULTIPLE_HOST                                error         VM is seen on more than one host
UNMANAGED_VM_LPM_DETECTED                          warning       Unmanaged VM performing LPM
VM_MON_SETUP_ISSUES                                error         VM agent within VM could not establish connection with VIOS host monitoring subsystem
VM_MON_MISSEDHB_SETUP_ISSUES                       error         Cannot issue relocation for VM as it is not being monitored by VIOS
VIOS_MONITOR_MODE                                  info          Monitor mode changed
VIOS_NODE_ACTIVE                                   info          Cluster is active on VIOS
REPOSITORY_DISK_ACCESSIBLE                         info          SSP repository disk is now accessible
HA_DISK_ACCESSIBLE                                 info          SSP HA disk is now accessible
NETWORK_INTERFACE_ADDED                            info          Network interface has been configured
NETWORK_INTERFACE_ACTIVE                           info          Network interface is now active
NETWORK_INTERFACE_DELETED                          info          Network interface has been deleted
REPOSITORY_DISK_FAILURE                            error         Reading from SSP repository disk failed
HA_DISK_FAILURE                                    error         Reading from SSP HA disk failed
VIOS_NODE_FAILURE                                  error         Cluster is down on VIOS
NETWORK_INTERFACE_FAILURE                          error         Network interface failure detected
SCRIPT_FAILURE_EVENT                               error         Script failed to execute for an event
VIOS_SLOW_IO_DETECTED                              error         Slow I/O for VIOS detected
VIOS_SLOW_IO_RECOVERED                             error         Recovered from slow I/O for VIOS
APP_FAILURE_DETECTED                               error         Application failure is detected
VIOS_HOST_MON_NONRESP_DETECTED                     error         VIOS: Host Monitor non-responsive
STG_UNREACHABLE                                    error         Storage is unreachable
SITE_MOVE_FAILED                                   error         Site move failed
HOSTGROUP_MOVE_FAILED                              error         Host group move failed
SITE_MOVE_STARTED                                  info          Site move started
HOSTGROUP_MOVE_STARTED                             info          Host group move started
STG_REACHABLE                                      info          Storage is reachable
FILESYSTEM_SPACE_SUFFICIENT                        info          Filesystem space is sufficient
DISCOVERY_SUCCESS                                  info          Discovery completed successfully
VIOS_FILE_SYSTEM_REACHED_THRESHOLD                 warning       VIOS Filesystem is reaching full condition
MIRROR_RELATIONSHIP_MISSING                        error         Disk has no mirror pair
WORKGROUP_MOVE_STARTED                             info          Workgroup move started
WORKGROUP_MOVE_FAILED                              error         Workgroup move failed
VM_CLEANUP_SUCCESS                                 info          VM cleanup is successful
DRTEST_DISCOVERY_FAILED                            error         Discovery failed with dr_test enabled
DRTEST_DISCOVERY_STARTED                           info          Discovery started with dr_test enabled
DRTEST_VERIFY_STARTED                              info          Verify started with dr_test enabled
DRTEST_VERIFY_FAILED                               error         Verify failed with dr_test enabled
DRTEST_DISCOVERY_SUCCESS                           info          Discovery successful with dr_test enabled
VM_MIGRATED_ACROSS_HG                              info          VM has been migrated across HG.
MIGRATED_VM_EXISTS_IN_WG                           info          Migrated VM exists in WG.
VIOS_CRITICAL_VG_SET                               info          Critical VG set for VIOS
VIOS_CRITICAL_VG_SET_FAILED                        error         Failure setting critical VG for VIOS
VM_DRREHEARSAL_FAILED                              error         VM DR Rehearsal has failed
VM_DRREHEARSAL_SUCCESS                             info          VM DR Rehearsal was a success
CAA_NODE_UP_EVENT                                  info          CAA node is up for VIOS
CAA_NODE_DOWN_EVENT                                error         CAA node is down for VIOS
NETWORK_ISOLATION_SUCCESS                          info          Network isolation configuration success for VIOS
NETWORK_ISOLATION_ERROR                            error         Network isolation configuration error for VIOS
REPOSITORY_DISK_UP                                 info          Repository disk is up for VIOS
REPOSITORY_DISK_DOWN                               error         Repository disk is down for VIOS
SITE_DISCOVERY_SUCCESS                             info          Site discovery is successful
SITE_VERIFY_SUCCESS                                info          Site verify is successful
VM_APPLICATION_PERMANENT_FAILURE                   error         VM application permanently failed
DRTEST_SITE_DISCOVERY_SUCCESS                      info          Site dr_test discovery is successful
DRTEST_SITE_DISCOVERY_STARTED                      info          Site dr_test discovery has started
DRTEST_SITE_DISCOVERY_FAILED                       error         Site dr_test discovery has failed
WORKGROUP_MOVE_SUCCESS                             info          WG move has succeeded
HOSTGROUP_MOVE_SUCCESS                             info          HG move has succeeded
HOSTGROUP_DRREHEARSAL_STARTED                      info          HG DR Rehearsal has started
HOSTGROUP_DRREHEARSAL_FAILED                       error         HG DR Rehearsal has failed
WORKGROUP_DRREHEARSAL_STARTED                      info          WG DR Rehearsal has started
WORKGROUP_DRREHEARSAL_FAILED                       error         WG DR Rehearsal has failed
HOSTGROUP_DRREHEARSAL_SUCCESS                      info          HG DR Rehearsal has succeeded
WORKGROUP_DRREHEARSAL_SUCCESS                      info          WG DR Rehearsal has succeeded
SITE_MOVE_SUCCESS                                  info          Site move has succeeded
SITE_DRREHEARSAL_SUCCESS                           info          Site DR Rehearsal has succeeded
AUTO_CHREPOS_COMPLETE                              error         Automatic Change of Repodisk has completed
DISK_VALIDATION_FAILURE                            error         Disk validation failed
CREATE_SSP_CLUSTER_SUCCESS                         info          SSP cluster successfully created
DISCOVERY_FAILED                                   error         Discovery has failed
DISCOVERY_STARTED                                  info          Discovery started
SITE_DISCOVERY_FAILED                              error         Site discovery has failed
SITE_DISCOVERY_STARTED                             info          Site discovery started
SITE_VERIFY_STARTED                                info          Site verify started
SITE_VERIFY_FAILED                                 error         Site verify has failed
VM_VERIFY_DONE                                     info          VM verification is done
DAILY_VERIFY_SUCCESS                               info          Daily verify has succeeded
DAILY_VERIFY_STARTED                               info          Daily verify started
VERIFY_SUCCESS                                     info          Verify has succeeded
VM_RESTART_SUCCESS                                 info          VM Restart has succeeded
DRTEST_VM_CLEANUP_SUCCESS                          info          Drtest VM cleanup has succeeded
DRTEST_VM_CLEANUP_STARTED                          info          Drtest VM cleanup has started
AHAFS_SFW_ADAP_UP                                  info          Adapter up event for VIOS belonging to Host Group.
AHAFS_SFW_ADAP_DOWN                                error         Adapter down event for VIOS belonging to Host Group.
VM_HB_LOSS_EXCEEDED                                error         VM missed heartbeats exceeded the failure detection time.
VM_HB_LOSS_RECOVERED                               info          VM no longer missing heartbeats
VM_HB_LOSS_DETECTED                                error         VM is missing heartbeats
DISCOVERY_BLOCKED                                  info          Quick auto discovery is blocked as HA operations are in progress for HGs
DAILY_VERIFY_BLOCKED                               info          Deep auto discovery is blocked as HA operations are in progress for HGs
INCONSISTENT_VIEW_OF_VIOS                          error         VIOSes in HG have an inconsistent view of VIOS.
RECOVERY_FAILED                                    info          Recovery operation has failed.
RECOVERY_SUCCESS                                   info          Recovery operation has succeeded.
RECOVERY_STARTED                                   info          Recovery operation has started.
DRTEST_VM_CLEANUP_FAILED                           error         Drtest cleanup operation has failed.
SITE_DRREHEARSAL_START                             info          Site DR Rehearsal has started
VM_REBOOT_FOR_APPLICATION_FAILURE_SKIPPED          info          Vm Reboot for Application Failure Skipped
VM_REBOOT_SKIPPED_FOR_APPLICATION_FAILURE          error         Vm Reboot Skipped for Application failure
VM_RESTART_FOR_APPLICATION_FAILURE_SKIPPED         info          Vm Restart for Application Failure Skipped
HA_NW_ADAPTER_DELETE_FAILED_FOR_VM                 error         HA Network Adapter Deletion failed for VM
VM_NO_TARGET_HOST                                  error         No Target Host Available for VM
DRTEST_VERIFY_SUCCESS                              info          Verify successful with dr_test enabled
VM_CPU_THRESHOLD_EXCEEDED                          warning       VM's CPU utilization exceeds threshold
ADD_DISK_TO_GROUP_SUCCESS                          info          Add Disk to CG succeeded
SSP_CLUSTER_CREATE_STARTED                         info          SSP cluster creation started
VM_HB_STARTED                                      info          VM has established heartbeats/communication with VIOS and KSYS will now start monitoring VM & applications inside the VM
VM_CLEANUP_ON_HMC_SUCCESS                          info          VM cleanup operation on HMC SUCCESS
INSUFFICIENT_HOST_CAPACITY                         error         Migration to backup partner host will be affected due to the scarcity of resources on backup partner host. Require additional resources (Memory, Processors) on partner host
SITE_DRREHEARSAL_FAILED                            error         Site DR Rehearsal failed
VM_STORAGE_COLLECTION_FAILURE_FROM_ALL_VIOSES      error         Failed to get storage information for VM
MISSED_VM_EXISTS_IN_WG                             error         VM missing from HG which is part of WorkGroup VMList
LPM_OPERATION_START                                info          LPM operation started
LPM_OPERATION_COMPLETE                             info          LPM operation complete
DO_PUT_LSI_FAILED                                  error         Do Put LSI Failed
ADD_DISK_TO_GROUP_FAILED                           error         addDisksToGroup() failed
DISK_GROUP_CREATION_FAILED                         error         Disk group creation failed
STG_IP_UNREACHABLE warning IP Address of Storage Agent is not reachable
VM_RESTART_SKIPPED_FOR_APPLICATION_FAILURE         error         Restart operation skipped for application failure
VM_REBOOT_SKIPPED                                  info          Reboot operation skipped for VM
VM_RESTART_SKIPPED                                 info          Restart operation skipped for VM
APP_ADDED                                          info          Application is added
APP_DELETED                                        info          Application is deleted
SFW_PORT_UP                                        info          Port up event for VIOS belonging to Host Group
VM_PACKET_THRESHOLD_EXCEEDED warning VM exceeded dropped packets threshold
SFW_PORT_DOWN                                      error         Port down event for VIOS belonging to Host Group
VM_CLEANUP_SKIPPED                                 info          VM cleanup skipped, as VM does not exist on host
VM_CONFIG_REDISCOVERY_SUCCESS                      info          VM configuration rediscovery successful
VM_CONFIG_REDISCOVERY_FAILED                       error         VM configuration rediscovery failed
WG_VM_INVALID_SITE                                 warning       VM is part of Workgroup but not in Active Site
VSWITCH_VLAN_VALIDATION_FAILED                     warning       User configured and system discovered vswitch or vlan IDs are not matched
KSYS_MANAGING_NODE_CHANGED                         warning       WARNING: Managing node changed
KSYS_NODE_STATUS_CHANGED                           warning       WARNING: Ksysnode status changed
DISKPAIR_FAILED                                    error         DiskPair ERROR.
SSP_ADD_NODE_SUCCESS                               info          SSP add node succeeded for VIOS
SSP_ADD_NODE_FAILED                                error         SSP add node failed for VIOS
SSP_REMOVE_NODE_SUCCESS                            info          SSP remove node succeeded for VIOS
SSP_REMOVE_NODE_FAILED                             error         SSP remove node failed for VIOS
SWITCH_UPDATE_FAILED                               error         Switch update failure for host

To query the system wide persistent attribute for the ksysmgr command, use the following command syntax:

ksysmgr query system

An output that is similar to the following example is displayed:

ksysmgr query system

System-Wide Persistent Attributes
auto_discovery_time:         00:00 hours
lose_vios_redundancy:        no
auto_reverse_mirror:         yes
notification_level:          low
dup_event_processing:        yes
replication_type:            Asynchronous
network_mapping:             Enabled
vlanmap:                     Not currently set
vswitchmap:                  Not currently set
drvlanmap:                   Not currently set
drvswitchmap:                Not currently set
quick_discovery_interval:    60 minutes
quick_discovery:             enable
deep_discovery:              enable
vm_auto_discovery:           enable
custom_script_timeout:       none
trace_file_size:             not set
memory_capacity:     Priority Based Settings
                             low:100
                             medium:100
                             high:100
CPU_capacity:        Priority Based Settings
                             low:100
                             medium:100
                             high:100
ping:                        enable
min_redundancy_paths:        disable
cleanup_files_interval:      7 days
ksys_lang:
optimise_put_storage:       disable
connection_timeout:         75 seconds (default)
storage_group_update:       no
hmc_port_number:            443
vpmem_override:             disable
ksys_spooling:              not set
spool_dest_dir:             not set
spool_dir_max_size:         not set
host_lun_validation:        enable
hmc_ping_timer:             10 seconds
sa_ping_timer:              10 seconds
User Scripts for Host Group:
PreOfflineScript:            None
PreOnlineScript:             /opt/IBM/ksys/samples/custom_validation/PreHGOnline
PostOfflineScript:           /opt/IBM/ksys/samples/custom_validation/PostHGOffline
PostOnlineScript:            /opt/IBM/ksys/samples/custom_validation/PostHGOnline
PreVerifyScript:             /opt/IBM/ksys/samples/custom_validation/PreHGVerify
PostVerifyScript:            /opt/IBM/ksys/samples/custom_validation/PostHGVerify
User Scripts for VM: None

where,

auto_discovery_time: Sets time for daily discovery at each 24 hours a day.
min_redundancy_paths: Sets the minimum number of virtual Fiber Channel adapters that the VIOS requires for the virtual machine during a disaster recovery operation if the maximum number of virtual fibre channel adapters are not available.
lose_vios_redundancy: Sets the lose_vios_redundancy variable configuration.
notification_level: Notifies the generation of events based on severity of an event failure.
replication_type: Modifies the storage replication type.
vlanmap, vswitchmap: Sets the target site network on DR.
drvlanmap, drvswitchmap: Sets the target site network for DR rehearsal. These values are updated when the user performs the modify operation by specifying the required values with the vlanmap and vswitchmap options along with the dr_test=yes option.
quick_discovery_interval: Sets the time duration for the quick discovery operation.
quick_discovery: Sets the quick_discovery variable to enable/disable.
deep_discovery: Sets the deep_discovery variable to enable/disable.

Note: If the network_isolation attribute is set to ALL, the action attribute must have delete value. This will delete all the IP's.

To modify the system wide persistent attribute for the ksysmgr command:

ksysmgr modify system

For example,

ksysmgr modify system
      [auto_discovery_time=<hh:mm>]
        hh - hour:   00 to 23
        mm - minute: 00 to 59
      [quick_discovery_interval=<mm>]
        mm - minute: 5 to 480
      [custom_script_timeout=<sec>]
        sec - seconds: Any positive integer
      [quick_discovery=<enable | disable>]
      [deep_discovery=<enable | disable>]
      [trace_file_size=<MB>]
        MB - Megabyte: Between 1 and 50 for single node KSYS cluster
                       Between 1 and 25 for Multiple node KSYS cluster
      [ksys_spooling=<enable | disable>] 
      [spool_dest_dir=<path>] 
      [spool_dir_max_size=<MB>]
       MB - Megabyte: Between 1 and 10240
      [lose_vios_redundancy=<yes | no>]
      [notification_level=<low | medium | high | disable>]
      [dup_event_processing=<yes | no>]
      [replication_type=<async | sync | shared>   sites=<A,B>]
      [network_mapping=<enable | disable>]
      [vm_auto_discovery=<enable | disable>]
      [ping=<enable | disable>]
      [optimise_put_storage=<enable | disable>]
      [min_redundancy_paths=< no.of paths - for enable | 0 - for disable>]
      [hmc_ping_timer=<(10-30) time in seconds>] [sa_ping_timer=<(10-30) time in seconds>]
      [cleanup_files_interval=<disable | (1-30) days>]
      [memory_capacity=<(1-1000) | minimum | current_desired | none | default>
        priority=<low | medium | high>]
      [cpu_capacity=<(1-1000) | minimum | current_desired | none | default>
        priority=<low | medium | high>]
      [network=<vlanmap | vswitchmap>  sites=<siteA,siteB>
      siteA=<#[,#,...] || all> siteB=<#[,#,...] || all> [dr_test=<yes|no>]
      [policy=delete]]
      [ksys_lang=<language>] 
      [connection_timeout=<default | (1-600) time in seconds>] 
      [storage_group_update=<yes | no>] 
      [hmc_port_number=<val>] 
          val - 443 or 12443 
      [vpmem_override=<enable | disable>]
      [host_lun_validation=<enable | disable>]
    modify => mod*, ch*, set
    system => sys*
    Note: Not advisable to modify quick_discovery_interval with values less than 60 mins.
    Note: If custom_script_timeout value is set to 0, it will be considered as no timeout is set.
    Note: trace_file_size limit check errors can be overriden with -f flag for only single node
          ksys cluster,trace_file_size can be set to maximum of 25MB in case of multi-node KSYS cluster
    Note: If you specify cpu_capacity and memory_capacity together, only specify a single priority. 
          If multiple priorities are given the latest priority value is modified. 
    Note: Supported locales for ksys_lang are DE_DE, FR_FR, JA_JP, PT_BR, ZH_TW, ES_ES, IT_IT, ZH_CN, en_US 
          By default language is considered to be en_US

Note: The value in the sa_ping_timer attribute and the hmc_ping_timer attribute must be in the range 10 and 30. You can view the configured value in the output of the ksysmgr query system command.

Quick discovery example

To enable or disable the quick-discovery feature:

ksysmgr modify system quick_discovery=<enable | disable>

To set the time duration for the quick-discovery operation:
```
ksysmgr modify system quick_discovery_interval = <minutes>
```

Notification configuration examples

To add an email or SMS notification for a specific user:

ksysmgr add notify user=username contact=email_address
ksysmgr add notify user=username contact=10_digit_phone_number@phone_carrier_email_address

For example,

ksysmgr add notify user=John contact=john.doe@testmail.com
ksysmgr add notify user=John contact=1234567890@tmomail.net

To modify an email address or SMS number for a specific user:

ksysmgr modify notify oldcontact=old_username newcontact=new_username
ksysmgr modify notify oldcontact=old_email_address newcontact=new_email_address

To query all the registered contact details:
```
ksysmgr query notify contact
```
To delete all the contact information for a specific user:
```
ksysmgr delete notify user=username
```

To add a script for a predefined set of notifications and subsequent actions for a specific event:

ksysmgr add notify script=full_path_script event=event_name

For example,

ksysmgr add notify script=/tmp/script.sh event=HMC_DOWN

To modify a script:

ksysmgr modify notify oldscript=old_file_name newscript=new_file_name

To remove a script:

ksysmgr delete notify script=full path script

To query a script:
```
ksysmgr query notify script
```

System-wide attributes configuration

To query details about system-wide attributes:
```
ksysmgr query system
```
To enable the KSYS subsystem to automatically rediscover the resources at noon every day:
```
ksysmgr modify system auto_discovery_time=12:00
```
To change the notification level of your system to receive notification for all critical errors and warnings of all events:
```
ksysmgr modify system notification_level=medium
```
To change the duplicate event processing option to receive notification for all events, even if the events are duplicated:
```
ksysmgr modify system dup_event_processing=no
```
To change the storage replication mode between two sites from asynchronous mode to synchronous mode:
```
ksysmgr modify system replication_type=sync sites=SiteA,SiteB
```
To enable the network mapping function and to create network mapping policy for all hosts and host groups across the active and the backup site:
```
ksysmgr modify system network_mapping=enable network=vlanmap|vswitchmap sites=siteA,siteB
       siteA=<#,[#,...]> siteB=<#,[#,...]>]
```
To set the Ping tunable at the system level, run the following command:
```
ksysmgr modify system Ping=<enable|disable>
```
To set storage ping timer to 15 seconds at the system level:
```
ksysmgr modify system sa_ping_timer=15
```
To set HMC ping timer to 20 seconds at the system level:
```
ksysmgr modify system hmc_ping_timer=20
```

Disaster recovery failover rehearsal examples

To discover, set up, and enter into DR failover rehearsal mode:
```
ksysmgr discover workgroup|host_group|site name dr_test=yes
```
To check whether the test environment is ready for the test-move operation:
```
ksysmgr verify workgroup|host_group|site name dr_test=yes
```

To start the VMs in the DR test environment:

ksysmgr move workgroup|host_group|site name from=Site1 to=Site2 dr_test=yes

To clean up the DR test environment and to retain the normal DR backup state:
```
ksysmgr cleanup workgroup|host_group|site name dr_test=yes
```

Verify the host group for the DR only validations:

# ksysmgr verify host_group -?
ksysmgr verify host_group <name>
      [dr_test=<yes|no>]
      [option=<DRonly>]
    verify => ver*, validate
    host_group => hg, host_g*

Disaster recovery operation examples

To initiate a site-switch operation:
```
ksysmgr move site|host_group|workgroup <site_name|hg_name|workgroup_name>
      from=sitename
      to=sitename
      [force=true|false]
      [lose_vios_redundancy=yes|no]
      [dr_test=yes|no]
      [dr_type=planned|unplanned]
      [site_down=yes|no]
      [cleanup=yes|no]
      [skip_shutdown=yes|no]
    move => mov*, mv, swi*
    site => sit*
```
Where, the default value of the force attribute is false, the default value of the lose_vios_redundancy attribute is no, and the default value of the dr_type attribute is planned. The site_down attribute is applicable only for unplanned move operations.
For example:
```
ksysmgr move site from=Site1 to=Site2 dr_type=planned cleanup=no
```
If you do not specify the cleanup attribute, for a planned disaster recovery operation, the KSYS subsystem automatically cleans up the source site from the location where the site-switch operation was initiated.
To recover failed virtual machines after the move operation is complete:
```
ksysmgr recover host_group|workgroup name
```
To perform cleanup operation after successful move operation:
```
ksysmgr cleanup site|host_group|workgroup name
```
For an unplanned recovery, you must manually clean up the source site after the HMC and hosts become operational.

Snapshot examples

To save a snapshot of the KSYS cluster configuration and resources:

ksysmgr add snapshot filepath=full_file_prefix_path|file_prefix ]

For example,

ksysmgr add snapshot filepath=/home/ksysdir/myksysbackup

To view a snapshot, use the following command syntax:

ksysmgr query snapshot filepath=full_file_prefix_path

For example,

ksysmgr query snapshot filepath=/home/ksysdir/myksysbackup_2016_06_23_04:54:30.xml.tar.gz

To restore the configuration data on a KSYS node:
```
ksysmgr restore snapshot filepath=full_file_prefix_path
```
For example:
```
ksysmgr restore snapshot filepath=/home/ksysdir/myksysbackup_2016_06_23_04:54:30.xml.tar.gz
```
This command decompresses and unarchives the snapshot file, and then applies the configuration settings to the KSYS node.

Collecting log files for the KSYS subsystem

To collect log files for the KSYS subsystem, run the following command:
```
snap vmsnap
```
Note: For the cluster type, DR, the HM and VM logs will not be collected.
To print trace information from KSYS trace files to standard output:
```
ksysmgr trace logs=ksys|fde|fdelong|krest|krestlong|user|ALL
```

VM auto discovery

To check whether the VM auto-discovery property is enabled or disabled to automatically manage discovered resources across the site, run the following command:
```
ksysmgr -a vm_auto_discovery query system
```
An output that is similar to the following example is displayed:
```
System-Wide Persistent Attributes
  vm_auto_discovery: enable
```
To enable the VM auto-discovery property, run the following command:
```
ksysmgr modify system vm_auto_discovery=enable
```
To disable the VM auto-discovery property, run the following command:
```
ksysmgr modify system vm_auto_discovery=disable
```

Disk-pair status example

To query status of disk-pairs, run the following command:

ksysmgr query disk_pair
[storage_agent=<storage_agent_name>  |
host_group=<host_group_name> |
vm=<vmname> |
workgroup=<workgroup_name>]
[status=<yes|no>]
[vm-wwpn=<yes|no>]
query => q*, ls, get, sh*
disk_pair => dp, disk_p*

Note: The SVC storage agent, EMC storage agent, and Hitachi storage agent supports the status attribute with the yes option (status=yes). The vm-wwpn attribute and the status attribute are mutually exclusive.

An output that is similar to the following example is displayed:

# ksysmgr query disk_pair vm=<vm_name>
Storage: xivsalocal_XIVXXX (India<-> Storage: xivsaremote_XIVXXY (Ame      VM-WWPN                                    VIOS
===========================================================================================================================================
001738002778XXX                 <-> 00173800277AXXXY                 C0507606D917MMMM,C0507606D917NNNN  d66crimeav1,d66crimeav2
Storage: xivsalocal_XIVXXX (India<->              Storage:  (America)      VM-WWPN                                    VIOS
===========================================================================================================================================
00173800XXXXXXXX                 <->                                  C0507606XXXXXXXX,C0507606XXXXXXXX  d66crimeav1,d66crimeav2        (UNMANAGED)

This default output displays a maximum of two VM-WWPNs. To view all available VM-WWPNs and VIOS details, run the vm-wwpn=yes command with the vm filter.

For example:

# ksysmgr query disk_pair vm=d66crimea001 vm-wwpn=yes

Storage       : xivsalocal_XIVXXX (India)<->xivsaremote_XIVXXY (America)
Disk Pair     : 001738002XXXXXXXX<->00173800XXXXXXXY
[VM-WWPN:VIOS]: [C0507606XXXXXXXX:d66crimeav1],[C0507606XXXXXXXY:d66crimeav2]

Storage       : xivsalocal_XIVXXX (India)<-> (America)
Disk Pair     : 00173800XXXXXXXX<->    (UNMANAGED)
[VM-WWPN:VIOS]: [C0507606DXXXXXXXX:d66crimeav1],[C0507606XXXXXXXX:d66crimeav2]

To query status of disk pairs in the SVC storage, run the following command:
```
ksysmgr query disk_pair status=yes
```

Disk group status example

To query status of a specific disk group, run the following command:
```
ksysmgr query disk_group disk_group_name status=yes
```
To query status of all disk groups, run the following command:
```
ksysmgr query disk_group status=yes
```
An output that is similar to the following example is displayed:
```
Name: VMRDG_DR_Site1 
Site: Site1 
Hosts: <Hosts> 
Storages: saremote_emc 
CG: CG1 
State: SYNC (saremote_emc) 
Name: VMRDG_DR_Site2 
Site: Site2 
Hosts: <Hosts> 
Storages: salocal_emc 
CG: CG2 
State: SYNC (salocal_emc)
```
Note: The SVC storage agent, EMC storage agent, Hitachi storage agent, and EMC Unity storage agent supports the status attribute with the yes option, that is status=yes.

Unmanage disk example

To unmanage a disk, run the following command:
```
ksysmgr unmanage disk diskid=<diskid1[,...]>
```
To manage an unmanaged disk, run the following command:
```
ksysmgr manage disk diskid=<diskid1[,...]>
```
To view all unmanaged disks, run any of the following commands:
- ```
ksysmgr query disk_pair
```
- ```
ksysmgr query disk state=<unmanaged>
```

Viewing disk details

To view the details of a disk, run the following command:
```
ksysmgr query diskview
      vm=<vm_name> | wwpn=<wwpn1[,wwpn2,...]
      targethost=<hostname>
      [vios=<target_vios_name>]
```
Note:
- Enter the name of the virtual machine or WWPN along with the target hostname.
- Virtual I/O Sever (VIOS) name is optional. If you do not specify a VIOS name, the command retrieves disks from all Virtual I/O Severs of the target host.
- Current host of the virtual machine and the target host must not be the same.
- The diskview feature is supported only for VIOS 3.1.4.00, or later.
- For a KSYS cluster with shared replication type, specify only WWPN in the query diskview command.

Custom consistency group (CG) name

Custom consistency group (CG) name for EMC storage subsystem
- To create a host group with custom consistency group name at the home site and at the backup site, run the following command:
```
ksysmgr add host_group hg_name
          [site=<sitename>] hosts=host1,[host2,...] | file=filepath
        [mxn_pairing=<yes>] [workgroup_support=<no>][customcgname=<CgName>]
        [bsitecgname=<CgName>]
```
- To modify a host group with custom consistency group name at the home site and at the backup site, run the following command:
```
ksysmgr modify host_group <host_group_name> options customcgname=<CgName> bsitecgname=<CgName>
```
- To view the details of a custom consistency group name, run the following command:
```
ksysmgr query hg <host_group_name>
```
- To create a workgroup with custom consistency group name at the home site and at the backup site, run the following command:
```
ksysmgr add workgroup <name> vm=<vmname1[,vmname2...]> [customcgname=<CgName>] [bsitecgname=<CgName>]
```
- To modify a workgroup with custom consistency group name at the home site and at the backup site, run the following command:
```
ksysmgr modify workgroup <name>
       [policy=<add | delete> vm=<vmname1|vmuuid1[,vmname2|vmuuid2...]>]
        [customcgname=<CgName>]
        [bsitecgname=<CgName>] 
```
Custom consistency group (CG) name for IBM XIV, IBM SAN Volume Controller (SVC) and Hitachi storage systems:
- To create a host group with custom consistency group name, run the following command:
```
ksysmgr add host_group <host_group_name> hosts=<host1[,host2,...]> [customcgname=<CgName>] 
```
- To modify a host group with custom consistency group name, run the following command:
```
ksysmgr modify host_group <name> options [customcgname=<CgName>]
```
- To create a workgroup with custom consistency group name, run the following command:
```
ksysmgr add workgroup <name> vm=<vmname1[,vmname2...]> [customcgname=<CgName>]
```
- To modify a workgroup with custom consistency group name, run the following command:
```
ksysmgr modify workgroup <name>
      { [policy=<add | delete> vm=<vmname1|vmuuid1[,vmname2|vmuuid2...]>]
        [customcgname=<CgName>] }
```
- To view the details of a custom consistency group name in a host group, run the following command:
```
ksysmgr query hg [<host_group_name>]
```
- To view the details of a custom consistency group name in a workgroup, run the following command:
```
ksysmgr query wg [<WGName>]
```

Setting up the Single-Root I/O Virtualization (SR-IOV) override flag status

To set up the Single-Root I/O Virtualization (SR-IOV) override flag status for a host group, run the following command:
```
ksysmgr modify host_group <HGNAME> options sriov_override=yes/no
```
To set up the Single-Root I/O Virtualization (SR-IOV) override flag status for a site, run the following command:
```
ksysmgr modify site <SITENAME> sriov_override=yes/no
```

Configuring HADR

To configure the HADR cluster type, specify HADR while configuring the cluster. For example:
```
ksysmgr add ksyscluster cluster_name ksysnodes=ksysnode1 type=HADR
```

To create an HADR cluster, run the following command:

ksysmgr add ksyscluster <HADR_CLUSTER_NAME> type=HADR ksysnodes=<hostname> sync=yes

For symmetric host group, to pair an identified host (for example, host name: Host1, Host3) in the active site to the identified host (for example, host name: Host2, Host4) in the backup site, run the following commands:
```
ksysmgr pair host Host1 pair=Host2
```
```
ksysmgr pair host Host3 pair=Host4
```
To create a symmetric host group ( for example host group name : Host_group1, ha_disk : SSP_disk1 , repo_disk :SSP_disk2), run the following command:
```
ksysmgr add host_group Host_group1 hosts=Host1,Host3 ha_disk=SSP_disk1 repo_disk=SSP_disk2
```
To create an asymmetric host group (for example, host group name: Host_group1, HA disk: SSP_disk1, repo_disk : SSP_disk2) and to add the existing hosts (for example, hosts: Host1, Host2, Host3) to the asymmetric host group, run the following command:
```
ksysmgr add host_group Host_group1 hosts=Host1,Host2,Host3 mxn_pairing=yes ha_disk=SSP_disk1 repo_disk=SSP_disk2
```
To perform the discovery operation and the verify operation at the site level, run the following command:
```
ksysmgr discover site Austin verify=yes
```
Pair the identified host (for example, host name: Host1, Host3) in the active site to the identified host (for example, host name: Host2, Host4) in the backup site by running the following commands:
```
ksysmgr pair host <host1> pair=<host2>
```
```
ksysmgr pair host <host3> pair=<host4>
```

To set HA monitoring for an entity, run the following command:

ksysmgr modify system|host_group|host|vm name ha_monitor=enable|disable

Configuring HADRHA

To create an HADRHA cluster type, specify HADRHA while configuring the cluster. For example:
```
ksysmgr add ksyscluster HA_DR type=HADRHA ksysnodes=ksysnode1 sync=yes
```

KSYS spooling

To query a KSYS spooling:

ksysmgr q system

An output that is similar to the following example is displayed:

trace_file_size: not set
ksys_spooling: not set
spool_dest_dir: not set
spool_dir_max_size: not set

To enable KSYS spooling:
```
ksysmgr modify system ksys_spooling=enable
```
Note: If you enable the ksys_spooling and the spool_dest_dir value is not set for the KSYS spooling, then the spool_dest_dir will get a default value as /tmp/ksys/rm.

To modify the KSYS spool_dir_max_size:

ksysmgr modify system spool_dir_max_size=<value>

Example:
```
ksysmgr modify system spool_dir_max_size=10240
```
An output that is similar to the following example is displayed:
```
trace_file_size: 25 MB
ksys_spooling: enable
spool_dest_dir: /S1
spool_dir_max_size: 10240 MB
hmc_ping_timer: 0 seconds
```
Note: You will get an error message if the spool_dir_max_size value is greater than 10240 MB (10 GB).

To modify the ksys_spooling, trace_file_size, and spool_dest_dir:

ksysmgr modify system trace_file_size=<value>
ksys_spooling=<enable/disable> spool_dest_dir=<path>

An output that is similar to the following example is displayed:

vlanmap: Not currently set
vswitchmap: Not currently set
drvlanmap: Not currently set
drvswitchmap: Not currently set
trace_file_size: 30 MB
ksys_spooling: enable
spool_dest_dir: /S1

Policy-based replication

Set replication type and specify sites: To initiate policy-based replication, you need to set the replication type to policy-based and specify the sites that are involved in the disaster recovery setup. This command identifies the primary and secondary sites for replication.Run the following command:
```
ksysmgr modify system replication_type=policybased sites=SiteA,SiteB
```
The following example output is displayed:
```
# date;ksysmgr modify system replication_type=policybased sites=Austin,India
Wed Oct 9 08:36:23 CDT 2024 KSYS replication_type has been updated between Austin and India
```
Define the replication policy name: This step assigns a specific replication policy to the system. The policy name defines the replication rules and settings that are applied consistently across the specified sites.
Note: You must manually update the policy name in KSYS and ensure that the policy exists in the storage system. It is required that you create the policy at the storage level and then assign it to the system, host group (HG), and workgroup (WG) in VMRM that is based on your requirements.

Note: Ensure that the storage pool mapping is completed before you initiate the replication configuration.
Run the following command:
```
ksysmgr modify system replicationpolicyname=<replication policy name>
```
The following example output is displayed:
```
# date;ksysmgr modify system replicationpolicyname=test_system_policy
Wed Oct 9 08:36:51 CDT 2024 KSYS ReplicationPolicyName has been updated
```
Apply the replication policy to a Host Group: Assigning the replication policy to a host group allows all resources within that group to adhere to the specified replication rules. This ensures that the replication policy is enforced uniformly across the host group. Run the following command:
```
ksysmgr modify host_group <host group name> options replicationpolicyname=<replication policy name>
```
The following example output is displayed:
```
# date;ksysmgr modify host_group HG1 options replicationpolicyname=test_hg_policy
Thu Oct 10 01:02:32 CDT 2024 For Host Group HG1 attribute(s) 'replicationpolicyname' was successfully modified.
```
Assign the replication policy to a Workgroup: Similar to the host group, applying the replication policy to a workgroup ensures that all resources within the workgroup follow the designated replication settings. This adds an extra layer of organization and control for disaster recovery.Run the following command:
```
ksysmgr modify work_group <work group name> options replicationpolicyname=<replication policy name>
```
The following example output is displayed:
```
# ksysmgr modify work_group wg1 replicationpolicyname=test_wg_policy
For Workgroup wg1 attribute(s) 'replicationpolicyname' was successfully modified.
```

Note:

Policy based replication can be set at both the HG and WG levels using different policies.

If a policy-based replication name should be added at WG level, one needs to update the policy name as mentioned above before the first discovery of the workgroup.

Policy based replication name cannot be modified after the first discovery of the Workgroup.
During cluster removal or workgroup modification, hosts that are created on target storage by ksys discovery might not be automatically removed. Manual cleanup might be required to ensure accurate configuration.
The replication policy name is validated with the name in the storage module (level) during discovery.

vPMEM override

To enable the vPMEM flag for overriding physical memory. For example:
```
ksysmgr modify sys vpmem_override=enable
```
To disable the vPMEM flag for overriding physical memory. For example:
```
ksysmgr modify sys vpmem_override=disable
```

Note: By default, the value of vPMEM variable is disabled.

Note: You can use the vPMEM override in HMC 1050 and the firmware P10, or later, versions.

RDF group auto update

To check system attribute for storage_group_update, run the following command:
```
ksysmgr query system | grep storage_group_update
```
An output that is similar to the following example is displayed:
```
 storage_group_update: no
```
Note: By default, the storage_group_update attribute is set to no.
To enable or disable the RDF group auto update feature, run the following command:
```
ksysmgr modify system storage_group_update=<yes|no>
```
An output that is similar to the following example is displayed:
```
ksysmgr modify system storage_group_update=yes
 KSYS storage_group_update has been updated
```