Chris's AIX Blog

LPM and viosecure

cggibbo Tags: lpm ipsec firewall vios hscla230 2 Comments 8,652 Views

I came across this strange LPM issue recently. Thought I’d share it with you.

All the customers VIOS were configured with the viosecure level set to high.

When the VIOS is configured with a security profile set to high, PCI or DoD, a new feature is enabled during LPM. This new feature is called “Secure LPM”. As a result when you initiate an LPM operation, the “Secure LPM” feature automatically enables the VIOS firewall and configures a secure (ipsec) tunnel for all LPM traffic over the network.

Before we started LPM, the VIOS firewall is off:

$ viosecure -firewall view

IPv4 Firewall OFF

ALLOWED PORTS

Local Remote

Interface Port Port Service IPAddress Expiration

Time(seconds)

--------- ---- ---- ------- --------- ---------------

And it's activated after we start LPM:

$ viosecure -firewall view

IPv4 Firewall ON

ALLOWED PORTS

Local Remote

Interface Port Port Service IPAddress Expiration

Time(seconds)

--------- ---- ---- ------- --------- ---------------

In the customer’s case, we moved a partition from a 795 running a pair of VIOS at 2.2.1.3 to another 795 running another pair of VIOS at 2.2.2.1. This worked OK. When we attempted to move the partition back, we received an error stating that the source MSP had rejected the request.

HSCLA230

The mover service partition on the source managed system has rejected the request to stop the migration. Verify that the migration state of the partition is Migration Starting, and try the operation again.

After a while we discovered that when the LPM operation started, the source VIOS would enable its firewall and we would immediately loose connectivity (i.e. our SSH session to the VIOS would hang). We also found that RMC connectivity between the source VIOS and HMC also dropped. Somehow the VIOS firewall was blocking network connectivity to pretty much everything.

Eventually, after much digging we found that the source VIOS was the victim of an errant firewall rule (configured on the source VIOS itself).

# smitty ips4_conf_filter

List IP Security Filter Rules

1 permit 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 no udp eq 4001 eq 4001 both both no all packets 0 all 0 none  Default Rule

2 *** Dynamic filter placement rule for IKE tunnels *** no

3 deny 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 y all any 0 any 0 all both inbound no yes 0 no 0 patt_none

0 permit 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 yes all any 0 any 0 both both no all packets 0 all 0 none  Default Rule

We removed this rule (after some discussion with IBM support).

# smitty ips4_conf_filter

Delete IP Security Filter Rules

3 deny 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 y all any 0 any 0 all both inbound no yes 0 no 0 patt_none

# smitty ips4_conf_filter

List IP Security Filter Rules

1 permit 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 no udp eq 4001 eq 4001 both both no all packets 0 all 0 none  Default Rule

2 *** Dynamic filter placement rule for IKE tunnels *** no

0 permit 0.0.0.0 0.0.0.0 0.0.0.0 0.0.0.0 yes all any 0 any 0 both both no all packets 0 all 0 none  Default Rule

We were then able to move the partition back to the original frame. We are still searching for an answer to how this happened. The IBM support team believe the rule must have been created by an administrator at some point. Of course, the administrator claims that is untrue.

Modified by cggibbo

LPM ifix for JS22 blades and IVM.

cggibbo 3,943 Views

If you come across the following LPM issue with IVM and JS22 blades,

IZ63357: LPM HUNG AT 100% ON BOTH SOURCE AND DESTINATION IVMS

Error description

The problem that these customers are seeing is the same

As described in this particular defect.

Migrating mobile partition from one IVM to another:

The destination IVM GUI shows the migration is complete,

but the percent complete is "unknown".

The source IVM GUI shows the "migration status" as

Migration Complete and "percent Complete" as 100%.

Once the recover operation is performed, the status shows

that the LPAR is now "Running" on the target and removed

from the source, as expected.

There are two PMRs with the same issue and the

Environment is the same on both the cases:

* Firmware level FSP: EA340_075

* IVM VIOS Level: 2.1.1.10-FP21

* Machine Type: JS22 blade

Local fix

Run the "Recover" operation as the Migration appears to

be "hung". Once the recover is complete, the status shows

that the LPAR is now "Running" on the target and removed

from the source, as expected.

Then I recommend you request the following ifix from IBM support:

IZ63357F21.091110.epkg.Z

From IBM support:

“It appears that this problem is due to a timing issue.

The trace data shows that IVM is calling the migmover -m end command on the destination partition. That command in turn calls a system call. When everything is in the correct state, that system call will end the migration and cleanup. The problem is it is not being called because the mover has not yet been notified that VASI is in the END state as indicated by the mover's state being: MIG_STATE_COMPLETED. The trace shows this notification arriving a split second after the migmover -m end command was issued.

On a good migration, the mover is notified by VASI of the state change before IVM calls migmover -m. IVM runs migmover -m end once it gets a migration completed message from phyp. If this message is issued about the same time as the message that tells VASI that
the migration has ended, a timing issue seems a real possibility.

The problem was fixed in the VIOS code. A new call was added when the "end" state change comes in from IVM/HMC. If the state is complete we are good to go, otherwise we issue an error.”

Installing this fix on my VIOS resolved the problem. My VIOS are running 2.1.1.10-FP21. If you are running FP22 for VIOS 2.1, then you must request a different ifix from IBM support.

lsdevinfo

cggibbo Tags: chris aix gibson lsdevinfo 11,447 Views

The lsdevinfo utility appears to an undocumented command. However it looks like it could come in handy occasionally. Here are some examples of using it on AIX 7.1:

# oslevel -s

7100-00-01-1037

# lslpp -w /usr/sbin/lsdevinfo

File Fileset Type

----------------------------------------------------------------------------

/usr/sbin/lsdevinfo bos.rte.methods File

# lsdevinfo -h

Usage:

lsdevinfo [-q Criteria] [-F Format] [-R] [-c] [-h]

# lsdevinfo | more

device:

name = "sys0"

status = "1"

chgstatus = "1"

ddins = "planar_pal_chrp"

aixloc = ""

uniquetype = "sys/node/chrp"

class = "sys"

subclass = "node"

type = "chrp"

prefix = "sys"

devid = "chrp"

base = "1"

hasvpd = "0"

detectable = "0"

busext = "0"

fru = "1"

led = "2065"

catalog = "devices.cat"

setno = "1"

msgno = "1"

driver = "planar_pal_chrp"

define = "/usr/lib/methods/define_rspc -n"

configure = "/usr/lib/methods/cfgsys_chrp"

change = "/usr/lib/methods/chggen"

unconfigure = "/usr/lib/methods/ucfgdevice"

undefine = "/usr/lib/methods/undefine"

start = ""

stop = ""

inventory = "0"

syscons = "/dev/vty0"

sysconsp = ""

modelcode = "0x0800004c"

keylock = "normal"

maxbuf = "20"

maxmbuf = "0"

maxuproc = "128"

autorestart = "true"

iostat = "false"

realmem = "6291456"

conslogin = "enable"

fwversion = "IBM,AM710_058"

maxpout = "8193"

minpout = "4096"

fullcore = "false"

pre430core = "false"

ncargs = "256"

pre520tune = "disable"

max_logname = "9"

platform_desc = "chrp"

rtasversion = "1"

modelname = "IBM,9179-MHB"

........

# lsdevinfo | grep -p hdisk0

device:

name = "hdisk0"

status = "1"

chgstatus = "2"

ddins = "scsidisk"

aixloc = ""

uniquetype = "disk/vscsi/vdisk"

class = "disk"

subclass = "vscsi"

type = "vdisk"

prefix = "hdisk"

devid = ""

base = "1"

hasvpd = "0"

detectable = "1"

busext = "0"

fru = "0"

led = "9652"

catalog = "vscsi_init.cat"

setno = "2"

msgno = "0"

driver = "scsidisk"

define = "/etc/methods/define"

configure = "/etc/methods/cfgscsidisk"

change = "/etc/methods/chgdisk"

unconfigure = "/etc/methods/ucfgdevice"

undefine = "/etc/methods/undefine"

start = ""

stop = ""

inventory = "0"

pv = "no"

pvid = "00f6050a2cd79ef80000000000000000"

model_name = "VCS_DEFLT_DISK "

queue_depth = "3"

max_transfer = "0x40000"

cfgmgr_psafe = ""

unique_id = "48333321360050768019183002800000000000F1304214503IBMfcp05VDASD03AIXvscsi"

sn_location = "609F"

reserve_policy = "no_reserve"

PCM = "PCM/friend/vscsi"

safe_relocate = "0"

kext_errsave = "yes"

message_no = "1"

# lsdevinfo | grep -p cluster0

device:

name = "cluster0"

status = "1"

chgstatus = "1"

ddins = ""

aixloc = ""

uniquetype = "pseudo/node/cluster"

class = "pseudo"

subclass = "node"

type = "cluster"

prefix = "cluster"

devid = ""

base = "0"

hasvpd = "0"

detectable = "0"

busext = "0"

fru = "0"

led = "8232"

catalog = "devices.cat"

setno = "170"

msgno = "1"

driver = ""

define = "/usr/lib/methods/define -n"

configure = "/usr/lib/methods/cfgcluster"

change = "/usr/lib/methods/chgcluster"

unconfigure = "/usr/lib/methods/ucfgdevice"

undefine = "/usr/lib/methods/undefine"

start = ""

stop = ""

inventory = "0"

clvdisk = "60050768-0191-8300-2800-000000000f15"

node_uuid = "60510e58-be33-11e0-8387-66da93d16b18"

# lsdevinfo -c

name="sys0",status="1",chgstatus="1",ddins="planar_pal_chrp",aixloc="",uniquetype="sys/node/chrp",class="sys",subclass="node",type="chrp",prefix="sys",devid="chrp",base="1",hasvpd="0",detectable="0",busext="0",fru="1",led="2065",catalog="devices.cat",setno="1",msgno="1",driver="planar_pal_chrp",define="/usr/lib/methods/define_rspc -n",configure="/usr/lib/methods/cfgsys_chrp",change="/usr/lib/methods/chggen",unconfigure="/usr/lib/methods/ucfgdevice",undefine="/usr/lib/methods/undefine",start="",stop="",inventory="0",syscons="/dev/vty0",sysconsp="",modelcode="0x0800004c",keylock="normal",maxbuf="20",maxmbuf="0",maxuproc="128",autorestart="true",iostat="false",realmem="6291456",conslogin="enable",fwversion="IBM,AM710_058",maxpout="8193",minpout="4096",fullcore="false",pre430core="false",ncargs="256",pre520tune="disable",max_logname="9",platform_desc="chrp",rtasversion="1",modelname="IBM,9179-MHB",systemid="IBM,02100488P",boottype="disk",dasd_spin_delay="5",SW_dist_intr="false",pci_full_cfg="1",cpuguard="enable",frequency="6400000000",capacity_inc="0.01",capped="false",dedicated="false",ent_capacity="0.50",min_capacity="0.10",max_capacity="0.50",variable_weight="128",cod_cust_info="",cod_mail="",cod_modelsys="",cod_proc="0",id_to_system="0X800000302B900000",id_to_partition="0X800000302B900043",ngroups_allowed="128",sed_config="select",nfs4_acl_compat="secure",enhanced_RBAC="true",log_pg_dealloc="true",ghostdev="0"

........................

# lsdevinfo -q name="cluster0"

device:

name = "cluster0"

status = "1"

chgstatus = "1"

ddins = ""

aixloc = ""

uniquetype = "pseudo/node/cluster"

class = "pseudo"

subclass = "node"

type = "cluster"

prefix = "cluster"

devid = ""

base = "0"

hasvpd = "0"

detectable = "0"

busext = "0"

fru = "0"

led = "8232"

catalog = "devices.cat"

setno = "170"

msgno = "1"

driver = ""

define = "/usr/lib/methods/define -n"

configure = "/usr/lib/methods/cfgcluster"

change = "/usr/lib/methods/chgcluster"

unconfigure = "/usr/lib/methods/ucfgdevice"

undefine = "/usr/lib/methods/undefine"

start = ""

stop = ""

inventory = "0"

clvdisk = "60050768-0191-8300-2800-000000000f15"

node_uuid = "60510e58-be33-11e0-8387-66da93d16b18"

Make sure you monitor file system space on your VIOS.

cggibbo Tags: root_filesystem_is_full this_may_or_may_not_indic... vios aix mkvdev cli_debug=33 powervm 0514-548 chris_gibson the_command's_response_wa... 5 Comments 22,853 Views

Make sure you are monitoring file system space on your VIOS.

Why?

If you run out of space in the root file system, odd things can happen when you try to map virtual devices to virtual adapters with mkvdev.

For example, a colleague of mine was attempting to map a new hdisk to a vhost adapter on a pair of VIOS. The VIOS was running a recent version of code. He received the following error message (see below). It wasn’t a very helpful message. At first I thought it was due to the fact that he had not set the reserve_policy attribute for the new disk to no_reserve on both VIOS. Changing the value for that attribute did not help.

$ ioslevel

2.2.1.3

$ mkvdev -vdev hdisk1 -vadapter vhost0 -dev vhdisk1

*******************************************************************************

The command's response was not recognized. This may or may not indicate a problem.

*******************************************************************************

I enabled debug output from the padmin command line and discovered that the mkdev command was failing for some reason.

$ export CLI_DEBUG=33

$ mkvdev -vdev hdisk1 -vadapter vhost0 -dev vhdisk1

AIX: "lspv -l hdisk1 2>&1 | grep 0516-320"

AIX: "mkdev -V hdisk1 -p vhost0 -l vhdisk1 "

*******************************************************************************

The command's response was not recognized. This may or may not indicate a problem.

*******************************************************************************

I entered oem_setup_env and ran the mkdev command manually. I was presented with a very clear error message describing the problem. The root file system (/) was full!

$ oem_setup_env

# mkdev -V hdisk1 -p vhost0 -l vhdisk1

mkdev: 0514-548 Cannot perform the requested function because

the / filesystem is full or is out of inodes.

# df -g /

Filesystem GB blocks Free %Used Iused %Iused Mounted on

/dev/hd4 0.50 0.00 100% 7728 75% /

I found a 45MB core file (generated by cimserver) in the root file system. I moved the file to another file system (with plenty of free space) for further investigation.

# mv core /tmp/core.cg

# df -m .

Filesystem MB blocks Free %Used Iused %Iused Mounted on

/dev/hd4 512.00 45.93 92% 7727 38% /

Then I ran the mkvdev command successfully.

$ mkvdev -vdev hdisk1 -vadapter vhost0 -dev vhdisk1

AIX: "lspv -l hdisk1 2>&1 | grep 0516-320"

AIX: "mkdev -V hdisk1 -p vhost0 -l vhdisk1 "

vhdisk1 Available

AIX: "lsattr -El hdisk1 -a max_transfer |cut -d" " -f2"

$ lsmap -all

AIX: "lsdev -c adapter -t IBM,v-scsi-host -s vdevice -F "name" | wc -l -c"

AIX: "lsdev -c adapter -t IBM,v-scsi-host -s vdevice -F "name""

AIX: "lsdev -C -l vhost0 -F "physloc""

AIX: "lsdev -p vhost0 -F "name" | wc -l -c"

AIX: "lsdev -p vhost0 -F "name""

SVSA Physloc Client Partition ID

--------------- -------------------------------------------- ------------------

vhost0 U9119.FHA.87654B1-V6-C21 0x0000002e

VTD vhdisk1

Status Available

LUN 0x8100000000000000

Backing device hdisk1

Physloc U5873.001.9SS005Y-P1-C9-T1-W50060E8006D0206A-L1000000000000

Mirrored false

AIX: "lsdev -C -l vhost1 -F "physloc""

AIX: "lsdev -p vhost1 -F "name" | wc -l -c"

AIX: "lsdev -p vhost1 -F "name""

SVSA Physloc Client Partition ID

--------------- -------------------------------------------- ------------------

vhost1 U9119.FHA.87654B1-V6-C22 0x0000002g

VTD NO VIRTUAL TARGET DEVICE FOUND

I found the same issue on the second VIOS i.e. a full root file system due to a core file (from cimserver). I also found no trace of a full file system event in the error report. Perhaps someone had taken it upon themselves to “clean house” at some point and had removed entries from the VIOS error log.

Make sure you monitor file system space on your VIOS. Who knows what else might fail if you run out of space in a critical file system.

Managing an existing AIX virtual machine with PowerVC

cggibbo Tags: powervc virtual aix machines managing existing 6,785 Views

It’s possible, with PowerVC, to manage existing AIX and Linux partitions in your environment. This may be useful for customers that have recently installed PowerVC in their environment and have existing AIX partitions. The question that is often asked is “Can I manage these existing partitions with PowerVC?”. These existing partitions were deployed outside of PowerVC, often long before PowerVC was even available as product from IBM. Fortunately the answer to this question is most likely yes!

Before you add an existing virtual machine to PowerVC, ensure that the following requirements are met:

http://www-01.ibm.com/support/knowledgecenter/SSXK2N_1.2.1/com.ibm.powervc.standard.help.doc/powervc_addingpre-existingvms_hmc.html

Here’s an example of managing an existing partition with PowerVC. Click on the Hosts icon from the PowerVC home screen then click on the “Manage Existing Virtual Machines” icon.

You are presented with a drop-down list of all the (Power Systems) Hosts known to PowerVC. Select the appropriate host where the existing partition currently resides.

Next you’ll be asked if you want to manage any supported virtual machines that are not currently being managed by PowerVC. “Supported” (with PowerVC Standard Edition v1.2.1) means any virtual machine that is either using virtual fibre channel adapters attached to supported SAN and storage devices (typically brocade switches and v7000) and/or virtual SCSI adapters attached to disk in a VIOS shared storage pool, that is currently managed by PowerVC.

Or you can select a specific virtual machine (or machines) to manage. I’ve selected one specific partition I wish to manage with PowerVC. It is an AIX partition that is connected to a shared storage pool (SSP). The SSP is already managed by PowerVC but the AIX partition is not.

A list of virtual machines (partitions) will be displayed next. Select the partition you’d like to manage with PowerVC and click on Manage.

Under the Virtual Machines view you’ll see the partition that you selected. It will appear in an Active and Pending state.

After a minute or two, the partition will change to an Active and OK state. The virtual machine is now under the management of PowerVC. You can now stop, start, restart, resize, migrate, attach volumes to, capture and/or delete the virtual machine through PowerVC.

Just as a point of interest, you can also access the Manage Existing option from the Virtual Machines view in PowerVC.

If you need to remove a partition (virtual machine) from PowerVC management (without deleting the partition) you can do so using the following procedure. From the Hosts view, select the desired Host and under Virtual Machines, select the partition you want to un-manage.

PowerVC will prompt you to confirm the removal of the virtual machine from PowerVC management. This does not delete the virtual machine; it will continue to run without disruption.

The virtual machine is removed from PowerVC management.

Managing users with dsh and chpasswd - (plus info on DSM and dping).

cggibbo Tags: run if remote exit installed pwdadm a accepted. code expired dping csm has not been you aix have license had dsh shell -l new release admchg 255 csmconfig 2651-095 or chpasswd 1 Comment 20,876 Views

Starting with AIX 7.1, CSM is no longer supported or available. It has been replaced by Distributed Systems Managment (DSM). Section 5.2 of the IBM AIX 7.1 Differences Guide Redbook provides details of the new DSM capabilities.

Fortunately DSM still provides access to the dsh command. I’ve written about how I’ve used this utility in the past. The new dsh command (and other tools) are provided in the new DSM filesets named dsm.core and dsm.dsh.

root@nim# lslpp -L dsm*

Fileset Level State Type Description (Uninstaller)

----------------------------------------------------------------------------

dsm.core 7.1.1.2 A F Distributed Systems Management

Core

dsm.dsh 7.1.0.0 C F Distributed Systems Management

Dsh

root@nim# lslpp -w /usr/bin/dsh

File Fileset Type

----------------------------------------------------------------------------

/usr/bin/dsh dsm.dsh Symlink

These filesets are NOT installed by default. You must manually install them. They can be found on your AIX 7.1 media.

If dsh is something you use, then I recommend you read the section on DSM in the Redbook. Also take a look at section 5.2.7 Using DSM and NIM, in which it describes how you can integrate DSM and NIM and completely automate the installation of AIX:

“The AIX Network Installation Manager (NIM) has been enhanced to work with the Distributed System Management (DSM) commands. This integration enables the automatic installation of new AIX systems that are either currently powered on or off.”

Although I’ve written about the dsh command before, there’s one usage I’ve not covered. And that is using dsh to manage users across a group of LPARs. In particular, changing a user’s password.

Before I go any further, I should state that for the following to work you must first configure ssh keys on your NIM master (or central mgmt AIX system) so that you can communicate with all of your AIX systems via SSH, as root, without being prompted for a password. Read my article on dsh to find out how to do this if necessary.

In the following example, I use dsh from my NIM master. It is my central point of control for my AIX environment.

My ssh keys for root on my NIM master have been generated and distributed to all of my LPARs.

root@nim# ssh-keygen -d

Generating public/private dsa key pair.

Enter file in which to save the key (/.ssh/id_dsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /.ssh/id_dsa.

Your public key has been saved in /.ssh/id_dsa.pub.

The key fingerprint is:

ed:18:e9:00:37:13:7c:7c:74:6a:a9:e0:ad:c0:09:a9 root@nim

The key's randomart image is:

+--[ DSA 1024]----+

| ... .. . |

| . ..o .+ |

| o . =. .+ |

| . o = = = |

|E + o S . |

| . + + |

| . o . |

| |

+-----------------+

root@nim# ls -ltra

total 40

-rw------- 1 root system 214 17 Sep 2010 authorized_keys

drwxr-xr-x 7 root system 4096 16 Nov 11:43 ..

-rw-r--r-- 1 root system 3615 16 Nov 12:04 known_hosts

-rw-r--r-- 1 root system 601 16 Nov 12:06 id_dsa.pub

-rw------- 1 root system 672 16 Nov 12:06 id_dsa

drwx------ 2 root system 256 16 Nov 12:06 .

On my AIX LPARs, the authorized_keys file has been updated with the public ssh key from my NIM master:

root@aixlpar1 [/.ssh]# chmod 600 known_hosts authorized_keys

root@aixlpar1 [/.ssh]# ls -ltr

total 0

-rw------- 1 root system 406 Sep 29 15:30 known_hosts

-rw------- 1 root system 601 Nov 16 12:07 authorized_keys

root@aixlpar1 [/.ssh]# cat authorized_keys

ssh-dss AAAAB3NzaC1kc3MAAACBAPJpekVQgV2CTu5tCOsukwXwMrkfBabLAN7vwjgPk3S2p65COJkFwaDRWwBoK87cnMrrc7/GuOLdk/RiqYw1K8Z0d9RWnsymYrNm0MBALWioGGXwQ+YlLiYHIQsSArnajZOH6Ye36qz81uWNoj6u4xIxi/iGSYoU4x3UE/c/H833AAAAFQDKEtWVFk3G+KAyxUsI6FcpbJZqDQAAAIAVUVRak/s+X0mMElCaEs8Rh96GDfhorXWV1YzI0nbRo8QX2AMix+3LSdEmYCRn8TYpk1Udln3GG4gzy5oI/3Rd1RTWgAo+067LwytR6IJfwYt219Pwh2ktrm5iZu3c+yDBqQj6hsga2YIun5lxqYFy39CZNtslnR1wTx9uCcYkowAAAIEAuVFTsW9ugVM76CgluspNo0faz9xFo4DCOjCgye+k/ZqSDaNq6Xrvxktw+dkHuKHbhvofPgeiMrlqhdbrE0ovLUr8aadxArRhitnu10CFYbRwlrjAxyAaRwjD2pXG8M3a5D1XXAWDCDm6bEs5yxJqcl1wq+8UAhpKiS5yj2pwApk= root@nim

On the NIM master, the root user was configured for the DSH environment. The following entry was placed in roots .profile:

root@nim# cat /.profile

ENV=$HOME/.kshrc

The following entry was placed in roots .kshrc file:

root@nim# cat /.kshrc

export DSH_NODE_RSH=/usr/bin/ssh

export DSH_NODE_LIST=/usr/local/etc/nodes

A /usr/local/etc/nodes file was created on the NIM master. This file contains a list of each of the nodes that dsh can communicate with from NIM:

root@nim# cat /usr/local/etc/nodes

aixlpar1

aixlpar2

aixlpar3

aixlpar4

aixlpar5

aixlpar6

aixlpar7

aixlpar8

aixlpar9

aixlpar10

aixlpar11

The first time that the dsh command is run against a new host, the following message will be displayed. dsh uses the FQDN, and the FQDN needs to be added to the known_hosts file for ssh. Therefore you must make an ssh connection first with FQDN to the host:

root@nim# dsh uptime

aixlpar1.cg.com.au : Host key verification failed.

dsh: 2617-009 aixlpar1.cg.com.au remote shell had exit code 255

It is necessary to ssh directly to each node using its FQDN. This step is only required once for each node. For example:

root@nim# ssh aixlpar1.cg.com.au

The authenticity of host 'aixlpar1.cg.com.au (172.1.6.17)' can't be established.

RSA key fingerprint is 83:13:1d:cb:97:ee:00:7a:04:75:62:98:bd:e8:30:79.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'aixlpar1.cg.com.au ' (RSA) to the list of known hosts.

Last unsuccessful login: Tue Sep 27 03:04:33 GMT+10:00 2011 on ssh from 172.1.6.15

Last login: Wed Nov 16 12:08:08 GMT+11:00 2011 on /dev/pts/3 from nim

root@aixlpar1 [/root]# exit

Connection to aixlpar1.cg.com.au closed.

Now we can use the dsh command to run a command across all the nodes defined in the DSH_NODE_LIST. For example, the following command will run the uptime command on all nodes:

root@nim# dsh uptime

aixlpar1.cg.com.au : 12:14PM up 10 days, 22:32, 3 users, load average: 0.47, 0.53, 0.64

aixlpar2.cg.com.au : 12:14PM up 21 days, 21:13, 1 users, load average: 0.11, 0.31, 0.51

aixlpar3.cg.com.au : 12:14PM up 15 days, 20:33, 5 users, load average: 0.17, 0.43, 0.54

...etc...

With dsh now configured, we can use it to perform several admin tasks from a central location. For example, the following commands will do the following:

1. Create a user, named cg, on all nodes.

2. Set cg’s password to abc123 using the chpasswd command.

3. Remove the ADMCHG flag for the user cg. This will stop them from being prompted to change their password on their first login attempt.

First I check that user does not exist on the AIX systems.

root@nim# dsh lsuser cg

aixlpar1: 3004-687 User "cg" does not exist.

aixlpar2: 3004-687 User "cg" does not exist.

aixlpar3: 3004-687 User "cg" does not exist.

aixlpar4: 3004-687 User "cg" does not exist.

aixlpar5: 3004-687 User "cg" does not exist.

...etc...

I create the user with the mkuser command.

root@nim# dsh mkuser cg

I confirm that the user has been created on all the AIX systems, with the lsuser command.

root@nim# dsh lsuser cg

aixlpar1: cg id=204 pgrp=staff groups=staff home=/home/cg shell=/usr/bin/ksh login=true su=true rlogin=true daemon=true admin=false sugroups=ALL admgroups= tpath=nosak ttys=ALL expires=0 auth1=SYSTEM auth2=NONE umask=22 registry=files SYSTEM=compat logintimes= loginretries=0 pwdwarntime=0 account_locked=false minage=0 maxage=0 maxexpired=-1 minalpha=0 minother=0 mindiff=0 maxrepeats=8 minlen=0 histexpire=0 histsize=0 pwdchecks= dictionlist= fsize=-1 cpu=-1 data=262144 stack=65536 core=2097151 rss=65536 nofiles=2000 roles=

aixlpar2: cg id=204 pgrp=staff groups=staff home=/home/cg shell=/usr/bin/ksh login=true su=true rlogin=true daemon=true admin=false sugroups=ALL admgroups= tpath=nosak ttys=ALL expires=0 auth1=SYSTEM auth2=NONE umask=22 registry=files SYSTEM=compat logintimes= loginretries=0 pwdwarntime=0 account_locked=false minage=0 maxage=0 maxexpired=-1 minalpha=0 minother=0 mindiff=0 maxrepeats=8 minlen=0 histexpire=0 histsize=0 pwdchecks= dictionlist= fsize=-1 cpu=-1 data=262144 stack=65536 core=2097151 rss=65536 nofiles=2000 roles=

aixlpar3: cg id=204 pgrp=staff groups=staff home=/home/cg shell=/usr/bin/ksh login=true su=true rlogin=true daemon=true admin=false sugroups=ALL admgroups= tpath=nosak ttys=ALL expires=0 auth1=SYSTEM auth2=NONE umask=22 registry=files SYSTEM=compat logintimes= loginretries=0 pwdwarntime=0 account_locked=false minage=0 maxage=0 maxexpired=-1 minalpha=0 minother=0 mindiff=0 maxrepeats=8 minlen=0 histexpire=0 histsize=0 pwdchecks= dictionlist= fsize=-1 cpu=-1 data=262144 stack=65536 core=2097151 rss=65536 nofiles=2000 roles=

I set the users password to abc123, using the chpasswd utility. I also remove the ADMCHG flag so that the user is not prompted to change their password on their first logon attempt.

root@nim# dsh 'echo cg:abc123 | chpasswd -c'

I confirm that I can logon with the new user with the specified password, on one of the AIX LPARs.

root@nim# ssh cg@aixlpar1

cg@aixlpar1’s password:

Last login: Thu Mar 1 20:05:01 CST 2012 on /dev/pts/1 from aix71

$ id

uid=204(cg) gid=1(staff)

Another nice feature of dsh is the dshbak utility. This utility presents formatted output from the dsh command. For example:

root@nim 520 [/.ssh]# dsh errpt | dshbak

HOST: aixlpar1.cg.com.au

------------------------------------

IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION

AA8AB241 1116110811 T O OPERATOR OPERATOR NOTIFICATION

A6DF45AA 1104135011 I O RMCdaemon The daemon is started.

2BFA76F6 1104134111 T S SYSPROC SYSTEM SHUTDOWN BY USER

9DBCFDEE 1104134111 T O errdemon ERROR LOGGING TURNED ON

HOST: aixlpar2.cg.com.au

-------------------------------

IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION

DE9A52D1 1111012611 I S rmt10 AAA1

4865FA9B 1111012211 P H rmt10 TAPE OPERATION ERROR

DE9A52D1 1110233511 I S rmt0 AAA1

4865FA9B 1110225511 P H rmt0 TAPE OPERATION ERROR

DE9A52D1 1109180311 I S rmt0 AAA1

4865FA9B 1109180011 P H rmt0 TAPE OPERATION ERROR

DE9A52D1 1108180411 I S rmt2 AAA1

4865FA9B 1108180211 P H rmt2 TAPE OPERATION ERROR

DE9A52D1 1108165711 I S rmt6 AAA1

4865FA9B 1108165111 P H rmt6 TAPE OPERATION ERROR

A2205861 1102085311 P S SYSPROC Excessive interrupt disablement time

F7FA22C9 1031134111 I O SYSJ2 UNABLE TO ALLOCATE SPACE IN FILE SYSTEM

DE9A52D1 1030163411 I S rmt0 AAA1

4865FA9B 1030163411 P H rmt0 TAPE OPERATION ERROR

....etc....

WARNING: Please be VERY CAREFUL when using the dsh command. Issuing the wrong command can cause damage to all your AIX LPARS!

The dsm.dsh package contains the following utilities:

# lslpp -f dsm.dsh | grep /usr/bin

/usr/bin/dcp -> /opt/ibm/sysmgt/dsm/bin/dcp

/usr/bin/dsh -> /opt/ibm/sysmgt/dsm/bin/dsh

/usr/bin/dping -> /opt/ibm/sysmgt/dsm/bin/dping

/usr/bin/dshbak -> /opt/ibm/sysmgt/dsm/bin/dshbak

If you are a fan of the dping command, you are going to be disappointed. Although the command is currently included in the dsm.dsh fileset, it probably won’t be for much longer.

The command works, “sort of”:

root@nim# dping aixlpar1

aixlpar1: ping (alive)

But if you run ‘dping –a’:

root@nim# dping -a

dping: 2651-095 CSM license has expired or has not been accepted. Run csmconfig -L if you have installed a new release.

According to the developers, dping is no longer supported and will eventually be removed from the DSM package. The response from the developers was as follows:

"The reason "dping -a" is failing with the license check is because the command is calling “/usr/bin/runact-api –c IBM.DmsCtrl::::isLicenseValid" and the license is not set. So the command fails. Since CSM is not supported anymore and went end of life. "

“... please consider the dping command as being "deprecated" code pending removal from the dsm.dsh package.”

When I asked why the command was listed in the AIX 7.1 online documentation if it was no longer available, I was informed: “We are in the process of working with component owner regarding the DOCs and updating them.”. At this stage I’ve not been able to find an alternative command (in AIX). If I find one, I’ll update this post.

If you are planning on migrating to AIX 7.1 please be aware that CSM is no longer supported or available with AIX 7.1. CSM is now ‘end of life’.

http://www.ibmsystemsmag.com/aix/tipstechniques/systemsmanagement/DSH-Simplifies-the-Task-of-Managing-Multiple-Syste/

http://www.redbooks.ibm.com/abstracts/sg247910.html?Open

http://publib.boulder.ibm.com/infocenter/aix/v7r1/topic/com.ibm.aix.cmds/doc/aixcmds1/chpasswd.htm

MDS reports

cggibbo Tags: invscout device report file mup gibson adapter mic power aix firmware microcode mds catalog chris 1 Comment 11,206 Views

I like using the IBM Microcode Discovery Service (MDS) to perform spot checks on POWER system and adapter firmware levels. This tool gives me snapshot of a POWER systems current firmware levels. I can tell almost immediately if I need to update the firmware on the system or any device installed in the server. This report is especially useful when performing a health check on a customer’s system.

The tool is easy to use. First you need to download the latest inventory scout catalog file from the IBM website. The most current microcode catalog can be downloaded here:

ftp://ftp.software.ibm.com/software/server/firmware/catalog.mic

Once downloaded, copy the catalog file to an AIX (or VIOS) partition on the physical server. In most cases, a Virtual I/O Server (VIOS) is the most logical place to copy this file. The VIOS typically owns all the physical adapter devices on the system.

# scp catalog.mic padmin@vio1:

padmin@vio1's password:

catalog.mic 100% 263KB 263.2KB/s 00:00

Next, login to the VIOS and move the new catalog file into the correct location for use with the inventory scout (invscout) tool.

# ssh padmin@vio1

padmin@vio1's password:

$ ls -ltr

total 968

drwxr-xr-x 3 padmin staff 256 May 17 21:20 tivoli

-rw-r--r-- 1 padmin staff 2389 Aug 19 04:43 smit.transaction

-rw-r--r-- 1 padmin staff 1290 Aug 19 04:43 smit.script

-rw-r--r-- 1 padmin staff 11247 Aug 27 11:57 smit.log

drwxr-xr-x 2 padmin staff 4096 Sep 15 15:26 SV810_081

-rw-r--r-- 1 root system 5103 Sep 15 15:53 install.log

drwxr-xr-x 2 padmin staff 256 Sep 15 15:53 fixes

-rw-r--r-- 1 padmin staff 176510 Sep 15 18:34 errlog.out

drwxrwxr-- 2 root staff 256 Sep 16 13:25 config

-rw-r----- 1 padmin staff 269478 Sep 17 20:18 catalog.mic

-rw-r--r-- 1 root staff 5143 Sep 17 20:18 ioscli.log

$ r oem

oem_setup_env

# ls -ltr

total 8

-rw-r--r-- 1 root system 88 May 17 20:26 catalog.mic

# cd /var/adm/invscout/microcode

# mv catalog.mic catalog.mic.old

# cp /home/padmin/catalog.mic .

# ls -ltr

total 536

-rw-r--r-- 1 root system 88 May 17 20:26 catalog.mic.old

-rw-r----- 1 root staff 269478 Sep 17 20:20 catalog.mic

Now all we need to do is run the invscout command to create a new MDS upload file (MUP file).

# invscout

****** Command ---- V2.2.0.20

****** Logic Database V2.2.0.2

Initializing ...

Identifying the system ...

Working ...

Getting system microcode level(s) ...

Scanning for device microcode level(s) ...

117 devices detected; each dot (.)

represents 10 devices processed:

...........

Writing Microcode Survey upload file ...

Microcode Survey complete

The output files can be found at:

Upload file: /var/adm/invscout/vio1.mup

Report file: /var/adm/invscout/invs.mrp

Report file: /var/adm/invscout/invs.mrrup

To transfer the invscout 'Upload file' for microcode

comparison, see your service provider's web page.

The new MUP file (vio1.mup) can be uploaded to the MDS tool at the following URL:

https://www-304.ibm.com/webapp/set2/mds/fetch?page=mdsUpload.html

In the following screenshot I’ve selected the file to be uploaded. Then I clicked on ‘Upload File’.

I’m presented with the MDS report screen.

Immediately my eyes are drawn to the items highlighted in red. Both the PCIe2 4-Port (10GbE SFP+ & 1GbE RJ45) and PCIe3 RAID SAS Adapter have new firmware available for them. I should update the firmware on these devices! But what fixes are included in the new firmware? Well, directly below the devices report is the ‘Microcode by Type’ table.

I can simply click on the ‘Readme’ hyperlink in the report and a new browser window opens with the readme file for the adapters firmware.

So, should I install these fixes right away? How critical are they? Again, the MDS report provides you with a quick way to answer these questions. Under the ‘Severity’ column, you will see several terms that describe the severity of the related fix. If you click on the word ‘Severity’ you’ll be presented with an ‘Understanding the MDS Report’ window.

On this page you’ll find the definition and description of each severity type. This will make it easier for you to ascertain if a fix should be installed ASAP or if it can wait until the next maintenance window.

I often save the MDS report as a PDF and include it in my health check reports. Alternatively you can save the entire page as a HTML document and send this to the customer. This will ensure the hyperlinks still work and can be referenced by the customer as needed e.g.

file:///C:/data/MDS/CustomerName%20Microcode%20Discovery%20Service.htm.

By the way, if the microcode catalog file you used to generate the MDS report is not the latest, the MDS tool will warn you. If you see this message, simply download the latest file and send it over to the server where it’s needed and re-run invscout and upload the new MUP file e.g.

“WARNING: the microcode catalog used in surveying host vio1 is now out of date. Version 2014.09.10 was used for the survey and version 2014.09.17 is the most current. We recommend that you either download and install the latest microcode catalog, and run this program again, or run the MDS Applet and allow it to automatically update the catalog.”

MDS and Inventory Scout

https://www-304.ibm.com/webapp/set2/mds/fetch?page=invreadme.html

Modified by cggibbo

Migrating to AIX 7.1 with nimadm.

cggibbo Tags: gibson migration chris nimadm aix 25 Comments 39,369 Views

Last year I wrote an article on using nimadm to migrate to AIX 6.1.

The procedures outlined how to migrate a live system with nimadm and a spare disk.

Another method that I have found to be extremely useful is the migration of mksysb images from one version of AIX to another using (you guessed it) nimadm.

In the following example I migrated an AIX 6.1 mksysb image to AIX 7.1 and then used that image to install a new LPAR via NIM.

The source LPAR was running AIX 6.1 TL6 SP3. I took a mksysb of this system to my NIM master.

# oslevel -s

6100-06-03-1048

The NIM mksysb resource was named cg-aix61

root@nim1 / # lsnim -t mksysb

vio1-mksysb resources mksysb

cg-aix61 resources mksysb

root@nim1 / # lsnim -l cg-aix61

cg-aix61:

class = resources

type = mksysb

arch = power

Rstate = ready for use

prev_state = unavailable for use

location = /export/mksysb/cg-aix61

version = 6

release = 1

mod = 6

oslevel_r = 6100-06

alloc_count = 0

server = master

creation_date = Thu Jan 6 10:00:22 2011

source_image = cg

root@nim1 / #

To migrate the mksysb image from AIX 6.1 to AIX 7.1, I ran the following nimadm command:

root@nim1 / # nimadm -T cg-aix61 -O /export/mksysb/cg-aix71 -s spotaix7tl0sp2 -l aix7tl0sp2 -j nimadmvg -Y -N cg-aix71

The flags indicate the following:

-T specifies the existing AIX 6.1 NIM mksysb resource.

-O specifies the output location for the migrated mksysb resource.

-s indicates the AIX 7.1 NIM SPOT resource for the migration.

-l indicates the AIX 7.1 NIM lpp_source resource for the migration.

-j identifies the volume group that will be used on the NIM master to create file systems.

-Y Agrees to required software license agreements for software to be installed.

-N specifies the name of the new AIX 7.1 NIM mksysb resource to be created after migration.

Before running this command I made sure that I had installed the bos.alt_disk_install.rte fileset into my AIX 7.1 SPOT. If this fileset was not installed I would have received the following message immediately after issuing the nimadm command:

0042-001 nim: processing error encountered on "master":

/usr/bin/lslpp: Fileset bos.alt_disk_install.rte not installed.

0505-204 nimadm: SPOT spotaix7tl0sp2 does not have bos.alt_disk_install.rte installed.

0505-205 nimadm: The level of bos.alt_disk_install.rte installed in SPOT

spotaix7tl0sp2 (0.0.0.0) does not match the NIM master's level (7.1.0.2).

Cleaning up alt_disk_migration on the NIM master.

I also had an AIX 7.1 lpp_source and SPOT ready and waiting on my NIM master.

root@nim1 / # lsnim -t lpp_source

aix61tl6sp3 resources lpp_source

openssh resources lpp_source

aix7tl0sp2 resources lpp_source

root@nim1 / # lsnim -t spot

vio1-spot resources spot

spotaix61tl6sp3 resources spot

spotaix7tl0sp2 resources spot

OK, back to my nimadm operation. Upon initiating the migration I saw the following output on my screen (click here for a copy of the complete log file):

root@nim1 / # nimadm -T cg-aix61 -O /export/mksysb/cg-aix71 -s spotaix7tl0sp2 -l aix7tl0sp2 -j nimadmvg -Y -N cg-aix71

Initializing the NIM master.

Verifying alt_disk_migration eligibility.

Initializing log: /var/adm/ras/alt_mig/cg-aix61_alt_mig.log

Starting Alternate Disk Migration.

+-----------------------------------------------------------------------------+

Executing nimadm phase 1.

+-----------------------------------------------------------------------------+

Processing target mksysb resource cg-aix61.

Restoring /image.data from mksysb image.

+-----------------------------------------------------------------------------+

Executing nimadm phase 2.

+-----------------------------------------------------------------------------+

Creating nimadm cache file systems on volume group nimadmvg.

Checking for initial required migration space.

Creating cache file system /cg-aix61_mm_alt/alt_inst

Creating cache file system /cg-aix61_mm_alt/alt_inst/admin

Creating cache file system /cg-aix61_mm_alt/alt_inst/home

Creating cache file system /cg-aix61_mm_alt/alt_inst/opt

Creating cache file system /cg-aix61_mm_alt/alt_inst/tmp

Creating cache file system /cg-aix61_mm_alt/alt_inst/usr

Creating cache file system /cg-aix61_mm_alt/alt_inst/var

Creating cache file system /cg-aix61_mm_alt/alt_inst/var/adm/ras/livedump

+-----------------------------------------------------------------------------+

Executing nimadm phase 3.

+-----------------------------------------------------------------------------+

Syncing mksysb data to cache ...

.....etc......

+-----------------------------------------------------------------------------+

Executing nimadm phase 8.

+-----------------------------------------------------------------------------+

Creating client boot image.

bosboot: Boot image is 45709 512 byte blocks.

+-----------------------------------------------------------------------------+

Executing nimadm phase 9.

+-----------------------------------------------------------------------------+

Adjusting client file system sizes ...

Adjusting size for /

Adjusting size for /admin

Adjusting size for /home

Adjusting size for /opt

Adjusting size for /tmp

Adjusting size for /usr

Adjusting size for /var

Adjusting size for /var/adm/ras/livedump

Backing up cache data to mksysb file /export/mksysb/cg-aix71 ...

+-----------------------------------------------------------------------------+

Executing nimadm phase 10.

+-----------------------------------------------------------------------------+

Unmounting client mounts on the NIM master.

forced unmount of /cg-aix61_mm_alt/alt_inst/var/adm/ras/livedump

forced unmount of /cg-aix61_mm_alt/alt_inst/var

forced unmount of /cg-aix61_mm_alt/alt_inst/usr

forced unmount of /cg-aix61_mm_alt/alt_inst/tmp

forced unmount of /cg-aix61_mm_alt/alt_inst/opt

forced unmount of /cg-aix61_mm_alt/alt_inst/home

forced unmount of /cg-aix61_mm_alt/alt_inst/admin

forced unmount of /cg-aix61_mm_alt/alt_inst

Removing nimadm cache file systems.

Removing cache file system /cg-aix61_mm_alt/alt_inst

Removing cache file system /cg-aix61_mm_alt/alt_inst/admin

Removing cache file system /cg-aix61_mm_alt/alt_inst/home

Removing cache file system /cg-aix61_mm_alt/alt_inst/opt

Removing cache file system /cg-aix61_mm_alt/alt_inst/tmp

Removing cache file system /cg-aix61_mm_alt/alt_inst/usr

Removing cache file system /cg-aix61_mm_alt/alt_inst/var

Removing cache file system /cg-aix61_mm_alt/alt_inst/var/adm/ras/livedump

+-----------------------------------------------------------------------------+

Executing nimadm phase 11.

+-----------------------------------------------------------------------------+

Defining NIM mksysb resource ...

New NIMmksysb resource name is cg-aix71

+-----------------------------------------------------------------------------+

Executing nimadm phase 12.

+-----------------------------------------------------------------------------+

Cleaning up alt_disk_migration on the NIM master.

root@nim1 / #

For a description of all the phases in a nimadm operation, I suggest you refer to my previous nimadm article or the AIX infocenter (links below).

Once the migration had completed, I found a new mksysb file in /export/mksysb (as expected). The file was named cg-aix71. And a new NIM mksysb resource was now available.

root@nim1 / # ls -ltr /export/mksysb/

total 30793928

drwxr-xr-x 2 root system 256 Dec 21 10:40 lost+found

-rw-r--r-- 1 root system 4893952000 Dec 21 14:42 vio1-mksysb

-rw-r--r-- 1 root system 2323097600 Jan 06 10:00 cg-aix61

-rw-r--r-- 1 root system 1967411200 Jan 06 11:00 cg-aix71

root@nim1 / # lsnim -t mksysb

vio1-mksysb resources mksysb

cg-aix61 resources mksysb

cg1-mksysb resources mksysb

cg-aix71 resources mksysb

root@nim1 / # lsnim -l cg-aix71

cg-aix71:

class = resources

type = mksysb

comments = Created by nimadm on Thu Jan 6 11:00:51 EETDT 2011

Rstate = ready for use

prev_state = unavailable for use

location = /export/mksysb/cg-aix71

version = 7

release = 1

mod = 0

oslevel_r = 7100-00

alloc_count = 0

server = master

creation_date = Thu Jan 6 11:00:54 2011

During the migration I noticed the following file systems were mounted on the NIM master.

root@nim1 / # df

Filesystem 512-blocks Free %Used Iused %Iused Mounted on

/dev/hd4 1572864 1149640 27% 10471 8% /

/dev/hd2 5242880 1597696 70% 42812 19% /usr

/dev/hd9var 1048576 250456 77% 8260 23% /var

/dev/hd3 8912896 8287752 8% 111 1% /tmp

/dev/hd1 524288 523520 1% 5 1% /home

/dev/hd11admin 524288 523488 1% 5 1% /admin

/proc - - - - - /proc

/dev/hd10opt 1048576 664176 37% 7262 9% /opt

/dev/livedump 524288 523552 1% 4 1% /var/adm/ras/livedump

/dev/tftpbootlv 524288 419656 20% 10 1% /tftpboot

/dev/exportlv 524288 523408 1% 13 1% /export

/dev/lppsrclv 25690112 4383632 83% 3114 1% /export/lppsource

/dev/spotlv 8912896 4848464 46% 43450 8% /export/spot

/dev/mksysblv 38273024 11315200 71% 9 1% /export/mksysb

/dev/lv00 425984 412432 4% 25 1% /cg-aix61_mm_alt/alt_inst

/dev/lv01 65536 63376 4% 18 1% /cg-aix61_mm_alt/alt_inst/admin

/dev/lv02 65536 63384 4% 17 1% /cg-aix61_mm_alt/alt_inst/home

/dev/lv03 557056 414720 26% 201 1% /cg-aix61_mm_alt/alt_inst/opt

/dev/lv04 294912 285184 4% 21 1% /cg-aix61_mm_alt/alt_inst/tmp

/dev/lv05 4653056 4506904 4% 17 1% /cg-aix61_mm_alt/alt_inst/usr

/dev/lv06 720896 698160 4% 20 1% /cg-aix61_mm_alt/alt_inst/var

/dev/lv07 65536 63384 4% 17 1% /cg-aix61_mm_alt/alt_inst/var/adm/ras/livedump

And in my nimadmvg volume group, there were several new logical volumes.

root@nim1 / # lsvg -l nimadmvg

nimadmvg:

LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT

loglv02 jfslog 1 1 1 open/syncd N/A

lv00 jfs 13 13 1 open/syncd /cg-aix61_mm_alt/alt_inst

lv01 jfs 2 2 1 open/syncd /cg-aix61_mm_alt/alt_inst/admin

lv02 jfs 2 2 1 open/syncd /cg-aix61_mm_alt/alt_inst/home

lv03 jfs 17 17 1 open/syncd /cg-aix61_mm_alt/alt_inst/opt

lv04 jfs 9 9 1 open/syncd /cg-aix61_mm_alt/alt_inst/tmp

lv05 jfs 142 142 1 open/syncd /cg-aix61_mm_alt/alt_inst/usr

lv06 jfs 22 22 1 open/syncd /cg-aix61_mm_alt/alt_inst/var

lv07 jfs 2 2 1 open/syncd /cg-aix61_mm_alt/alt_inst/var/adm/ras/livedump

root@nim1 / #

These file systems were used to extract the mksysb locally on the NIM master and then migrate the image to AIX 7.1. I noticed the restbyname process consuming some CPU during the restore process.

Topas Monitor for host:nim1 EVENTS/QUEUES FILE/TTY

Thu Jan 6 10:15:38 2011 Interval:2 Cswitch 1155 Readch 2655.6K

Syscall 3143 Writech 2664.7K

CPU User% Kern% Wait% Idle% Physc Entc% Reads 454 Rawin 0

Total 6.1 22.0 8.2 63.7 0.09 47.04 Writes 1168 Ttyout 578

Forks 12 Igets 85

Network BPS I-Pkts O-Pkts B-In B-Out Execs 0 Namei 521

Total 7.74K 5.50 1.50 5.74K 2.00K Runqueue 0 Dirblk 97

Waitqueue 1.0

Disk Busy% BPS TPS B-Read B-Writ MEMORY

Total 0.0 5.21M 383.0 2.02M 3.19M PAGING Real,MB 2048

Faults 776 % Comp 67

FileSystem BPS TPS B-Read B-Writ Steals 1118 % Noncomp 32

Total 2.03M 85.50 2.03M 0 PgspIn 0 % Client 19

PgspOut 0

Name PID CPU% PgSp Owner PageIn 517 PAGING SPACE

restbyna 15532232 16.4 1.21M root PageOut 816 Size,MB 512

vmmd 458766 5.5 1.19M root Sios 969 % Used 3

lrud 262152 1.5 512K root % Free 97

syncd 2097296 1.1 592K root NFS (calls/sec)

topas 16973864 0.5 1.98M root SerV2 0 WPAR Activ 0

xmgc 851994 0.5 448K root CliV2 0 WPAR Total 0

nimesis 12320772 0.3 2.41M root SerV3 0 Press: "h"-help

getty 7798832 0.1 580K root CliV3 0 "q"-quit

xmtopas 8847610 0.1 816K root

nfsd 5111964 0.1 4.75M root

xmtopasa 6947052 0.1 916K root

gil 1441836 0.0 960K root

java 4784136 0.0 57.5M pconsole

java 8716308 0.0 70.7M root

rpc.lock 6029528 0.0 1.19M root

clcomd 6160578 0.0 1.69M root

pilegc 786456 0.0 640K root

aixmibd 3997882 0.0 1.24M root

netm 1376298 0.0 448K root

topasrec 8454202 0.0 1.01M root

root@nim1 / # ps -ef | grep rest

root 15270132 3342336 1 10:15:55 pts/1 0:00 grep rest

root 15532232 10747910 37 10:14:02 pts/0 0:02 restbyname -x -q -f /export/mksysb/cg-aix61

root@nim1 / #

I didn’t encounter any issues with my nimadm operation but if I had I could have reviewed the nimadm log file located in /var/adm/ras:

root@nim1 /var/adm/ras/alt_mig # ls -ltr

total 856

-rw-r--r-- 1 root system 435009 Jan 06 11:00 cg-aix61_alt_mig.log

If you ever encounter problems with nimadm, I highly recommend that you re-run the operation with the –V and –D flags. This will enable verbose output, as well as debugging information which can assist you greatly in determining the root cause of your issue.

I was able to use the new, migrated, AIX 7.1 mksysb image to install a new LPAR.

AIX Version 7

root's Password:

*******************************************************************************

* *

* Welcome to AIX Version 7.1! *

* *

* Please see the README file in /usr/lpp/bos for information pertinent to *

* this release of the AIX Operating System. *

* *

*******************************************************************************

Last login: Thu Jan 6 13:21:04 EETDT 2011 on /dev/vty0

# oslevel -s

7100-00-02-1041

You’ve gotta love NIM! Take a look at the nimadm documentation for more information:

http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.cmds/doc/aixcmds4/nimadm.htm

By the way, if you are interested in migrating from AIX 4.3 to AIX 7.1, then you’ll need to consider the mksysb migration method. I’ve not done this myself (i.e. 4.3 to 7.1 migration) but the feedback I received from the AIX development team indicates that it does work as expected.

“... you had expressed some interest in mksysb migration. Since we were not aware of anyone having attempted it with 7.1. I went ahead and created a 4.3.3.75 system and created a mksysb and then used that mksysb in conjunction with nim and successfully installed another system using the 4.3 mksysb and 7.1 nim resources to perform a mksysb migration to 7.1. I just followed the steps as noted in the Installation guide for mksysb migration. “

Refer to the following document for further information on mksysb migration with NIM.

http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.install/doc/insgdrf/mksysbmigration.htm

NIM from A to Z in AIX 5L Redbook.

http://www.redbooks.ibm.com/redpieces/abstracts/sg247296.html?Open

Migrating you NIM Master to AIX 7.2 using the VIOS VML

cggibbo Tags: master using sddpcm 7.2 vml chris the nimadm to you migrating gibson nim vios aix 1,136 Views

On more than one occasion I’ve written articles and blogs on how to migrate AIX systems to a new version of the AIX operating system. You can find a list these articles (and others) here. Each of the NIM articles focuses on migrating AIX systems (which are configured as NIM clients). The approach is always the same; use a NIM master to migrate the NIM clients to the next release of AIX. What they don’t discuss is how to migrate your NIM master to a new version of the OS. This is the very first step you should take before you consider migrating your clients. The NIM master should always be at the same or higher AIX level than the level you are migrating to on a client. So, in this latest article, I’ll discuss, step by step, how to migrate your NIM master from AIX 7.1 to AIX 7.2 (the latest release of the AIX OS).

https://www.ibm.com/developerworks/community/blogs/cgaix/resource/Migrating_NIMMaster_AIX72_VML.pdf

Modified by cggibbo

Minimum memory requirement for AIX 7.2 Live Update

cggibbo Tags: gibson minimum partition aix for live chris memory 7.2 profile updates 2,449 Views

AIX 7.2 requires 2GB of memory to boot, but this minimum is not enforced in the LPAR profile except by Live Update (to ensure we'll be able to boot the surrogate LPAR). You can check your Minimum Memory setting in your LPARs profile by running the lparstat command (as shown below).

# lparstat -i | grep Memory
Online Memory : 4096 MB
Maximum Memory : 8192 MB
Minimum Memory : 2048 MB

If your partition does not meet the minimum memory profile requirement, you’ll receive the following error message when you perform a live update preview (with geninstall –p –k).

Checking lpar minimal memory size:

------------------------------------------

Required memory size: 2048 MB

Current memory size: 1024 MB

1430-119 FAILED: the lpar minimal memory size is not sufficient

You’ll need to change partition profile so that the minimum memory setting is, at least, 2048 (2GB) and then stop & start the partition for the profile update to take effect.

The /var/adm/ras/liveupdate/logs/lvupdlog log file will also contain error messages indicating the problem:

LVUPD 11/28/2016-18:52:00.716 DEBUG lvupdate_utils32.c - 6713 - lvup_check_minmem_size: Partition minimum memory size (1024 MB) on p8tlc-lvup is lower than the minimum memory size required (2048 MB).

LVUPD 11/28/2016-18:52:00.716 ERROR lvupdate_utils32.c - 8647 - lvup_preview: ERROR(s) while checking the current mimimal memory size against the computed required size.

Modified by cggibbo

mkwpar, truss and coffee!

cggibbo Tags: 0960-288 /wpars/wpar20 mkwpar: gibson file system mkwpar chris wpar exists. already the aix 7,293 Views

A colleague was attempting to recreate a WPAR. The WPAR had previously been removed using the WPAR Manager Web GUI. This worked, or so he tells me!

Anyway, when he tried to create the WPAR again from the GUI, using the same WPAR name (wpar20), it failed. The error claimed that /wpars/wpar20 already existed. But it didn’t!

Now, before I go any further, looking back at the issue now, it is pretty obvious what the problem might have been. However, the solution was not evident at first. This proves one thing for certain: some problems cannot be resolved without TWO cups of coffee in the morning.

Of course, I resorted to the command line to see what was REALLY happening.

The mkwpar command was indeed complaining that /wpars/wpar20 already existed.

# mkwpar -n wpar20

mkwpar: 0960-288 The /wpars/wpar20 file system already exists.

mkwpar: 0960-417 Specify -p to create this workload partition using the existing file system data.

I could not find any directory or file system with that name, anywhere under /wpars or elsewhere.

# ls -l /wpars/wpar20

/wpars/wpar20 not found

# ls -l /wpars

total 0

# df –g | grep –i wpar

# lsvg | lsvg –il | grep –i wpar

I ran the mkwpar with truss to try to find out where it was finding the reference to /wpars/wpar20. At this point you have probably already determined where the problem might be!

The truss output pointed me in the direction of /etc/filesystems.....of course! Another “facepalm“ moment!

# truss mkwpar -n wpar20 > /tmp/wpar20 2>&1

# vi /tmp/wpar220

statx("/wpars/wpar20", 0x30008AB8, 128, 010) = -1

kopen("/etc/filesystems", O_RDONLY|O_LARGEFILE) = 4

Whatever my learned colleague had done when he “removed” the WPAR with the GUI, it left entries for the WPARs file systems in /etc/filesystems.

# grep wpar20 /etc/filesystems | grep -v type

/wpars/wpar20:

/wpars/wpar20/home:

/wpars/wpar20/opt:

/wpars/wpar20/proc:

/wpars/wpar20/tmp:

/wpars/wpar20/usr:

/wpars/wpar20/var:

And after removing these entries with the rmfs command, like so:

# grep wpar20 /etc/filesystems | grep -v type | sed -e 's/://'

/wpars/wpar20

/wpars/wpar20/home

/wpars/wpar20/opt

/wpars/wpar20/proc

/wpars/wpar20/tmp

/wpars/wpar20/usr

/wpars/wpar20/var

# grep wpar20 /etc/filesystems | grep -v type | sed -e 's/://'

# cg="/wpars/wpar20

> /wpars/wpar20/home

> /wpars/wpar20/opt

> /wpars/wpar20/proc

> /wpars/wpar20/tmp

> /wpars/wpar20/usr

> /wpars/wpar20/var

> "

# echo $cg

/wpars/wpar20 /wpars/wpar20/home /wpars/wpar20/opt /wpars/wpar20/proc /wpars/wpar20/tmp /wpars/wpar20/usr /wpars/wpar20/var

# for i in $cg

> do

> rmfs $i

> done

We were able to create the WPAR once more without a problem.

# mkwpar -n wpar20

mkwpar: Creating file systems...

/home

/opt

/proc

/tmp

/usr

/var

Mounting all workload partition file systems.

x ./usr

x ./lib

x ./admin

x ./admin/tmp

x ./audit

x ./dev

x ./etc

x ./etc/check_config.files

x ./etc/consdef

x ./etc/cronlog.conf

x ./etc/csh.cshrc

x ./etc/csh.login

x ./etc/dlpi.conf

x ./etc/dumpdates

x ./etc/environment

x ./etc/ewlm

x ./etc/ewlm/limits

x ./etc/ewlm/trc

x ./etc/ewlm/trc/config_schema.xsd

x ./etc/ewlm/trc/output_schema.xsd

x ./etc/filesystems

x ./etc/group

x ./etc/inittab

x ./etc/magic

...etc...

Lessons I learnt from this were twofold, truss is (still) useful and drink more coffee before tackling a problem!

Monitoring Multiple Shared Processor Pools with topas.

cggibbo 7,225 Views

A quick tip for monitoring Multiple Shared Processor Pools with topas.

Execute topas from the AIX command line. Press the ‘p’ key to toggle on ‘pool metrics’. You will see the default pool (pool id 0) and any other pools defined on your system. In the example below, there are two pools, the default (0) and a user defined pool (1). The default display shows the LPARs in the default pool (0), which in this case is sys06_partition8.

Using the up and down arrow keys, I place the cursor on pool ID number one, in the pool column on the left-hand side and then press the ‘f’ key to toggle the focus. This changes what is displayed in the lower portion of the output. The LPAR associated with pool 1 (sys06_partition9) is shown.

# topas –C

Multiple Shared Processor Pools are supported on Power6 and Power7 systems.

Follow me on Twitter: http://twitter/cgibbo

More NIM troubleshooting....some basics...

cggibbo Tags: resource to filesystem m_mk_lpp_source: oslevel space failed all check a "master" 0042-185 on nim already copy due not exists aix c_ch_nfsexp: instfix rest or permissions link filesets 0042-081 nfs found. for /etc/.tmp.nim.exports export were and 1 Comment 28,213 Views

It’s no secret that I like NIM. I’ve written about NIM on several occasions, in Redbooks, technical articles and on my blog. It’s a great tool for installing and maintaining AIX. Occasionally though, it can throw you an unusual error or two when things go wrong. For the uninitiated resolving these errors can be challenging.

Here’s a couple of NIM related questions I received recently. Both are troubleshooting queries.

“I'm getting the following error when trying to create a NIM lpp_source for AIX 6.1”:

Define a Resource

Type or select values in entry fields.

Press Enter AFTER making all desired changes.

[Entry Fields]

* Resource Name [6100-06_lpp]

* Resource Type lpp_source

* Server of Resource [master] +

* Location of Resource [/export/eznim] /

NFS Client Security Method [] +

NFS Version Access [] +

Architecture of Resource [] +

Source of Install Images [/dev/cd0] +/

Names of Option Packages []

Show Progress [yes] +

Comments []

...

Preparing to copy install images (this will take several minutes)...

0042-001 nim: processing error encountered on "master":

0042-081 m_mk_lpp_source: a resource already exists on "master" at

location "/export/eznim/bid_ow"; due to NFS export restrictions, the

new location "/export/eznim" may not be used

...

I could see what could be causing the issue but I first asked to see the contents of the /etc/exports file. Just to make sure that /export/eznim was not explicitly exported. It wasn’t.

# grep eznim /etc/exports

/export/eznim/vios/mksysb_image -ro,root=vio1,access=vio1

/export/eznim/bid_ow -ro,root=nim1,access=nim1

I suggested they re-try the operation again but this time specify the full path to the lpp_source directory, like so:

Define a Resource

Type or select values in entry fields.

Press Enter AFTER making all desired changes.

[Entry Fields]

* Resource Name [6100-06_lpp]

* Resource Type lpp_source

* Server of Resource [master] +

* Location of Resource [/export/eznim/6100-06_lpp] /

NFS Client Security Method [] +

NFS Version Access [] +

Architecture of Resource [] +

Source of Install Images [/dev/cd0] +/

Names of Option Packages []

Show Progress [yes] +

Comments []

This worked fine. And the lpp_source was created successfully.

# lsnim -l 6100-06_lpp

6100-06_lpp:

class = resources

type = lpp_source

locked = 26738726

Rstate = unavailable for use

prev_state =

location = /export/eznim/6100-06_lpp

alloc_count = 0

server = master

The second issue related to down-level filesets after an AIX 6.1 migration with nimadm. There was also an issue with disk space that generated an error during NIM operations.

“Hi Chris,

Thanks for replying back to my post. I am a newbie to AIX/pSeries world.

I was following your document on upgrading AIX 5.3 to 6.1 using nimadm, it seemed to have worked partially for me, as I got to AIX 6100-00-00. The mksysb and the SPOT I created were for AIX 6100-06-05.

Before I re-attempt the upgrade, is there anything I need to be clean up on AIX 5.3 side that could have perhaps caused issues with the upgrade.

Thanks”

I recommended that the following command be run on the NIM client, to determine which filesets were down-level after the migration.

# oslevel -s

# instfix -i | grep AIX

# instfix -i | grep SP

If the output contained messages like "Not all filesets for 6100-06_AIX_ML were found", or something similar, then they should run the following command against each line (there could be more than one).

# instfix -icqk 6100-06_AIX_ML | grep ":-:"

This will tell you which filesets were not updated.

The output from ‘oslevel –s’and ‘instfix –i | grep AIX’ looked OK.

# oslevel -s

6100-06-04-1112

# instfix -i | grep AIX

All filesets for 6100-00_AIX_ML were found.

All filesets for 6100-01_AIX_ML were found.

All filesets for 6100-02_AIX_ML were found.

All filesets for 6100-03_AIX_ML were found.

All filesets for 6100-04_AIX_ML were found.

All filesets for 6100-05_AIX_ML were found.

All filesets for 6100-06_AIX_ML were found.

But the output from ‘instfix –i | grep SP’ indicated an issue with 61-06-051115_SP.

# instfix -i | grep SP

All filesets for 61-00-010748_SP were found.

All filesets for 61-00-020750_SP were found.

All filesets for 61-00-030808_SP were found.

All filesets for 61-00-040815_SP were found.

All filesets for 61-01-010823_SP were found.

All filesets for 61-00-050822_SP were found.

All filesets for 61-00-060834_SP were found.

All filesets for 61-01-020834_SP were found.

All filesets for 61-00-070846_SP were found.

All filesets for 61-01-030846_SP were found.

All filesets for 61-02-010847_SP were found.

All filesets for 61-02-020849_SP were found.

All filesets for 61-00-080909_SP were found.

All filesets for 61-01-040909_SP were found.

All filesets for 61-02-030909_SP were found.

All filesets for 61-03-010921_SP were found.

All filesets for 61-00-090920_SP were found.

All filesets for 61-01-050920_SP were found.

All filesets for 61-02-040920_SP were found.

All filesets for 61-00-100939_SP were found.

All filesets for 61-01-060939_SP were found.

All filesets for 61-02-050939_SP were found.

All filesets for 61-03-020939_SP were found.

All filesets for 61-04-010944_SP were found.

All filesets for 61-04-021007_SP were found.

All filesets for 61-04-031009_SP were found.

All filesets for 61-00-110943_SP were found.

All filesets for 61-01-081014_SP were found.

All filesets for 61-02-060943_SP were found.

All filesets for 61-03-030943_SP were found.

All filesets for 61-03-041014_SP were found.

All filesets for 61-04-041014_SP were found.

All filesets for 61-05-011016_SP were found.

All filesets for 61-01-091015_SP were found.

All filesets for 61-02-081015_SP were found.

All filesets for 61-03-051015_SP were found.

All filesets for 61-04-051015_SP were found.

All filesets for 61-01-070943_SP were found.

All filesets for 61-02-071014_SP were found.

All filesets for 61-02-091034_SP were found.

All filesets for 61-03-061034_SP were found.

All filesets for 61-04-061034_SP were found.

All filesets for 61-05-021034_SP were found.

All filesets for 61-06-011043_SP were found.

All filesets for 61-02-101036_SP were found.

All filesets for 61-03-071036_SP were found.

All filesets for 61-04-071036_SP were found.

All filesets for 61-05-031036_SP were found.

All filesets for 61-06-021044_SP were found.

All filesets for 61-06-031048_SP were found.

All filesets for 61-05-041048_SP were found.

All filesets for 61-04-081048_SP were found.

All filesets for 61-03-081048_SP were found.

All filesets for 61-03-091112_SP were found.

All filesets for 61-04-091112_SP were found.

All filesets for 61-05-051112_SP were found.

All filesets for 61-06-041112_SP were found.

Not all filesets for 61-06-051115_SP were found.

The culprit was the csm.hc_utils fileset from TL6 SP5. It was still at level 1.7.1.0 instead of the expected 1.7.1.1.

# instfix -ciqk "61-06-051115_SP" |grep ":-:"

61-06-051115_SP:csm.hc_utils:1.7.1.1:1.7.1.0:-:AIX 6100-06-05 Service Pack

I recommended that the csm.hc_utils 1.7.1.1 fileset be added to the AIX 6.1 lpp_source. The migration was attempted again and the client migrated to AIX 6.1 TLP6 SP5 successfully. I also suggested they determine if this fileset was really used and/or needed (CSM is no longer supported/available starting with AIX 7.1 anyway).

And just when I thought all his problems were resolved, he replied with: “however upon trying to install these from the NIM server, I get errors on NIM”.

*********************************************************

errors:

0042-001 nim: processing error encountered on "master": |

| 0042-001 m_showres: processing error encountered on "master": |

| 0042-185 c_ch_nfsexp: Failed to link or copy /etc/.tmp.nim.exports. |

| 12451894 to /etc/exports |

| Check permissions and filesystem space

*********************************************************

Fortunately a quick look at the output from the df command highlighted the problem immediately.

He was out of space in the root file system. The very place that /etc/exports, and the NIM database (/etc/objrepos), reside.

# df -g

Filesystem GB blocks Free %Used Iused %Iused Mounted on

/dev/hd4 2.50 0.00 100% 3388 1% /

/dev/hd2 4.50 1.74 62% 42150 4% /usr

/dev/hd9var 1.00 0.52 48% 3262 2% /var

/dev/hd3 4.00 3.84 5% 74 1% /tmp

/dev/hd1 0.50 0.37 27% 479 1% /home

/proc - - - - - /proc

/dev/hd10opt 0.75 0.21 73% 5698 3% /opt

/dev/livedump 0.25 0.24 4% 17 1% /var/adm/ras/livedump

/dev/exportlv 136.25 124.03 9% 20128 1% /export

After cleaning up unwanted files in / and making sure there was plenty of free space, the error no longer appeared in future NIM operations.

Multibos saved my bacon!

cggibbo Tags: bacon multibos max_xfer_size chris_gibson aix 8 Comments 12,488 Views

A colleague of mine was planning to modify the max_xfer_size attribute on a couple of FC adapters in one of his AIX LPARs. As he was describing his plan to me, I asked him how he intended to back out of the change should the LPAR fail to boot after the modifications. “But, what could possibly go wrong?” he fired back. I advised him to use multibos to create a standby (backup) instance of the AIX OS, just in case. He begrudgingly did so, just to keep me happy.

The next day he told me the following tale.

He had modified the FC adapters max_xfer_size attribute as planned. First, checking the current values, for the attribute on both adapters.

aixlpar1 : / # lsattr -El fcs0 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

aixlpar1 : / # lsattr -El fcs1 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

He’d created a standby AIX instance before making changes to the adapters. He also prevented multibos from changing the bootlist to the standby boot logical volume (BLV).

aixlpar1 : / # multibos -sXt

Initializing multibos methods ...

Initializing log /etc/multibos/logs/op.alog ...

Gathering system information ...

+-----------------------------------------------------------------------------+

Setup Operation

+-----------------------------------------------------------------------------+

Verifying operation parameters ...

Creating image.data file ...

He modified the FC adapters as planned.

aixlpar1 : / # chdev -l fcs0 -a max_xfer_size=0x200000 -P

fcs0 changed

aixlpar1 : / # chdev -l fcs1 -a max_xfer_size=0x200000 -P

fcs1 changed

aixlpar1 : / # lsattr -El fcs0 -a max_xfer_size

max_xfer_size 0x200000 Maximum Transfer Size True

aixlpar1 : / # lsattr -El fcs1 -a max_xfer_size

max_xfer_size 0x200000 Maximum Transfer Size True

He verified that the standby instance still held the original values for both FC adapters.

aixlpar1 : / # multibos -S

Initializing multibos methods ...

Initializing log /etc/multibos/logs/op.alog ...

Gathering system information ...

+-----------------------------------------------------------------------------+

Multibos Shell Operation

+-----------------------------------------------------------------------------+

Verifying operation parameters ...

+-----------------------------------------------------------------------------+

Mount Processing

+-----------------------------------------------------------------------------+

Mounting all standby BOS file systems ...

Mounting /bos_inst

Mounting /bos_inst/usr

Mounting /bos_inst/var

Mounting /bos_inst/opt

+-----------------------------------------------------------------------------+

Multibos Root Shell

+-----------------------------------------------------------------------------+

Starting multibos root shell ...

Active boot logical volume is hd5.

Script command is started. The file is /etc/multibos/logs/scriptlog.120713124518.txt.

aixlpar1 : / # lsattr -El fcs0 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

aixlpar1 : / # lsattr -El fcs1 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

aixlpar1 : / # exit

Script command is complete. The file is /etc/multibos/logs/scriptlog.120713124518.txt.

Stopping multibos root shell ...

Compressing script log file ...

Compressed script log file is /etc/multibos/logs/scriptlog.120713124518.txt.Z

+-----------------------------------------------------------------------------+

Mount Processing

+-----------------------------------------------------------------------------+

Unmounting all standby BOS file systems ...

Unmounting /bos_inst/opt

Unmounting /bos_inst/var

Unmounting /bos_inst/usr

Unmounting /bos_inst

Log file is /etc/multibos/logs/op.alog

Return Status = SUCCESS

Then he manually changed the LPARs boot list to include the standby BLV.

aixlpar1 : / # bootlist -m normal hdisk2 blv=hd5 hdisk2 blv=bos_hd5

aixlpar1 : / # bootlist -m normal -o

hdisk2 blv=hd5 pathid=0

hdisk2 blv=hd5 pathid=1

hdisk2 blv=bos_hd5 pathid=0

hdisk2 blv=bos_hd5 pathid=1

He carefully recorded the bootlist output, just in case the boot failed with new max_xfer_size values. He could use the vdevice name and location to manually select the standby BLV to start the system in an emergency.

aixlpar1 : / # bootlist -m normal -ov

'ibm,max-boot-devices' = 0x5

NVRAM variable: (boot-device=/vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:2 /vdevice/vfc-client@3000001e/disk@50060e8006d0207a,1000000000000:2 /vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:4 /vdevice/vfc-client@3000001e/disk@50060e8006d0207a,1000000000000:4)

Path name: (/vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:2)

match_specific_info: ut=disk/fcp/htcvspmpio

hdisk2 blv=hd5 pathid=0

Path name: (/vdevice/vfc-client@3000001e/disk@50060e8006d0207a,1000000000000:2)

match_specific_info: ut=disk/fcp/htcvspmpio

hdisk2 blv=hd5 pathid=1

Path name: (/vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:4)

match_specific_info: ut=disk/fcp/htcvspmpio

hdisk2 blv=bos_hd5 pathid=0

Path name: (/vdevice/vfc-client@3000001e/disk@50060e8006d0207a,1000000000000:4)

match_specific_info: ut=disk/fcp/htcvspmpio

hdisk2 blv=bos_hd5 pathid=1

He restarted the LPAR using the primary BLV, with the modified FC attributes.

The system hung at LED 554.

hscroot@hmc1:~> lsrefcode -m 795-1 -r lpar --filter "lpar_names=aixlpar1" -F lpar_name:refcode

aixlpar1:0554

He restarted the LPAR in Open Firmware mode.

At the Open Firmware prompt, he entered the following to boot the LPAR from the standby instance BLV:

IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM IBM

1 = SMS Menu 5 = Default Boot List

8 = Open Firmware Prompt 6 = Stored Boot List

Memory Keyboard Network SCSI Speaker ok

0 > boot /vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:4 |

The system booted OK on the standby AIX instance.

Elapsed time since release of system processors: 28858 mins 55 secs

-------------------------------------------------------------------------------

Welcome to AIX.

boot image timestamp: 02:39:13 07/13/2012

The current time and date: 02:54:02 07/13/2012

processor count: 1; memory size: 4096MB; kernel size: 35062697

boot device: /vdevice/vfc-client@30000014/disk@50060e8006d0206a,1000000000000:4

-------------------------------------------------------------------------------

The FC adapters were running on the previous values, stored in the standby instance of AIX.

aixlpar1 : / # lsattr -El fcs0 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

aixlpar1 : / # lsattr -El fcs1 -a max_xfer_size

max_xfer_size 0x100000 Maximum Transfer Size True

The original AIX instance still held the modified values for the FC adapters.

aixlpar1 : / # multibos -S

Initializing multibos methods ...

Initializing log /etc/multibos/logs/op.alog ...

Gathering system information ...

+-----------------------------------------------------------------------------+

Multibos Shell Operation

+-----------------------------------------------------------------------------+

Verifying operation parameters ...

+-----------------------------------------------------------------------------+

Mount Processing

+-----------------------------------------------------------------------------+

Mounting all standby BOS file systems ...

Mounting /bos_inst

Mounting /bos_inst/usr

Mounting /bos_inst/var

Mounting /bos_inst/opt

+-----------------------------------------------------------------------------+

Multibos Root Shell

+-----------------------------------------------------------------------------+

Starting multibos root shell ...

Active boot logical volume is bos_hd5.

Script command is started. The file is /etc/multibos/logs/scriptlog.120713125542.txt.

aixlpar1 : / # lsattr -El fcs0 -a max_xfer_size

max_xfer_size 0x200000 Maximum Transfer Size True

aixlpar1 : / # lsattr -El fcs1 -a max_xfer_size

max_xfer_size 0x200000 Maximum Transfer Size True

aixlpar1 : / # exit

Script command is complete. The file is /etc/multibos/logs/scriptlog.120713125542.txt.

Stopping multibos root shell ...

Compressing script log file ...

Compressed script log file is /etc/multibos/logs/scriptlog.120713125542.txt.Z

+-----------------------------------------------------------------------------+

Mount Processing

+-----------------------------------------------------------------------------+

Unmounting all standby BOS file systems ...

Unmounting /bos_inst/opt

Unmounting /bos_inst/var

Unmounting /bos_inst/usr

Unmounting /bos_inst

Log file is /etc/multibos/logs/op.alog

Return Status = SUCCESS

The cause of the 554 hang appeared to be related to the fact that the VIOS physical adapters needed their max_xfer_size value changed to the new value before the client LPAR virtual fibre channel adapters were modified.

My colleague was glad he used multibos. It saved his bacon.

My first POWER7 LPAR.

cggibbo 6,389 Views

Thanks to the IBM Virtual Loaner Program, I was able to try out my first POWER7 LPAR today. Some interesting items for me to research.

# lparstat -i

Node Name : node1

Partition Name : node1

Partition Number : 5

Type : Shared-SMT-4

Mode : Uncapped

Entitled Capacity : 0.50

Partition Group-ID : 32773

Shared Pool ID : 0

Online Virtual CPUs : 2

Maximum Virtual CPUs : 2

Minimum Virtual CPUs : 1

Online Memory : 4096 MB

Maximum Memory : 4096 MB

Minimum Memory : 1024 MB

Variable Capacity Weight : 128

Minimum Capacity : 0.25

Maximum Capacity : 0.50

Capacity Increment : 0.01

Maximum Physical CPUs in system : 32

Active Physical CPUs in system : 32

Active CPUs in Pool : 32

Shared Physical CPUs in system : 32

Maximum Capacity of Pool : 3200

Entitled Capacity of Pool : 550

Unallocated Capacity : 0.00

Physical CPU Percentage : 25.00%

Unallocated Weight : 0

Memory Mode : Dedicated

Total I/O Memory Entitlement : -

Variable Memory Capacity Weight : -

Memory Pool ID : -

Physical Memory in the Pool : -

Hypervisor Page Size : -

Unallocated Variable Memory Capacity Weight: -

Unallocated I/O Memory entitlement : -

Memory Group ID of LPAR : -

Desired Virtual CPUs : 2

Desired Memory : 4096 MB

Desired Variable Capacity Weight : 128

Desired Capacity : 0.50

Target Memory Expansion Factor : -

Target Memory Expansion Size : -

It appears to be a POWER7 750 (8233-E8B) with 32 x 3.55 GHz processors.

System Model: IBM,8233-E8B

Machine Serial Number: XXXXXXX

Processor Type: PowerPC_POWER7

Processor Implementation Mode: POWER 7

Processor Version: No Data Available

Number Of Processors: 2

Processor Clock Speed: 3550 MHz

CPU Type: 64-bit

Kernel Type: 64-bit

LPAR Info: 1 node1

Memory Size: 4096 MB

Good Memory Size: 4096 MB

Platform Firmware level: AL710_039

Firmware Version: IBM,AL710_039

My LPAR always boots into SMS. Why?

cggibbo 5 Comments 26,223 Views

If you turn on Multiboot from within the SMS menu (as shown below), your LPAR will always boot into SMS, regardless of your AIX boot list and/or LPAR profile settings.

PowerPC Firmware

Version EM310_048

-------------------------------------------------------------------------------

Multiboot

1. Select Install/Boot Device

2. Configure Boot Device Order

3. Multiboot Startup <ON>

-------------------------------------------------------------------------------

Navigation keys:

M = return to Main Menu

ESC key = return to previous screen X = eXit System Management Services

-------------------------------------------------------------------------------

Type menu item number and press Enter or select Navigation key:

If your LPAR always boots to the SMS menu, check the Multiboot Startup setting. If its set to ON, change it to OFF (as shown below). To access this option you must enter “5. Select Boot Options” from the main SMS menu.

PowerPC Firmware

Version EM310_048

-------------------------------------------------------------------------------

Multiboot

1. Select Install/Boot Device

2. Configure Boot Device Order

3. Multiboot Startup <OFF>

-------------------------------------------------------------------------------

Navigation keys:

M = return to Main Menu

ESC key = return to previous screen X = eXit System Management Services

-------------------------------------------------------------------------------

Type menu item number and press Enter or select Navigation key:

Enabling Multiboot is only useful if you’re planning on dual-booting AIX and Linux within an LPAR. With Multiboot turned on, each time you reboot the LPAR, you are taken to SMS so you can choose which disk to boot from i.e. the AIX boot disk or the Linux boot disk. I’ve never come across anyone actually doing this in the commercial world. Mathew Davis (from IBM) wrote an article about dual booting AIX and Linux on pSeries back in 2005:

http://www.ibmsystemsmag.com/aix/aprilmay05/tipstechniques/6640p1.aspx

There can be other reasons why an LPAR always boots into SMS. The following link has some information on this:

http://www.aixmind.com/?tag=keeps-booting-to-sms

This IBM document explains how to select a boot disk from within the SMS menu.

http://www-01.ibm.com/support/docview.wss?rs=1209&context=SWG10&dc=DB520&q1=multiboot+SMS&uid=isg3T1011805&loc=en_US&cs=UTF-8&lang=en

Remember, if the disk doesn’t have a valid boot image, it won’t appear in the SMS menu as a boot device!

One way to check you have a boot device on your system is to run the ipl_varyon command from the AIX command line (before restarting your LPAR!):

# ipl_varyon -i

PVNAME BOOT DEVICE PVID VOLUME GROUP ID

hdisk0 YES 00f6048868b4e0ef0000 00f6048800004c00

If you don’t see a boot device on your system, perhaps someone has accidentally removed the boot record? Or maybe it was never created?

For example, in the commands that follow, I have “accidentally” run the ‘chpv –c’ command against my boot disk. This clears the boot record from the disk! The ipl_varyon command no longer shows hdisk0 as a bootable device.

Leaving the system in this state would prevent the system from booting!

To recreate the boot image, I run the bosboot command against hdisk0. Now ipl_varyon recognises hdisk0 as a bootable device again. Phew!

# chpv -c hdisk0

# ipl_varyon -i

PVNAME BOOT DEVICE PVID VOLUME GROUP ID

hdisk0 NO 00f6048868b4e0ef0000 00f6048800004c00

# bosboot -a -d /dev/hdisk0

bosboot: Boot image is 42689 512 byte blocks.

# ipl_varyon -i

PVNAME BOOT DEVICE PVID VOLUME GROUP ID

hdisk0 YES 00f6048868b4e0ef0000 00f6048800004c00

My root volume group is called old_rootvg....HELP!!

cggibbo Tags: aix importvg old_rootvg chris_gibson rootvg synclvodm odmdelete 5 Comments 12,407 Views

So, your root volume group is called old_rootvg not rootvg?

Wait……what!?

aixlpar1 : # lspv

hdisk0 00c342c68dfcbdfb old_rootvg active < This is BAD!

hdisk1 00c342c6161c6b47 appvg active

How did you end up in this situation? Did you reinstall AIX with some strange incantation or did some alternate disk activity go horribly wrong? Not sure? Me neither!

Fortunately the following tech-note and associated script can help.

http://www-01.ibm.com/support/docview.wss?uid=isg3T1000119

“Question

Rebuilding a Volume Group's Customized Device Database.

Answer

Run the script in this document when the ODM (Object Data Manager) entries for the root volume group are corrupted.”

“The script below assumes you are rebuilding the ODM definitions for the rootvg, which cannot be exported and then re-imported on a live system.

For most other volume groups, an export/import is the best way to clean up ODM problems.

If you need to rebuild ODM definitions for volume groups other than the rootvg, substitute appropriate values for PV and VG.”

So here’s the script:

#!/usr/bin/ksh
PV=`lslv -l hd5 | grep hdisk | head -1 | awk '{print $1}'`
VG=rootvg
lqueryvg -Lp $PV | awk '{ print $2 }' | while read LVname; do
odmdelete -q "name = $LVname" -o CuAt
odmdelete -q "name = $LVname" -o CuDv
odmdelete -q "value3 = $LVname" -o CuDvDr
odmdelete -q "dependency = $LVname" -o CuDep
done
odmdelete -q "name = $VG" -o CuAt
odmdelete -q "parent = $VG" -o CuDv
odmdelete -q "name = $VG" -o CuDv
odmdelete -q "name = $VG" -o CuDep
odmdelete -q "dependency = $VG" -o CuDep
if [ "$VG" = rootvg ]
then
odmdelete -q "value1 = 10" -o CuDvDr
else
odmdelete -q "value1 = $VG" -o CuDvDr
fi
odmdelete -q "value3 = $VG" -o CuDvDr
importvg -y $VG $PV # ignore lvaryoffvg errors
varyonvg $VG
synclvodm -Pv $VG # use -P only if running AIX V4.2.1 or later
savebase

Let’s execute the script to resolve this problem.

Note: Try this on a crash’n’burn system before unleashing it’s fury on a real AIX system (i.e. one that has users that depend on it!). Always take a mksysb backup before performing this type of activity.

aixlpar1 : /tmp # ksh -x fixmyrootvg.ksh

+ + lslv -l hd5

+ grep hdisk

+ head -1

+ awk {print $1}

PV=hdisk0

+ VG=rootvg

+ lqueryvg -Lp hdisk0

+ awk { print $2 }

+ read LVname

+ odmdelete -q name = hd5 -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd5 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd5 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd5 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd6 -o CuAt

0518-307 odmdelete: 4 objects deleted.

+ odmdelete -q name = hd6 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd6 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd6 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd8 -o CuAt

0518-307 odmdelete: 3 objects deleted.

+ odmdelete -q name = hd8 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd8 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd8 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd4 -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd4 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd4 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd4 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd2 -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd2 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd2 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd2 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd9var -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd9var -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd9var -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd9var -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd3 -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd3 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd3 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd3 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd1 -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd1 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd1 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd1 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd10opt -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd10opt -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd10opt -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd10opt -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = local -o CuAt

0518-307 odmdelete: 4 objects deleted.

+ odmdelete -q name = local -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = local -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = local -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd7 -o CuAt

0518-307 odmdelete: 3 objects deleted.

+ odmdelete -q name = hd7 -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd7 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd7 -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = hd11admin -o CuAt

0518-307 odmdelete: 5 objects deleted.

+ odmdelete -q name = hd11admin -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = hd11admin -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q dependency = hd11admin -o CuDep

0518-307 odmdelete: 1 objects deleted.

+ read LVname

+ odmdelete -q name = rootvg -o CuAt

0518-307 odmdelete: 3 objects deleted.

+ odmdelete -q parent = rootvg -o CuDv

0518-307 odmdelete: 0 objects deleted.

+ odmdelete -q name = rootvg -o CuDv

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q name = rootvg -o CuDep

0518-307 odmdelete: 0 objects deleted.

+ odmdelete -q dependency = rootvg -o CuDep

0518-307 odmdelete: 0 objects deleted.

+ [ rootvg = rootvg ]

+ odmdelete -q value1 = 10 -o CuDvDr

0518-307 odmdelete: 1 objects deleted.

+ odmdelete -q value3 = rootvg -o CuDvDr

0518-307 odmdelete: 0 objects deleted.

+ importvg -y rootvg hdisk0

rootvg

0516-012 lvaryoffvg: Logical volume must be closed. If the logical

volume contains a filesystem, the umount command will close

the LV device.

0516-942 varyoffvg: Unable to vary off volume group rootvg.

+ varyonvg rootvg

+ synclvodm -Pv rootvg

synclvodm: Physical volume data updated.

synclvodm: Logical volume hd5 updated.

synclvodm: Logical volume hd6 updated.

synclvodm: Logical volume hd8 updated.

synclvodm: Logical volume hd4 updated.

synclvodm: Logical volume hd2 updated.

synclvodm: Logical volume hd9var updated.

synclvodm: Logical volume hd3 updated.

synclvodm: Logical volume hd1 updated.

synclvodm: Logical volume hd10opt updated.

synclvodm: Logical volume hd7 updated.

synclvodm: Logical volume hd11admin updated.

+ savebase

Hey Presto! The root volume group is now named rootvg just the way we like it!

aixlpar1 : # lspv

hdisk0 00c342c68dfcbdfb rootvg active

hdisk1 00c342c6161c6b47 appvg active

Never say never....AIX 5.3 on POWER8.

cggibbo 7 Comments 9,897 Views

I NEVER thought this would happen! But, “NEVER SAY NEVER!”. IBM just announced that AIX 5.3 can now run on POWER8.

http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?subtype=ca&infotype=an&appname=iSource&supplier=872&letternum=ENUSAP16-0152

As you’d expect, there are some caveats!

A special PTF (U86665.bff) provides POWER8 support on AIX 5.3 with the following limitations:

The LPAR must be at AIX 5.3 TL12 SP9 (latest 5.3 release).
The POWER8 LPAR will run in POWER6 compatibility mode and is limited to SMT2 mode. SMT2 mode results in some capacity loss compared to SMT4/SMT8 mode. IBM publishes SMT2 rPerf values that can be used to quantify Power8 SMT2 capacity.
AIX 5.3 POWER8 LPARs only support Virtual I/O configurations: vSCSI, NIPV, VLAN.
LPM is supported.

The only supported AIX 5.3 POWER8 installation methods are:

mksysb – First perform an in place update to a supported (P5/P6/P7) AIX 5.3 TL12 SP9 LPAR with PTF U866665. Standard mksysb command can then be used to capture a P8 capable mksysb image. The mksysb image can then be used to install POEWR8 LPARs
NIM – An AIX 5.3 TL12 SP9 NIM environment must be updated to support POWER8. An AIX 5.3 TL12 SP9 NIM lpp_source must be updated to include PTF U866665. A NIM SPOT must then be created to utilize the updated lpp_source.

All POWER8 systems are supported with the following restrictions:

POWER8 system must be at 840 firmware level.
POWER8 LPAR must be served by a 2.2.4.10 VIOS.

Other restrictions:

Remote restart is not supported with POWER8 5.3 LPARs
POWER8 AIX 5.3 LPARs are not supported with a VIOS using Shared Storage Pools.

I tested this in my lab and it all worked as expected. I installed the PTF on an existing AIX 5.3 LPAR on a POWER7 750 and rebooted the partition.

[root@53gibbo]/ # lspv

hdisk0 00f603cdcd9ba6de rootvg active

[root@53gibbo]/ # oslevel -s

5300-12-09-1341

[root@53gibbo]/ # lsconf | more

System Model: IBM,8233-E8B

Machine Serial Number: 1003CDP

Processor Type: PowerPC_POWER7

Processor Implementation Mode: POWER 6

Processor Version: PV_6_Compat

Number Of Processors: 2

Processor Clock Speed: 3550 MHz

CPU Type: 64-bit

Kernel Type: 64-bit

LPAR Info: 3 53gibbo

Memory Size: 2048 MB

Good Memory Size: 2048 MB

Platform Firmware level: AL730_142

Firmware Version: IBM,AL730_142

[root@53gibbo]/tmp/cg/aix53p8 # inutoc .

[root@53gibbo]/tmp/cg/aix53p8 # ls -ltr

total 35808

-rw-r----- 1 root system 8994816 Feb 8 09:35 U866665.bff

-rw-r----- 1 root system 8994816 Feb 8 09:37 bos.mp64.5.3.12.10.U

-rw-r--r-- 1 root system 328592 Feb 8 09:37 .toc

[root@53gibbo]/tmp/cg/aix53p8 # update_all -d .

install_all_updates: Initializing system parameters.

install_all_updates: Log file is /var/adm/ras/install_all_updates.log

install_all_updates: Checking for updated install utilities on media.

install_all_updates: Processing media.

install_all_updates: Generating list of updatable installp filesets.

install_all_updates: The following filesets have been selected as updates

to currently installed software:

bos.mp64 5.3.12.10

<< End of Fileset List >>

install_all_updates: Performing installp update.

+-----------------------------------------------------------------------------+

Pre-installation Verification...

+-----------------------------------------------------------------------------+

Verifying selections...done

Verifying requisites...done

Results...

SUCCESSES

---------

Filesets listed in this section passed pre-installation verification

and will be installed.

Selected Filesets

-----------------

bos.mp64 5.3.12.10 # Base Operating System 64-bit...

<< End of Success Section >>

+-----------------------------------------------------------------------------+

BUILDDATE Verification ...

+-----------------------------------------------------------------------------+

Verifying build dates...done

FILESET STATISTICS

------------------

1 Selected to be installed, of which:

1 Passed pre-installation verification

----

1 Total to be installed

installp: bosboot verification starting...

installp: bosboot verification completed.

+-----------------------------------------------------------------------------+

Installing Software...

+-----------------------------------------------------------------------------+

installp: APPLYING software for:

bos.mp64 5.3.12.10

. . . . . << Copyright notice for bos.mp64 >> . . . . . . .

Licensed Materials - Property of IBM

5765G0399

US Government Users Restricted Rights - Use, duplication or disclosure

restricted by GSA ADP Schedule Contract with IBM Corp.

. . . . . << End of copyright notice for bos.mp64 >>. . . .

Finished processing all filesets. (Total time: 3 secs).

installp: bosboot verification starting...

installp: bosboot verification completed.

installp: bosboot process starting...

bosboot: Boot image is 45802 512 byte blocks.

0503-292 This update will not fully take effect until after a

system reboot.

* * * A T T E N T I O N * * *

System boot image has been updated. You should reboot the

system as soon as possible to properly integrate the changes

and to avoid disruption of current functionality.

installp: bosboot process completed.

+-----------------------------------------------------------------------------+

Summaries:

+-----------------------------------------------------------------------------+

Installation Summary

--------------------

Name Level Part Event Result

-------------------------------------------------------------------------------

bos.mp64 5.3.12.10 USR APPLY SUCCESS

* * * A T T E N T I O N * * *

System boot image has been updated. You must reboot the system

after install_all_updates completes to properly integrate the updates.

install_all_updates: Checking for recommended maintenance level 5300-12.

install_all_updates: Executing /usr/bin/oslevel -rf, Result = 5300-12

install_all_updates: Verification completed.

install_all_updates: Log file is /var/adm/ras/install_all_updates.log

install_all_updates: Result = SUCCESS

[root@53gibbo]/tmp/cg/aix53p8 # oslevel -s

5300-12-09-1341

[root@53gibbo]/tmp/cg/aix53p8 # lslpp -l bos.mp64

Fileset Level State Description

----------------------------------------------------------------------------

Path: /usr/lib/objrepos

bos.mp64 5.3.12.10 APPLIED Base Operating System 64-bit

Multiprocessor Runtime

Path: /etc/objrepos

bos.mp64 5.3.11.2 COMMITTED Base Operating System 64-bit

Multiprocessor Runtime

[root@53gibbo]/tmp/cg/aix53p8 # at now

shutdown –Fr

My POWER8 system was an S824 running the latest firmware (840_056), with VIOS running the latest code (2.2.4.10) and the latest HMC code (8.4).

p8vio1:/home/padmin# lsconf | head -20

System Model: IBM,8286-42A

Machine Serial Number: 214F55V

Processor Type: PowerPC_POWER8

Processor Implementation Mode: POWER 7

Processor Version: PV_7_Compat

Number Of Processors: 2

Processor Clock Speed: 3525 MHz

CPU Type: 64-bit

Kernel Type: 64-bit

LPAR Info: 101 82861_vio1

Memory Size: 18432 MB

Good Memory Size: 18432 MB

Platform Firmware level: SV840_056

Firmware Version: IBM,FW840.00 (SV840_056)

On my NIM master I created a new lppsource and SPOT for AIX 5.3 TL12 SP9 and included the required PTF U86665.

# lsnim -t spot | grep 53

spotAIX53TL12SP9 resources spot

# lsnim -t lpp_source | grep 53

AIX53TL12SP9 resources lpp_source

# nim -o showres spotAIX53TL12SP9 | grep bos.mp64

bos.mp64 5.3.12.10 C F Base Operating System 64-bit

# nim -o showres AIX53TL12SP9_P8 | grep bos.mp64

bos.mp64 5.3.11.2 I b usr,root

bos.mp64 5.3.12.9 S b usr

bos.mp64 5.3.12.10 S b usr

I took a mksysb on the 5.3 LPAR to my NIM master. I then used the mksysb to perform an NIM mksysb install (restore) to a new partition on the S824. This worked flawlessly. The partition was configured in POWER6 mode with SMT2 only.

[root@53gibbo]/ # oslevel -s

5300-12-09-1341

[root@53gibbo]/ # lsconf | head -10

System Model: IBM,8286-42A

Machine Serial Number: 214F55V

Processor Type: PowerPC_POWER8

Processor Implementation Mode: POWER 6

Processor Version: PV_6_Compat

Number Of Processors: 2

Processor Clock Speed: 3525 MHz

CPU Type: 64-bit

Kernel Type: 64-bit

LPAR Info: 2 53gibbo

Memory Size: 2048 MB

Good Memory Size: 2048 MB

Platform Firmware level: SV840_056

Firmware Version: IBM,FW840.00 (SV840_056)

Console Login: enable

Auto Restart: true

Full Core: false

[root@53gibbo]/ # smtctl

This system is SMT capable.

SMT is currently enabled.

SMT boot mode is not set.

SMT threads are bound to the same virtual processor.

proc0 has 2 SMT threads.

Bind processor 0 is bound with proc0

Bind processor 1 is bound with proc0

proc2 has 2 SMT threads.

Bind processor 2 is bound with proc2

Bind processor 3 is bound with proc2

[root@53gibbo]/ # lparstat -i

Node Name : 53gibbo

Partition Name : 53gibbo

Partition Number : 2

Type : Shared-SMT

Mode : Uncapped

Entitled Capacity : 0.20

Partition Group-ID : 32770

Shared Pool ID : 0

Online Virtual CPUs : 2

Maximum Virtual CPUs : 4

Minimum Virtual CPUs : 1

Online Memory : 2048 MB

Maximum Memory : 4096 MB

Minimum Memory : 1024 MB

Variable Capacity Weight : 128

Minimum Capacity : 0.10

Maximum Capacity : 4.00

Capacity Increment : 0.01

Maximum Physical CPUs in system : 24

Active Physical CPUs in system : 24

Active CPUs in Pool : 14

Shared Physical CPUs in system : 14

Maximum Capacity of Pool : 1400

Entitled Capacity of Pool : 1160

Unallocated Capacity : 0.00

Physical CPU Percentage : 10.00%

Unallocated Weight : 0

Desired Virtual CPUs : 2

Desired Memory : 2048 MB

Desired Variable Capacity Weight : 128

Desired Capacity : 0.20

The example above shows you how to migrate an AIX 5.3 system using NIM and mksysb. But you don’t have to do this, as LPM is fully supported. The migration effort for many customers is as easy as kicking off an LPM operation to move their (updated) AIX 5.3 partitions from their older Power servers to POWER8. Easy!

So I guess customers that still need to run AIX 5.3 on POWER8 now have an alternative to AIX 5.3 Versioned WPARs. But please keep in mind the performance and support considerations when making your decision.

Additional README information available here:

https://www.ibm.com/developerworks/community/blogs/cgaix/resource/U866665README.pdf

Modified by cggibbo

New AIX Service and Release Strategy - Starting February 2011

cggibbo 1 Comment 10,910 Views

IBM recently announced some significant changes to their ‘AIX Release and Service Delivery’ strategy. Starting February 2011, the following changes will apply for both AIX 6 and AIX 7.

• Three years of support for each Technology Level (TL).

• A single Technology Level per AIX version, per year.

• Service Packs will be released approximately 4 times per year per Technology Level.

Some of the benefits of the new strategy are:

Longer support for Technology Levels.

– More closely matches client deployment requirements.

· Single Technology Level per year, per version, and about one Service Pack per quarter.

– Fewer updates reduce administrative workload.

· New hardware is supported on prior Technology Levels.

– Easier integration of new hardware into existing environments.

You can review the official presentation and details here. I particularly like this FAQ from the presentation.

Q: Why doesn’t the new strategy include AIX V5.3?

A: AIX V5.3 is nearing end of life with the End of Marketing for AIX V5.3 set for April 2011

So if you are on AIX 5.3, start planning a migration to AIX 7.1 (or at least AIX 6.1), NOW! I’ve provided some further information from the presentation below.

Longer Support for each Technology Level

The 2011 Release Strategy extends the support for each Technology Level for up to three years from the introduction of the Technology Level. This means that clients with a Software Maintenance Agreement for the AIX OS will receive IBM defect support during that two year period without having to move up to the latest Technology Level update.

For example, a Technology Level introduced in the second half of 2010 will be supported though the second half of 2013. Since each Technology Level will be supported for three years and IBM plans to release one Technology Level per year, IBM eventually will support up to six Technology Levels for each AIX release. Please note that due to minor variations in release dates, some Technology Levels will be supported for slightly more that three years and some will be supported for slightly less than three years. A three year service life for each Technology Level is an objective, not an absolute limit.

Fewer Technology Levels and Fewer Service Packs

Historically, new Technology Levels were the only way to deliver new hardware support to clients. Since 2007, we have supported most new hardware on previous Technology Levels via a service pack. Since this new approach to supporting new hardware has been well proven with the introduction of POWER7 in 2010, we decided to eliminate the “Spring” Technology Level which reduces the number of updates clients have to deal with. The number of Service Packs has been about 5 per year and we want to reduce this number to about once per quarter.

Between Technology Level releases, clients maintain their AIX operating systems by installing Service Packs (SP) or Interim Fixes for the entire support life of the Technology Level update. Service Packs are also used to provide support for newly released hardware.

The release dates and frequency of Service Packs are variable due to many factors. including new Technology Level releases, new hardware introduction and the need to deliver software fixes.

In the past IBM delivered new Service Packs about every eight to twelve weeks or about five to eight times per year per Technology Level. With the 2011 release strategy, IBM intends to reduce the number of Service Packs released per year, per AIX release to about four. As stated earlier, there are many factors that drive the release of a Service Pack, so releasing four Service Packs per year, per Technology Level represents a goal and objective, but not an absolute limit.

Note that new Technology Levels are almost always accompanied very closely by the first Service Pack for that new Technology Level. This first Service Pack includes fixes to problems that are discovered between the time the new Technology Level is released to manufacturing for the media to be replicated and the time the new Technology Level is actually available to clients.

Summary

The providers of all software, including platform software such as the AIX operating system, must balance the client’s need for stability against the need to enhance the software to provide new functionality. The changes introduced with the AIX OS Release and Service Strategy of 2011 balances those two conflicting goals to help provide our clients with significant improvements in the manageability of the AIX operating system.

The increased service life, better manageability and support for new hardware on older fix levels provided by the new strategy are in direct response to our client’s requirements.

As stated earlier, this new strategy represents goals and objectives of IBM and is subject to change. Our clients understand that substantial changes such as this new strategy are an indication of the commitment that IBM has to improving the market leading capabilities of the AIX operating system.

For more information and best practices please refer to:

https://www14.software.ibm.com/webapp/set2/sas/f/best/home.html

New FC adapter port queue depth tuning

cggibbo 4 Comments 9,911 Views

# fcstat –D fcs0

FIBRE CHANNEL STATISTICS REPORT: fcs0

…

FC SCSI Adapter Driver Queue Statistics

Number of active commands: 0

High water mark of active commands: 17

Number of pending commands: 0

High water mark of pending commands: 1

Number of commands in the Adapter Driver Held off queue: 0

High water mark of number of commands in the Adapter Driver Held off queue: 0

…

The –D is a new option on fcstat, not yet documented in the man pages.

Set num_cmd_elems to >= high water mark of active commands or the default, whichever is greater.

# chdev –l fcs0 –a num_cmd_elems=4096 –a max_xfer_size=0x200000 –P

; requires a reboot

Thanks to Dan Braden (IBM ATS) for this tip.

Modified by cggibbo

New Features in AIX Version 7.

cggibbo 2 Comments 11,091 Views

Today, IBM announced AIX version 7. http://www-03.ibm.com/systems/power/software/aix/v71/preview.html

Several new features were mentioned in the launch, but there were two new features that I found particularly interesting:

- AIX 5.2 WPARs for AIX 7

- Cluster Aware AIX

I thought I would briefly describe each feature in this post.

AIX 5.2 WPARs for AIX 7

In AIX version 7, administrators will now have the capability to create Workload Partitions (WPARs) that can run AIX 5.2, inside an AIX 7 operating system instance. This will be supported on the POWER7 server platform. This is pretty cool. IBM have done this to allow some customers, that are unable to migrate to later generations of AIX and Power, to move to POWER7 whilst keeping their legacy AIX 5.2 systems operational. So for those clients that MUST stay on AIX 5.2 (for various reasons such as Application support) but would like to run their systems on POWER7, this feature may be very attractive. It will help to reduce the effort required when consolidating older AIX 5.2 systems onto newer hardware. It may also reduce some of the risk associated with migrating applications from one version of the AIX operating system to another.

To migrate an existing AIX 5.2 system to an AIX 7 WPAR, administrators will first need to take a mksysb of the existing system. Then they can simply restore the mksysb image inside the AIX 7 WPAR. IBM will also offer limited defect and how-to support for the AIX 5.2 operating system in an AIX 7 WPAR. These WPARs can, of course, be managed via IBM Systems Director with the Workload Partitions Manager plug-in.

The following figure provides a visualization of how these AIX 5.2 systems will fit into an AIX 7 WPAR. The WPARs in blue are native AIX 7 WPARs, while the WPARs in orange are AIX 5.2 WPARs running in the same AIX 7 instance. Pretty amazing really!

I can only speculate as to what other versions of AIX will be supported in this manner in the future. Just imagine an AIX 5.3 WPAR inside AIX7?

Cluster Aware AIX

Another very interesting feature of AIX 7 is a new technology known as “Cluster Aware AIX”. Believe it or not, administrators will now be able to create a cluster of AIX systems using features of the new AIX 7 kernel. IBM have introduced this “in built” clustering to the AIX OS in order to simplify the configuration and management of highly available clusters. This new AIX clustering has been designed to allow for:

- The easy creation of clusters of AIX instances for scale-out computing or high availability.

- Significantly simplify cluster configuration, construction, and maintenance.
- Improve availability by reducing the time to discover failures.

- Capabilities such as common device naming to help simplify administration.

- Built in event management and monitoring.

- A foundation for future AIX capabilities and the next generation of PowerHA SystemMirror.

This does not replace PowerHA but it does change the way in which AIX traditionally integrates with cluster software like HACMP and PowerHA. A lot of the HA cluster functionality is now available in the AIX 7 kernel itself. However, the mature RSCT technology is still a component of the AIX and PowerHA configuration. I’m looking forward to reading more about this new technology and it’s capabilities.

These are just two of the many features introduced in AIX 7. I’m eagerly looking forward to what these features and others mean for the future of the AIX operating system. It’s exciting to watch this operating system grow and strengthen over time. I can’t wait to get my hands on an AIX 7 system so that I can trial these new features.

And speaking of trialing AIX 7, there is good news. IBM plan on running another AIX Open Beta program for AIX 7 mid 2010. Just as they did with AIX Version 6, customers will be given the opportunity to download a beta version of AIX 7 and trial it on their own systems in their own environment. This is very exciting and I’m really looking forward to it.

I encourage you to read the official AIX 7 announcement to learn more about the future of the AIX operating system and what it can do for you and your organization…..for many years to come!

New HMC commands: chnportlogin and lsnportlogin - Login in Virtual Fibre Channel adapters for NPIV client LPARs.

cggibbo Tags: virtual chnportlogin aix gibson lsnportlogin channel npiv adapter chris fibre 4 Comments 37,281 Views

There are two new HMC (V7.7.3.0) commands that can force a client Virtual Fibre Channel adapter to log into a SAN. This should make the life of the AIX and SAN administrator easier, as they will no longer need to install AIX in order for the new VFC adapters to log into the SAN. Although there was an unsupported method* for doing this already (see links below). Nor will the SAN admins need to “blind” zone the WWPNs.

There was some indication of this in the latest VIOS FP readme:

https://www-304.ibm.com/support/docview.wss?rs=0&uid=isg400000693

· Enabled SAN login from VIOS

And there was this:

IZ95569: ENABLE SAN LOGIN FROM VIOS FOR IMPROVED UABILITY

https://www-304.ibm.com/support/docview.wss?uid=isg1IZ95569

Problem summary

· Customer will be able to instruct the VIOS to login to the SAN

· for a given WWPN on a given virtual adapter to allow the

· customer to see the WWPN in SAN management tools and easing

· the task of configuring new NPIV client

· partitions.

Problem conclusion

· add commands to VIOS manager interface to allow the HMC to

· login, logout and query virtual adapters.

And finally this:

http://www-304.ibm.com/webapp/set2/sas/f/hmcl/power7/v7730.Readme.html#enhance7730

The new lsnportlogin and chnportlogin commands on the 7.730 HMC provide the ability to utilize the new function in the VIOS. From the Readme:

Added the chnportlogin and lsnportlogin commands.

1. The chnportlogin command allows you to perform N Port login and logout operations for virtual fibre channel client adapters that are configured in a partition or a partition profile. Use this command to help you in zoning WWPNs on a Storage Area Network (SAN). A login operation activates all inactive WWPNs, including the second WWPN in the pair assigned to each virtual fibre channel client adapter. This feature is particularly useful for Logical Partition Migration. A logout operation deactivates all WWPNs not in use. A successful login of a virtual fibre channel adapter requires that the corresponding virtual fiber channel server adapter must exist and that it must be mapped.

2. The lsnportlogin command also allows you to list WWPN login status information for virtual fibre channel client adapters configured in partitions or partition profiles.

I found a lot of this information on the IBM developerWorks PowerVM forum:

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=388796

Thanks to http://twitter.com/Kristijan for pointing me in the direction of the PowerVM forum.

Here’s an example of using the lsnportlogin command on one of my systems:

hscroot@HMC1:~> lsnportlogin -m 770-1-Rack_D191 --filter "profile_names=normal"

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=32,wwpn=c050760405f0000c,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=32,wwpn=c050760405f0000d,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=33,wwpn=c050760405f0000e,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=33,wwpn=c050760405f0000f,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=30,wwpn=c050760405f00000,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=30,wwpn=c050760405f00001,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=31,wwpn=c050760405f00002,wwpn_status=0

lpar_name=nim1,lpar_id=3,profile_name=normal,slot_num=31,wwpn=c050760405f00003,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=32,wwpn=c050760405f00004,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=32,wwpn=c050760405f00005,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=33,wwpn=c050760405f00006,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=33,wwpn=c050760405f00007,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=30,wwpn=c050760405f00008,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=30,wwpn=c050760405f00009,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=31,wwpn=c050760405f0000a,wwpn_status=0

lpar_name=aix01adm,lpar_id=4,profile_name=normal,slot_num=31,wwpn=c050760405f0000b,wwpn_status=0

Descriptions of selected command attributes:

wwpn_status

The WWPN status. Possible values are:

0 - WWPN is not activated

1 - WWPN is activated

2 - WWPN status is unknown

Some related links:

lsnportlogin

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7edm/lsnportlogin.html

chnportlogin

http://publib.boulder.ibm.com/infocenter/powersys/v3r1m5/topic/p7edm/chnportlogin.html

*How to force a vfc-client device to log in to the SAN (The OLD way)!

https://www.ibm.com/developerworks/wikis/display/virtualization/vfclogin

How to Capture SAN Boot Debug for Virtual I/O Server and AIX on P6 Systems

~~https://www-304.ibm.com/support/docview.wss?uid=isg3T1012199~~

http://unixadmin.free.fr/?p=245

Disk path design for AIX including SAN zoning

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101914

Modified by cggibbo

New HMC system wide setting - Allow Migration with Inactive Source Storage VIOS

cggibbo 3,243 Views

A new HMC & Firmware 840 feature allows LPM in a dual VIOS configuration when one VIOS is failed. Previously, LPM was not allowed if one of the source VIOS (in a dual VIOS configuration) was in a failed state. Both VIOSes had to be operational to perform LPM. The new support allows the HMC to cache adapter configuration from both VIOSes. Whenever changes are made to the configuration, the cached information will be updated on the HMC. If one VIOS is failed, instead of querying the failed VIOS, the HMC cache is used instead to create the new configuration on the target VIOS. This support was needed to cover the situation where there’s failed hardware which is causing an outage on the VIOS and requires a disruptive repair action. This new feature is enabled using a server wide HMC setting to enable the automatic caching of VIOS configuration details.

You can enable/disable this setting using the new Enhanced+ HMC interface. You’ll find it under General Properties of the Managed System (Server).

You can also enabled/disable this feature using the HMC command line, with migrlpar.

$ migrlpar -o set -m <managed system> -r sys -i “allow_inactive_source_storage_vios=1/0”

$ lslparmigr -r sys -m <managed system> -F allow_inactive_source_storage_vios

Whilst we’re on the subject of new HMC features; you may have heard that it is now possible to disable LPM on a per partition basis. You read more about this feature here:

https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/HMC%20per-partition%20disablement%20of%20Live%20Partition%20Mobility%20%28LPM%29

This feature can be enabled/disabled, per partition, using the HMC Enhanced+ interface. You’ll find this under General Properties of a Partition.

You can also use the HMC command, chsyscfg, to toggle this setting.

$ chsyscfg -r lpar -m Server-8286-42A-SN991F99V -i "name=RHEL_6_5,migration_disabled=1"

$ lssyscfg -r lpar -m Server-8286-42A-SN991F99V -F name,migration_disabled

Useful links and additional information.

Live Partition Mobility (LPM) improvements in PowerVM 2.2.4

https://www.ibm.com/developerworks/community/wikis/home?lang=en_us#!/wiki/Power%20Systems/page/Live%20Partition%20Mobility%20%28LPM%29%20improvements%20in%20PowerVM%202.2.4

HMC V8 R8.4.0 Command Line specification

https://www-304.ibm.com/webapp/set2/sas/f/hmcl/power8/hmc_man_8.4.0.pdf

HMC Manual Reference Pages - MIGRLPAR (1)

https://www.ibm.com/support/knowledgecenter/HW4P4/p8edm/migrlpar.html

IBM Power Systems HMC Implementation and Usage Guide

http://www.redbooks.ibm.com/redpieces/pdfs/sg248334.pdf

Hardware Management Console - Related technical information

https://www-304.ibm.com/webapp/set2/sas/f/hmcl/resources.html

New resolv.conf options in AIX 6.1

cggibbo 2 Comments 14,342 Views

FYI.

There are new options for /etc/resolv.conf in AIX 6.1 (not in 5.3):

...
timeout
        Enables you to specify the initial timeout for a query to a nameserver. The default value is five seconds. The maximum value is 30 seconds. For
        the second and successive rounds of queries, the resolver doubles the initial timeout and is divided by the number of nameservers in the
        resolv.conf file.

attempts
Enables you to specify how many queries the resolver should send to each nameserver in the resolv.conf file before it stops execution. The
default value is 2. The maximum value is 5.

rotate
Enables the resolver to use all the nameservers in the resolv.conf file, not just the first one.
...

New VIOS tunables with v2.2.2.2.

cggibbo 9,170 Views

New SEA thread tuning parameters with VIOS level 2.2.2.2

Number of threads (nthreads)

Specifies the number of threads in threaded mode, where the value of the thread parameter is 1. This value applies only when the thread mode is enabled. The nthreads attribute can be set to any value between 1 and 128. The default value is 7.

Queue size (queue_size)

Specifies the queue size for the Shared Ethernet Adapter (SEA) threads in threaded mode where the value of the thread parameter is 1. This attribute indicates the number of packets that can be accommodated in each thread queue. This value applies only when the thread mode is enabled. When you change this value, the change does not take effect until the system restarts. The queue_size attribute can be set to any value between 2 and 65535. The default value is 8192.

Hash algorithms (hash_algo)

Specifies the hash algorithm that is used to assign connections to Shared Ethernet Adapter (SEA) threads in threaded mode, where the value of the thread parameter is 1. When the hash_algo parameter is set to 0 (the default), an addition operation of the source and destination Media Access Control (MAC) addresses, IP addresses, and port numbers is done. When the hash_algo parameter is set to 1, a murmur3 hash function is done instead of an addition operation. The murmur3 hash function is slower, but it achieves better distribution. This value applies only when the thread mode is enabled.

$ lsdev -dev ent6 -attr

attribute value description user_settable

accounting enabled Enable per-client accounting of network statistics True

ctl_chan Control Channel adapter for SEA failover True

gvrp no Enable GARP VLAN Registration Protocol (GVRP) True

ha_mode disabled High Availability Mode True

hash_algo 0 Hash algorithm used to select a SEA thread True

jumbo_frames no Enable Gigabit Ethernet Jumbo Frames True

large_receive no Enable receive TCP segment aggregation True

largesend 0 Enable Hardware Transmit TCP Resegmentation True

lldpsvc no Enable IEEE 802.1qbg services True

netaddr 0 Address to ping True

nthreads 7 Number of SEA threads in Thread mode True

pvid 50 PVID to use for the SEA device True

pvid_adapter ent4 Default virtual adapter to use for non-VLAN-tagged packets True

qos_mode disabled N/A True

queue_size 8192 Queue size for a SEA thread True

real_adapter ent0 Physical adapter associated with the SEA True

thread 1 Thread mode enabled (1) or disabled (0) True

virt_adapters ent4 List of virtual adapters associated with the SEA (comma separated) True

# lsattr -Rl ent6 -a hash_algo

0...1 (+1)

# lsattr -Rl ent6 -a nthreads

1...128 (+1)

# lsattr -Rl ent6 -a queue_size

2...65535 (+1)

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=%2Fp7hb1%2Fiphb1_vios_managing_sea_attr.htm

New LPM tuning parameters with VIOS level 2.2.2.2.

A new pseudo device is available to support Live Partition Mobility performance tuning.

The pseudo device vioslpm0 is created by default when VIO server version 2.2.2.0 or higher is installed.
Device attributes for vioslpm0 can be used to control live partition mobility operations.
Use normal lsdev/chdev padmin commands to query/change attributes.

$ lsdev –dev vioslpm0 –attr

$ chdev –dev vioslpm0 –attr cfg_msp_ops=5

$ lsdev -dev vioslpm0 -attr

attribute value description user_settable

cfg_msp_lpm_ops 8 Configured number of concurrent LPM operations for this MSP True

concurrency_lvl 3 Concurrency level True

lpm_msnap_succ 1 Create mini-snap for successful migrations True

max_lpm_vasi 1 Maximum number of VASI adapters used for LPM operations False

max_vasi_ops 8 Maximum number of concurrent LPM operations per VASI False

tcp_port_high 0 TCP highest ephemeral port True

tcp_port_low 0 TCP lowest ephemeral port True

Pseudodevice attributes and definition
Attribute	Value	Description	User modifiable
cfg_msp_lpm_ops	8	Number of concurrent partition mobility operations for the mover service partition	True
concurrency_lvl	3	Concurrency level	True
lpm_msnap_succ	1	Create a mini-snap (when a migration ends, the set of information related to a specific migration, that is gathered and packed on each mover service partition involved in the migration), for successful migrations	True
max_lpm_vasi	1	Maximum number of Virtual Asynchronous Services Interface (VASI) adapters used for partition mobility operations	False
max_vasi_ops	8	Maximum number of concurrent partition mobility operations per VASI	False
tcp_port_high	0	TCP highest ephemeral port	True
tcp_port_low	0	TCP lowest ephemeral port	True

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/topic/p7hc3/iphc3vioslpmpseudo.htm

New features of the migrlpar command with VIOS level 2.2.2.2.

Specify fibre channel ports using vios_fc_port_name. Run the lslparmigr command to show a list of available slot IDs for a VIOS partition. Run the migrlpar command to accomplish the following tasks:

Specify virtual slot IDs for one or more virtual adapter mappings.
Validate the specified slot IDs.
Note: You can specify the port name of the Fibre Channel to be used for creating Fibre Channel mapping on the source server when you are performing partition migration.

You can use the HMC command line interface to specify the port name. List all the valid port names of the Fibre Channel by running the lsnports command. From the list of the valid port names, specify the port name that you want to use by running the migrlpar command with the attribute

vios_fc_port_name specifying the port name you want to use.

The following attributes of pseudo device can be modified by using the migrlpar command:

num_active_migrations_configured
concurr_migration_perf_level

Run the following HMC command to modify the attribute values of the pseudo device, for example to set the number of active migrations to 8 run:

$ migrlpar -o set -r lpar -m 795-1 -p lpar1 -i "num_active_migrations_configured=8“

The default value for this attribute is 4. To run the maximum number of supported partition mobility operations on the VIOS, this value must be set the supported maximum number.

To set the amount of resources allocated for each mobility operation to a value of 2, run the following command:

$ migrlpar -o set -r lpar -m 795-1 -p lpar1 -i "concurr_migration_perf_level=2"

The range of the concurr_migration_perf_level attribute value is 1 to 5. A value of 1 indicates optimal performance, and a value of 5 indicates limited resources. The default value is 3.

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=%2Fp7hc3%2Fiphc3hmclpmpseudo.htm

New LPM error codes.

There are new error codes available for troubleshooting partition mobility:

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=%2Fp7hc3%2Fiphc3hmctroubleshoot.htm

http://pic.dhe.ibm.com/infocenter/powersys/v3r1m5/index.jsp?topic=%2Fp7hc3%2Fiphc3vioserror.htm

VIOS error codes
Error Code	Definition
1	The virtual adapter is not ready to be moved. The source virtual Ethernet is not bridged.
2	The virtual adapter can be moved with less capability. All virtual local area networks (VLAN) are not bridged on the destination. Hence, the virtual Ethernet adapter has less capability on the target system compared to the source system.
3	The stream ID is still in use.
64	The migmgr command cannot be started.
65	The stream ID is invalid.
66	The virtual adapter type is invalid.
67	The virtual adapter DLPAR resource connector (DRC) name is not recognized.
68	The virtual adapter method cannot be started, or it was prematurely terminated.
69	There is a lack of resources (that is, the ENOMEM error code).
80	The storage that is being used by the adapter is specific to the VIOS and cannot be accessed by another VIOS. Hence, the virtual adapter cannot complete the mobility operation.
81	The virtual adapter is not configured.
82	The virtual adapter cannot be placed in a migration state.
83	The virtual devices are not found.
84	The virtual adapter VIOS level is insufficient.
85	The virtual adapter cannot be configured.
86	The virtual adapter is busy and cannot be unconfigured.
87	The virtual adapter or device minimum patch level is insufficient.
88	The device description is invalid.
89	The command argument is invalid.
90	The virtual target device cannot be created because of incompatible backing device attributes. Typically, this is because of a mismatch in the maximum transfer (MTU) size or SCSI reserve attributes of the backing device between the source VIOS and the target VIOS.
91	The DRC name passed to the migration code is for an adapter that exists.

Modified by cggibbo

Feed for Blog Entries ▼ Feed for Blog Comments ▼

▼ Similar Blogs

▼ Similar Ideation Blogs

▼ Archive

▼ Blog Authors

Chris's AIX Blog

IZ63357: LPM HUNG AT 100% ON BOTH SOURCE AND DESTINATION IVMS

Error description

Local fix

NIM from A to Z in AIX 5L Redbook.