Chris's AIX Blog

with Tags: lvm

Using the AIX splitvg command.

cggibbo Tags: aix lvm 1 Comment 20,925 Views

Just the other day, I needed to use the AIX splitvg command in order to copy some data from one system to another.

I thought I’d share the experience here.

The splitvg command can split a single mirror copy of a fully mirrored volume group into a separate “snapshot” volume group.

From the man page:

The original volume group VGname will stop using the disks that are now part of the snapshot volume group SnapVGname. Both volume groups will keep track of the writes within the volume group so that when the snapshot volume group is rejoined with the original volume group consistent data is maintained across the rejoined mirrors copies. Notes:

1         To split a volume group, all logical volumes in the volume group must have the target mirror copy and the mirror must exist on a disk or set of disks. Only the target mirror copy must exist on the target disk or disks.

2         The splitvg command will fail if any of the disks to be split are not active within the original volume group.

3         In the unlikely event of a system crash or loss of quorum while running this command, the joinvg command must be run to rejoin the disks back to the original volume group.

4         There is no concurrent or enhanced concurrent mode support for creating snapshot volume groups.

5         New logical volumes and file system mount points will be created in the snapshot volume group.

6         The splitvg command is not supported for the rootvg.

7         The splitvg command is not supported for a volume group that has an active paging space.

8         When the splitvg command targets a concurrent-capable volume group which is varied on in non-concurrent mode, the new volume group that is created will not be varied on when the splitvg command completes. The new volume group must be varied on manually.

So, looking at point 4, above, if you are using enhanced concurrent volume groups (for example with PowerHA), then you will not be able to use the splitvg command. This is disappointing as this would have been very handy in some of the large PowerHA systems I have worked with…..perhaps this will be supported in the future?
Anyway, back to my example. I had wanted to break-off one of the mirrors of a mirrored volume group and then assign the “split” volume group to another host to copy some data off.
The volume group datavg contained two disks, hdisk0 and hdisk3, as shown in the lspv output below.

# lspv
hdisk1          00c01c705bdc6136                    old_rootvg
hdisk0          00c01c70c050810f                    datavg              active
hdisk2          00c01c7018a47201                    rootvg              active
hdisk3          00c01c70fed9e41a                    datavg              active

There were only two logical volumes (loglvl00, jfs2 log and fslv00, data) and a single file system (/data) in this volume group.

# lsvg -l datavg
datavg:
LV NAME             TYPE       LPs     PPs     PVs LV STATE      MOUNT POINT
loglv00             jfs2log    1       2       2    open/syncd    N/A
fslv00              jfs2       16      32      2    open/syncd    /data

The volume group datavg, was mirrored across hdisk0 and hdisk3, as shown in the lsvg output below.

# lsvg -p datavg
datavg:
PV_NAME           PV STATE          TOTAL PPs   FREE PPs    FREE DISTRIBUTION
hdisk0            active            583         566         117..100..116..116..117
hdisk3            active            583         566         117..100..116..116..117

The logical volumes for the /data file system and the JFS2 log were both mirrored, as shown in the lslv/lspv output.

# lslv fslv00
LOGICAL VOLUME:     fslv00                 VOLUME GROUP:   datavg
LV IDENTIFIER:      00c01c7000004c0000000122e9038480.2 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2                   WRITE VERIFY:   off
MAX LPs:            512                    PP SIZE:        256 megabyte(s)
COPIES:             2                      SCHED POLICY:   parallel
LPs:                16                     PPs:            32
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    32
MOUNT POINT:        /data                    LABEL:          /data
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO

# lslv loglv00
LOGICAL VOLUME:     loglv00                VOLUME GROUP:   datavg
LV IDENTIFIER:      00c01c7000004c0000000122e9038480.1 PERMISSION:     read/write
VG STATE:           active/complete        LV STATE:       opened/syncd
TYPE:               jfs2log                WRITE VERIFY:   off
MAX LPs:            512                    PP SIZE:        256 megabyte(s)
COPIES:             2                      SCHED POLICY:   parallel
LPs:                1                      PPs:            2
STALE PPs:          0                      BB POLICY:      relocatable
INTER-POLICY:       minimum                RELOCATABLE:    yes
INTRA-POLICY:       middle                 UPPER BOUND:    32
MOUNT POINT:        N/A                    LABEL:          None
MIRROR WRITE CONSISTENCY: on/ACTIVE
EACH LP COPY ON A SEPARATE PV ?: yes
Serialize IO ?:     NO

# lspv -l hdisk0
hdisk0:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
fslv00                16      16      00..16..00..00..00    /data
loglv00               1       1       00..01..00..00..00    N/A

# lspv -l hdisk3
hdisk3:
LV NAME               LPs     PPs     DISTRIBUTION          MOUNT POINT
fslv00                16      16      00..16..00..00..00    /data
loglv00               1       1       00..01..00..00..00    N/A

Using the splitvg command I was able to break-off one of the disks from the mirrored pair. This created a new volume group on hdisk3 called vg00.

# splitvg -c1 datavg
# lspv
hdisk1          00c01c705bdc6136                    old_rootvg
hdisk0          00c01c70c050810f                    datavg            active
hdisk2          00c01c7018a47201                    rootvg          active
hdisk3          00c01c70fed9e41a                    vg00            active

The new volume group contains a new logical volume (pre-fixed with fs i.e. fsfslv00) and a file system (pre-fixed with /fs i.e. /fs/data). I can mount this file system and access the data in the file system and create and/or modify files (as shown below).

# mount /fs/data
Replaying log for /dev/fsfslv00.
# cd /fs/data
# ls -ltr
total 112
drwxrwxrwx    2 root     system        53248 Jan 30 2009 AIX61TL2SP2
drwxr-xr-x    2 root     system          256 Aug 05 15:24 lost+found
-rw-r--r--    1 root     system            0 Sep 15 10:29 2
# touch 3
# ls -ltr
total 112
drwxrwxrwx    2 root     system        53248 Jan 30 2009 AIX61TL2SP2
drwxr-xr-x    2 root     system          256 Aug 05 15:24 lost+found
-rw-r--r--    1 root     system            0 Sep 15 10:29 2
-rw-r--r--    1 root     system            0 Sep 15 10:46 3

At this point I was able to export the volume group and import it on another system. I had to re-map the Virtual SCSI disk on my VIOS first.

# cd
# umount /fs/data
# varyoffvg vg00
# exportvg vg00
# rmdev –dl hdisk3
; Re-map the disk at the VIOS layer to the other AIX LPAR.
; Configure the disk on the other LPAR (with cfgmgr) and import the volume group (with importvg).

Once I was finished with vg00 on the other LPAR, I re-mapped the disk to the original AIX LPAR, configured the disk and then re-joined the disk to the datavg volume group, with the joinvg command.

; Export the VG and remove the disk on the other AIX LPAR.
; Re-map the disk at the VIOS layer to the original AIX LPAR.
# cfgmgr
# lsdev –Cc | grep hdisk3
hdisk3 Available Virtual SCSI Disk Drive

# joinvg datavg
# lspv
hdisk1          00c01c705bdc6136                    old_rootvg
hdisk0          00c01c70c050810f                    datavg            active
hdisk2          00c01c7018a47201                    rootvg          active
hdisk3          00c01c70fed9e41a                    datavg            active

The volume group is now fully mirrored again. However the partitions are stale and need to be sync’ed. I manually re-sync the volume group with the syncvg command.

# lsvg -l datavg
datavg:
LV NAME             TYPE       LPs     PPs     PVs LV STATE      MOUNT POINT
loglv00             jfs2log    1       2       2    open/stale    N/A
fslv00              jfs2       16      32      2    open/stale    /data
# syncvg –v datavg
# lsvg -l datavg
datavg:
LV NAME             TYPE       LPs     PPs     PVs LV STATE      MOUNT POINT
loglv00             jfs2log    1       2       2    open/syncd    N/A
fslv00              jfs2       16      32      2    open/syncd    /data

More information can be found here: http://www-01.ibm.com/support/docview.wss?uid=isg3T1010934

Working with AIX scalable volume groups.

cggibbo Tags: gibson aix vg manager lvm pv chris chvg hdisk max physical lv scalable groups group mkvg logical volume 4 Comments 37,015 Views

Scalable volume groups first appeared in AIX 5.3. They allow a maximum of 1024 physical volumes and 4096 logical volumes per volume group. This is pretty cool for large systems. In the ‘old days’ we could end up in situations where we’d reached the maximum PV count (32 or 128) for a VG and have to create a new VG to work around the issue.

I use scalable VGs for all my new large AIX system builds.

The following command will create a scalable VG (named svg) on hdisk4. You’ll notice that MAX PVs is set to 1024 and MAX LVs is set to 256.

# mkvg -S -ysvg hdisk4

svg

# lsvg svg

VOLUME GROUP: svg VG IDENTIFIER: 00f6048800004c0000000133336703a0

VG STATE: active PP SIZE: 32 megabyte(s)

VG PERMISSION: read/write TOTAL PPs: 1597 (51104 megabytes)

MAX LVs: 256 FREE PPs: 1597 (51104 megabytes)

LVs: 0 USED PPs: 0 (0 megabytes)

OPEN LVs: 0 QUORUM: 2 (Enabled)

TOTAL PVs: 1 VG DESCRIPTORS: 2

STALE PVs: 0 STALE PPs: 0

ACTIVE PVs: 1 AUTO ON: yes

MAX PPs per VG: 32768 MAX PVs: 1024

LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no

HOT SPARE: no BB POLICY: relocatable

PV RESTRICTION: none

So, MAX LVs is set to 256 by default in a scalable VG. Fine, but what if I need to grow higher than that? Well, we can use the chvg command to set MAX LVs to a higher number, such as 4096 (the maximum).

# chvg -v 4096 svg

We made this change while the VG was varied on and file systems are open.....very nice indeed!

# lsvg svg

VOLUME GROUP: svg VG IDENTIFIER: 00f6048800004c0000000133336703a0

VG STATE: active PP SIZE: 32 megabyte(s)

VG PERMISSION: read/write TOTAL PPs: 1597 (51104 megabytes)

MAX LVs: 4096 FREE PPs: 1597 (51104 megabytes)

LVs: 0 USED PPs: 0 (0 megabytes)

OPEN LVs: 0 QUORUM: 2 (Enabled)

TOTAL PVs: 1 VG DESCRIPTORS: 2

STALE PVs: 0 STALE PPs: 0

ACTIVE PVs: 1 AUTO ON: yes

MAX PPs per VG: 32768 MAX PVs: 1024

LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no

HOT SPARE: no BB POLICY: relocatable

PV RESTRICTION: none

l488pp062_pub[/] >

From the chvg man page:

# man chvg

.....

-v LogicalVolumes

Increases the number of logical volumes that can be created. Valid values are 512, 1024, 2048 and 4096. The value should be larger than the current value or no action is taken. This option is only valid with Scalable-type volume groups. This option is not available for volume groups varied on in concurrent mode.

Once we’ve changed MAX LVs to a higher value, we can’t go backwards. For example, if we try to change MAX LVs to a lower value, we receive the following message:

# chvg -v 256 svg

0516-1617 lchangevg: 256 is an invalid value for maxlvs, the value must be greater

than the current value, 4096, and cannot be greater than the limit, 4096.

0516-732 chvg: Unable to change volume group svg.

If we attempted to set MAX LVs to a higher value than what’s supported we’d receive the following message:

# chvg -v 8192 svg

0516-1614 lchangevg: The valid values for maxlvs include 256, 512, 1024,

2048, and 4096. This option is only usable for scalable

volume groups.

Usage: lchangevg -g VGid -p PVid [-r RemoveMode] [-a AllocateMode]

0516-732 chvg: Unable to change volume group svg.

Working with AIX scalable volume groups - Part 2 - converting to scalable.

cggibbo Tags: aix lvm chvg pv gibson convert -g mkvg manager hdisk scalable chris conversion group groups volume max vg physical lv logical 29,916 Views

Following on from my previous post, on working with scalable VGs.

https://www.ibm.com/developerworks/mydeveloperworks/blogs/cgaix/entry/working_with_aix_scalable_volume_groups1?lang=en

You may find yourself in a situation where you’d like to convert a VG to scalable.

For example, you might start off with a normal VG i.e. one that can support a maximum of 256 LVs and 32 PVs.

# mkvg -s 512M -y myvg hdisk5

myvg

# lsvg myvg

VOLUME GROUP: myvg VG IDENTIFIER: 00f6027300004c0000000133aa24d1f6

VG STATE: active PP SIZE: 512 megabyte(s)

VG PERMISSION: read/write TOTAL PPs: 99 (50688 megabytes)

MAX LVs: 256 FREE PPs: 0 (0 megabytes)

LVs: 1 USED PPs: 99 (50688 megabytes)

OPEN LVs: 1 QUORUM: 2 (Enabled)

TOTAL PVs: 1 VG DESCRIPTORS: 2

STALE PVs: 0 STALE PPs: 0

ACTIVE PVs: 1 AUTO ON: yes

MAX PPs per VG: 32512

MAX PPs per PV: 1016 MAX PVs: 32

LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no

HOT SPARE: no BB POLICY: relocatable

PV RESTRICTION: none

Sometime later you discover you need to grow the volume group and add more hdisks. But you can’t because you’ve reached the maximum PV limit of 32.

You examine the chvg man page and discover the –G flag! Of course you take note of the special considerations when converting to scalable.

Changes the volume group to Scalable VG format. This can accommodate up to 1024 physical volumes and 4096 logical volumes. Notes:

1 The -G flag cannot be used if there are any stale physical partitions.

2 Once the volume group is converted, it cannot be imported into AIX(R) 5.2 or lower versions.

3 The -G flag cannot be used if the volume group is varied on.

4 There must be enough free partitions available on each physical volume for the VGDA expansion for this operation to be successful.

5 Since the VGDA resides on the edge of the disk and it requires contiguous space for expansion, the free partitions are required on the edge of the disk. If those partitions are allocated for user usage, they will be migrated to other free partitions on the same disk. The rest of the physical partitions will be renumbered to reflect the loss of the partitions for VGDA usage. This will change the mappings of the logical to physical partitions in all the PVs of this VG. If you have saved the mappings of the LVs for a potential recovery operation, you should generate the maps again after the completion of the conversion operation. Also, if the backup of the VG is taken with the map option and you plan to restore using those maps, the restore operation may fail since the partition number may no longer exist (due to reduction). It is recommended that backup is taken before the conversion, and right after the conversion if the map option is utilized.

6 Because the VGDA space has been increased substantially, every VGDA update operation (creating a logical volume, changing a log ical volume, adding a physical volume, and so on) may take considerably longer to run.

Excellent, I can convert my VG to scalable.

The volume group must be varied off before it can be converted.

# chvg -G myvg

0516-1707 chvg: The volume group must be varied off during conversion to

scalable volume group format.

0516-732 chvg: Unable to change volume group myvg.

# umount /myfs

# lsvg -l myvg

myvg:

LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT

mylv jfs2 99 99 1 closed/syncd /myfs

# varyoffvg myvg

There must be enough free PPs in the volume group (on each hdisk) in order for the conversion to work. If there aren’t then you’ll see this error:

# chvg -G myvg

0516-1214 chvg: Not enough free physical partitions exist on hdisk5 for the

expansion of the volume group descriptor area. Migrate/reorganize to free up

1 partitions and run chvg again.

0516-732 chvg: Unable to change volume group myvg.

In this example, we vary on the VG again, and shrink the file system by 1 PP, giving us 1 PP free in the volume group. Remember, “There must be enough free partitions available on each physical volume for the VGDA expansion for this operation to be successful.”

# varyonvg myvg

# chfs -a size=-512M /myfs

Filesystem size changed to 102760448

Inlinelog size changed to 196 MB.

# lsvg myvg

VOLUME GROUP: myvg VG IDENTIFIER: 00f6027300004c0000000133aa24d1f6

VG STATE: active PP SIZE: 512 megabyte(s)

VG PERMISSION: read/write TOTAL PPs: 99 (50688 megabytes)

MAX LVs: 256 FREE PPs: 1 (512 megabytes)

LVs: 1 USED PPs: 98 (50176 megabytes)

OPEN LVs: 0 QUORUM: 2 (Enabled)

TOTAL PVs: 1 VG DESCRIPTORS: 2

STALE PVs: 0 STALE PPs: 0

ACTIVE PVs: 1 AUTO ON: yes

MAX PPs per VG: 32512

MAX PPs per PV: 1016 MAX PVs: 32

LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no

HOT SPARE: no BB POLICY: relocatable

PV RESTRICTION: none

The VG is varied off again.

# varyoffvg myvg

# lsvg -o

rootvg

We convert the VG to scalable.

# chvg -G myvg

0516-1224 chvg: WARNING, once this operation is completed, volume group myvg

cannot be imported into AIX 5.2 or lower versions. Continue (y/n) ? y

0516-1216 chvg: Physical partitions are being migrated for volume group

descriptor area expansion. Please wait.

0516-1712 chvg: Volume group myvg changed. myvg can include up to 1024 physical volumes with 2097152 total physical partitions in the volume group.

We vary on the volume group and check that it is now scalable. You’ll notice that we’ve now lost that free PP in the VG as it has been used in the conversion process.

# varyonvg myvg

# lsvg myvg

VOLUME GROUP: myvg VG IDENTIFIER: 00f6027300004c0000000133aa24d1f6

VG STATE: active PP SIZE: 512 megabyte(s)

VG PERMISSION: read/write TOTAL PPs: 98 (50176 megabytes)

MAX LVs: 256 FREE PPs: 0 (0 megabytes)

LVs: 1 USED PPs: 98 (50176 megabytes)

OPEN LVs: 0 QUORUM: 2 (Enabled)

TOTAL PVs: 1 VG DESCRIPTORS: 2

STALE PVs: 0 STALE PPs: 0

ACTIVE PVs: 1 AUTO ON: yes

MAX PPs per VG: 32768 MAX PVs: 1024

LTG size (Dynamic): 256 kilobyte(s) AUTO SYNC: no

HOT SPARE: no BB POLICY: relocatable

PV RESTRICTION: none

If you are unable to free up a physical partition on a your hdisk (or hdisks) then you may need to use a tool like migratelp to move logical partitions around in order to free up some space. From the migratelp man page:

migratelp Command

Purpose

Moves allocated logical partition from one physical partition to another physical partition on a different physical volume.

Feed for Blog Entries ▼

Blogs

Chris's AIX Blog

▼ Tags

▼ Similar Blogs

AIX Down Unde...

Application P...

Asset Managem...

Application I...

AIXpert Blog

▼ Similar Ideation Blogs

IBM PureData-...

IBM Forms Exp...

WebSphere App...

IBM Platform ...

Rational Soft...

▼ Archive

▼ Blog Authors

Chris's AIX Blog

Using the AIX splitvg command.

Working with AIX scalable volume groups.

Working with AIX scalable volume groups - Part 2 - converting to scalable.