My splitvg Went Splat (updated)
AnthonyEnglish 270000RKFN Comments (2) Visits (6544)
A couple of years ago I wrote about the splitvg command. This takes a volume group (VG) that has been mirrored using mirrorvg, and effectively breaks off one copy of the mirror, turning it into a point-in-time snapshot. The snapshot can then be used for reports, backups or for building a new database. I'm not sure how much this is used in the real world these days, but if you are still doing your backups this way, perhaps it's time to revisit the strategy.
Split & Resynch
Never heard of it? Here's how it works.
After you've done the split of the VG, you can continue business on the "live" copy of the volume group, and then make use of the frozen snapshot copy, for example for backups, or to run some reports. It's a little like creating an alternate root disk, or running multibos to create a standby instance of the OS. With breaking the volume group mirror, you have one live copy and one snapshot copy of the same VG. As the copy which has been snapshotted (with apologies to any grammarians) starts life as a perfect replica of the primary live copy, after you've finished with the snapshot and want to resynchronise the two copies again, the process is usually faster than when you break a volume group mirror altogether using unmirrorvg. Of course, the longer the database is up on the primary copy without its mirror, and the more updates are done to that live database, the more data there will be to synchronise when you're ready to turn the snapshot back into a real mirrored copy.
I've seen this split/resynch approach used in smaller sites, especially before the days of SAN storage. One manufacturing company I worked with in the 1990s couldn't afford to shut down their database every night, do a cold backup for 3 or more hours, and then start the database again. So they did the shutdown of the database, split the mirror, and then got the database started again. Database downtime was about 10 minutes. The frozen copy was then backed up to tape. After a (hopefully) successful backup, the snapshot copy was then resynchronised, so that the two active copies of the database were back in place for the purposes of redundancy.
Back to the 90s
Now if you remember this approach from a time way back in your IT career, you may be feeling nostalgic for the 1990s when we were suspicious of Storage Area Networks (and all networks, for that matter), and when businesses relied on a nightly cold backup (and a weekly reboot: just to clear out the cobwebs). Who had ever heard the phrase "24 X 7"? Ah yes, those were the days! These days it's far more common to do backups online (without the database being shut down), or by using utilities such as SAN replication. Also, since most sites today use at the very least a small SAN, there is little value in doing software mirroring on the OS, since the SAN handles the redundancy. Nevertheless, there may still be some systems out there using the split-the-mirror approach. Often this is simply because the approach has worked for them, even if it does mean a cold backup (shutting the database down for the duration of the backup).
Splitting a Mirror Can Bring Bad Luck
Now, if you are using the split volume group strategy, and you're doing it for backups as I've just described, keep in mind two issues.
1. You will lose your redundancy from the time of running the splitvg command until you resynch the mirror again.
2. If you lose a disk, you not only lose redundancy. You also lose your backups until the disk is replaced and the mirror reinstated. This is pretty serious.
You may be able to get by for a while with losing a single disk, but that's only because you have the comfort of the data being on other disks, and at the very worst, you could revert to a backup. However, if you lose the disk and the backup, you may have a lot of explaining to do. And that's why a broken (volume group) mirror can bring you bad luck.
Update: Jim Carstensen has kindly offered some good comments. The AIX Logical Volume Manager (LVM) allows you to have three copies of a logical volume, or of an entire volume group. So you could have a three-copy mirror, split off just one copy for your snapshot, and still have redundancy by maintaining two good copies on production. It's a very good point. He also notes that even if the splitvg snapshot for backups isn't optimal, it's still better than nothing.
Look at the mirror
If you're still using this split volume group approach for your nightly backups, it may be time to look at a new strategy. Online backups, even database exports, might be preferable. Otherwise, losing a disk may mean losing a database, and leave you with no recent backup to rely on.