C-SPOC: Changing a file system fails with "VG is concurrent" error.

Troubleshooting

Problem

Using the C-SPOC facility to make changes to a shared file system size on all nodes in a PowerHA cluster is a simple task. However, it can fail when the volume group has incorrect permissions. This document provides the procedures to troubleshoot and solve this problem.

Symptom

The wrong permission problem returns the following error:

Cause

This issue occurs when an enhanced concurrent volume group is manually activated with read and write permissions on all nodes.
Doing that also increases the risk of data corruption.

PowerHA® SystemMirror® automatically handles the volume group permissions during cluster startup or resource group failover.
It varies on the volume group in the appropriate active or passive state depending on the role the node performs in the cluster.

If the volume group permissions are set incorrectly, the C-SPOC facility returns an error.

Environment

All versions of PowerHA SystemMirror for AIX.

Diagnosing The Problem

► Step 1: Check which node has the resource group in online state by using clRGinfo command:

► Step 2: Check which node in the cluster has the volume group permission set to read/write other than the node where the resource group is online.

The lsvg output shows that the volume group is activated with read/write permission on the secondary node halab14 the same as node halab13 (Primary node).

Resolving The Problem

"To fix it, you need to vary on the volume group in passive mode". No downtime is needed to run those steps.

Note: Always verify that the file systems are not mounted on more than one node when you get this error.

1- run the mount command to check whether the file systems in the volume group are mounted on the secondary node. If they are mounted, then unmount them.

In this test, no file systems were mounted simultaneously on both nodes. The mount guard flag was set to prevent simultaneous mount.

If file systems are simultaneously mounted on more than one node, run fsck command to check for file system inconsistency issues. Follow the steps in this documentation to fix the file system. If it is corrupted beyond repair, you must restore it from backup.

Fixing a damaged file system

2- Vary on the volume group in passive mode.

# varyonvg -ncP vg1

The volume group is in the correct state on all nodes.
It is now possible to change the shared file system size.

Example: I was able to increase the file system "/tst" size to 4 Gigabytes by using C_SPOC

Enter the fast path smit cspoc.
In SMIT, select the Storage > file system > Change-Show Characteristics of a file system and press Enter.

SMIT displays a picklist of existing file systems.
Select the file system to change.

SMIT displays a window containing the characteristics of the file system.
Select the unit size and enter the new size of the file system in the fields and press Enter. The C-SPOC utility changes the file system characteristics on all nodes in the resource group.

Now the file system size changed to 4 GB.

Related Information

Changing a shared file system in PowerHA SystemMirror by using C-SPOC

Understanding active and passive varyon in enhanced concurrent mode

JFS2 file system concurrent mount protection

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SSPHQG","label":"PowerHA SystemMirror"},"ARM Category":[{"code":"a8m3p000000hAxRAAU","label":"PowerHA System Mirror-\u003EC-SPOC-\u003EFilesystem"}],"ARM Case Number":"","Platform":[{"code":"PF002","label":"AIX"}],"Version":"All Versions"}]

Tips