IT35264: MISLEADING MESSAGE "CTGGA4077,[<VM_IP>] DETECTED DUPLICATE UUIDS FOR ONE OR MORE PHYSICAL VOLUMES"

APAR status

• Closed as program error.

Error description

• When using IBM Spectrum Protect Plus to backup Linux guests :
  - having some or all filesystems managed with the Logical
    Volume Manager (LVM)
  and
  - with the agent option "Catalog file metadata" enabled,
  the job log will display warning messages as seen in the
  following example :
  
  SUMMARY,<timestamp>,CTGGA2399,Starting job for policy BACKUP
          with job name <SLAName> (ID:xxxx). id -> <JobID>. IBM
          Spectrum Protect Plus version 10.1.7-3043.
  ...
   DETAIL,<timestamp>,CTGGA0671,Backing up VM (<VMName>) from
          remote proxy(IP: <vADPName>  Host name: <vADPHostName>)
     INFO,<timestamp>,CTGGA2110,Will do incremental backup for
          VM <VMName>
  ...
     WARN,<timestamp>,CTGGA4077,[<VM_IP>] Detected duplicate
          UUIDs for one or more physical volumes. Aborting
          discovery of volume group information.
     WARN,<timestamp>,CTGGA4075,[<VM_IP>] Failed to discover
          volume group information.
     WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine
          covering disks for volume /dev/mapper/fs1.
     WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine
          covering disks for volume /dev/mapper/fs2.
   DETAIL,<timestamp>,CTGGA4088,[<VM_IP>] Collect file metadata
          under mountpoint /<non-lvm-filesystem>.
   DETAIL,<timestamp>,CTGGA4090,[<VM_IP>] Collected metadata for
          yyy file path(s) under mountpoint /<non-lvm-filesystem>
     WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine
          covering disks for volume /dev/mapper/fs3.
     WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine
          covering disks for volume /dev/mapper/fs4.
     ...
     WARN,<timestamp>,CTGGA4084,[<VM_IP>] Failed to determine
          covering disks for volume /dev/mapper/fs<z>.
   DETAIL,<timestamp>,CTGGA4002,[<VM_IP>] Discovered files and
          directories from 1 mounted volume(s) on 1 disk(s) in
          <xx>s.
     INFO,<timestamp>,CTGGA2219,[<VM_IP>] Discovery on Host
          <VMName>  completed with status success
  ...
  SUMMARY,<timestamp>,CTGGA2384,
          VM: <VMName>;
          Proxy: <vADPHostName>;
          vSnap(s): <vSnapHostName>;
          Type: Incremental;
          Transport: [hotadd];
          Transferred: xx GB;
          Throughput: yy MB/s;
          Queue Time: 0 second;
          Protected VMDKs: z of z;
          Status: complete
  ...
  SUMMARY,<timestamp>,CTGGA2340,Job policy <SLAName> completed
          in <elapsed time> and ended with status COMPLETED.
          id <JobID>
  
  The application log found in the job log bundle will display
  the following as seen for one of the guest file systems :
  
  [<timestamp>] JOBLOG pid:<PID> MainThread main: <CTGGA4001>
                Hostname: <VMHostName> / Operating System:
                Red Hat Enterprise Linux Server 7.9 x86_64 /
                Python: 3.6.9
  [<timestamp>] JOBLOG pid:<PID> MainThread discovery: Paths
                excluded from file discovery: /mnt/spp; /tmp;
                /usr/bin; /bin; /sbin
  [<timestamp>] JOBLOG pid:<PID> MainThread discovery:
                <CTGGA4160> Discovering disks and mounted volumes
  [<timestamp>] INFO   pid:<PID> MainThread
                linux_populate_blockdevice_info: Collecting block
                device information
  [<timestamp>] WARN   pid:<PID> MainThread
                linux_populate_volume_group_info_pvs: <CTGGA4077>
                Detected duplicate UUIDs for one or more physical
                volumes. Aborting discovery of volume group
                information.
  [<timestamp>] WARN   pid:<PID> MainThread
                populate_volume_group_info: <CTGGA4075> Failed to
                discover volume group information.
  [<timestamp>] INFO   pid:<PID> MainThread
                populate_mounted_volume_info: Collecting mounted
                volume information
  [<timestamp>] INFO   pid:<PID> MainThread
                get_volume_discovery_json: Traceback (most recent
                call last):
    File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 911
    , in get_volume_discovery_json
    File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 813
    , in get_covering_disks
  Exception: No physical volumes found for
             <lvm_volume_group_name>
  [<timestamp>] WARN   pid:<PID> MainThread
                get_volume_discovery_json: <CTGGA4084> Failed to
                determine covering disks for volume
                /dev/mapper/fs1.
  [<timestamp>] INFO   pid:<PID> MainThread
                get_volume_discovery_json: Traceback (most recent
                call last):
    File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 911
    , in get_volume_discovery_json
    File "/opt/IBM/SPP/agents_root/agtcomm/diskutil.py", line 813
    , in get_covering_disks
  Exception: No physical volumes found for
             <lvm_volume_group_name>
  
  Regardless of whether the VM backup is successful or not, the
  cataloging of the file metadata will fail.
  
  Verifying the guest disks Universal Unique Identifiers (UUID)
  will not display any duplicates.
  IBM Spectrum Protect Plus queries the guest defined volume
  groups with the in-guest command : 'pvs'.
  In this occurrence, the output does provide the information but
  with a warning message.
  Example of a guest with two defined volume groups :
  
  WARNING: Failed to connect to lvmetad. Falling back to device
           scanning.
   PV        VG  Fmt  Attr PSize   PFree
   /dev/sdb2 vg1 lvm2 a--  480.00g   4.00m
   /dev/sdc1 vg2 lvm2 a--  300.00g  50.00g
   /dev/sdd1 vg2 lvm2 a--  600.00g 100.00g
  
  When this command has a warning message, it is incorrectly
  parsed and results with the misleading message seen above in
  the application log :
  <CTGGA4077> Detected duplicate UUIDs for one or more physical
              volumes.
  
  The above 'pvs' command warning means it is trying to use the
  'lvmetad' service that is not active.
  Disabling lvmetad is documented in the IBM Spectrum Protect
  Plus file indexing and restore requirements :
  
  "If data resides on LVM volumes, the lvm2-lvmetad service must
   be disabled, as it can interfere with the ability of IBM
   Spectrum Protect Plus to mount and resignature volume group
   snapshots and clones.
   To disable the service, complete the following steps:
   - Run the following commands:
       systemctl stop lvm2-lvmetad
       systemctl disable lvm2-lvmetad
   - Edit the /etc/lvm/lvm.conf file and specify the following
     setting:
       use_lvmetad = 0"
  
  This APAR addresses the incorrect warning message parsing.
  
  IBM Spectrum Protect Plus Versions Affected:
  IBM Spectrum Protect Plus 10.1.3 to 10.1.7 when enabling
  "Catalog file metadata" option for Linux guests using LVM.
  
  | MDVREGR 10.1.2 5737SPLUS | IT24930
  
  Initial Impact: Medium
  
  Additional Keywords: SPP, SPPLUS, TS004585641, LVM, UUID
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Spectrum Protect Plus levels 10.1.3 to 10.1.7.           *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply the fixing level when available. This problem is       *
  * projected to be fixed in IBM Spectrum Protect Plus levels    *
  * 10.1.8 patch1 and 10.1.9. Note that this is subject to       *
  * change at the discretion of IBM.                             *
  ****************************************************************
  

Problem conclusion

• When a Linux VM is configured to use lvmetad, a specific error
  message will be displayed indicating lvm is configured to use
  lvmetad and instructions on how to fix this. Additionally, test
  and configure will now check if lvm is configured to use lvmetad
  and if so flag it as an error.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels