IT47264: MAINTENANCE JOB ENDS IN STATUS PARTIAL WITH NO EXPLANATORY MESSAGING IN THE JOB LOG

APAR status

• Closed as program error.

Error description

• The IBM Storage Protect Plus Maintenance job can end with the
  status partial without any messaging in the job log that can
  explain the reason to the operator.
  
  In the job log, the partial status can be seen for the
  'Hypervisor Copy' sub task that triggers the overall job partial
  
  status :
  
  SUMMARY,..,CTGGA2398,Starting job for policy Maintenance. id ->
                      <JobID>. IBM Spectrum Protect Plus version
                      10.1.16-502.
  ... DETAIL,..,CTGGA0814,Task 2 : Hypervisor Copy completed with
                               status PARTIAL
  SUMMARY,..,CTGGA2340,Job policy Maintenance completed in 15 min
                      (s) 2 sec(s) and ended with status PARTIAL.
                      id <JobID>
  
  In the corresponding virgo log, a bit more messaging can be
  seen :
  
  .. INFO .. Job: [FlowJob: [name=basicOneStepJob]] launched with
                 the following parameters: [{session=<JobID>,
                 type=Hypervisor Copy, taskName=basicOneStepJob,
                 resource=<JobID>_1, weight=1, executor=
                 HYPERVISOR_PROTECTION_MAINTENANCE_TASK,
                 taskId=2}]
  ...
  .. INFO .. cleaning up the policy :::: vmware_<SLA_Name>.
  ...
  .. INFO .. allCatalogSnapPks snapshot for Job vmware_<SLA_Name>
            jobId <SLA_ID> are :::
  .. INFO .. allCatalogSnapPks :::<ID>.snapshot.<SnapID_1> ,
            <ID>.snapshot.<SnapID_2> ...
  .. INFO .. trying to remove relationship ::::
            <ReplicationRelationshipID> id <ID>.repGroup.
            <Repl_ID> for policy vmware_<SLA_Name>
  .. INFO .. OSSM storage exception:  com.catalogic.ecx.
            serviceprovider.common.storage.exception.
            StorageVolumeNotFoundException: Not Found
         at ...
  .. INFO .. Storage Replication target volume not found for
            relationship <ReplicationRelationshipID> vol name
            <ReplicationRelationshipID> for policy
            vmware_<SLA_Name>
  .. INFO .. Removing relationship <ReplicationRelationshipID> for
  
            policy vmware_<SLA_Name> .. INFO .. Storage Cache
            :::: Get Volume  <ID>:<Repl_ID> .. INFO .. OSSM api
            call ::GET https://<OSSMHostAddress>:3337/api/v1beta
            /volume/<Repl_ID>
  .. INFO .. Respone Body :: {
      "id" : <Repl_ID>,
      "name" : "<VolumeName>",
      "path" : "/ossm/volmapping/<PoolName>/<FSID>/<VolumeName>",
      "type" : "filesystem",
      "vendor" : null,
      "serial" : null,
      "maxNameLength" : 0,
      "size" : 0,
      "creationTime" : "<Timestamp>",
      "volume_type" : "filesystem",
      "pool_id" : <PoolID>,
      "pool_name" : "<PoolName>",
      "is_clone" : false,
      "time_created" : "<Timestamp>",
      "time_updated" : "<Timestamp>",
      "share_id" : 0,
      "snapshot_id" : 0,
      "is_replica" : true
      }
  .. INFO  .. OSSM api call ::DELETE https://<OSSMHostAddress>:
             3337/api/v1beta/relationship/<Repl_ID>
  .. ERROR .. Failed to cleanup the job : name(vmware_<SLA_Name>)
             reason(Relationship <Repl_ID> not found).
  ...
  .. INFO  .. Job: [FlowJob: [name=basicOneStepJob]] completed
             with the following parameters: [{session=<JobID>,
             type=Hypervisor Copy, taskName=basicOneStepJob,
             resource=<JobID>_1, weight=1,
             executor=HYPERVISOR_PROTECTION_MAINTENANCE_TASK,
             taskId=2}] and the following status: [FAILED] in
             <Duration>
  
  This behaviour can happen in case of a stale OSSM
  (Open Snap Store Manager) replication relationship.
  
  
  IBM Storage Protect Plus Versions Affected:
  Since OSSM was available starting with IBM Storage Protect Plus
  version 10.1.13, the problem can be seen only for version
  10.1.13 and later.
  
  Additional Keywords: SPP, SPPLUS, TS017259126
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Storage Protect Plus users 10.1.13 and later             *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See Error Description                                        *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  

Problem conclusion

• In IBM Storage Protect Plus version 10.1.13, Volume Replication
  functionality was added to OSSM. But, due to incomplete
  replication if the volume is not created on the Storage Protect,
  it might cause the maintenance job to show "partial" status
  without displaying the reason in the Job Logs. This has been
  fixed in IBM Storage Protect Plus version 10.1.16.4; the system
  now shows "Volume not found" exception in the Job Logs.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  SP PLUS

• Fixed component ID

  5737SPLUS

Applicable component levels