IBM Support

IT34454: APPLICATION INVENTORY FOR KUBERNETES CLIENT STOPS WITH "ERROR MESSAGE: NOT FOUND"

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • The inventory for a Kubernetes client using IBM Spectrum
    Protect Plus will display in the virgo log :
    
    
    [<timestamp>]
    INFO .. Starting job for policy Application Server
            Inventory. id -> <JobID>.
            IBM Spectrum Protect Plus version 10.1.6-2040
    ...
    .. INFO .. Unix Host Agent 10.1.6.3443 running as
               irSrH8ZGDIqp9ri for appdiscovery (task ID: <TaskID>)
    ..
       INFO .. Hostname: <KubernetesHostName> / Operating System:
               Red Hat Enterprise Linux Server 7.8 x86_64 /
               Python: 3.6.9
    .. INFO .. Discovering disks and mounted volumes
    .. INFO .. Discovering network and initiator information.
    .. INFO .. Discovering application instances.
    .. INFO .. --- Running discovery for k8s agent ---
    .. INFO .. Starting the inventory of a persistent volume.
    .. ERROR.. Failed to complete Kubernetes API call.
               Error message: Not Found
    .. ERROR.. An exception was encountered while running
               application discovery: (404).
    .. INFO .. [<KubernetesHostIP>] Reason: Not Found
    .. INFO .. [<KubernetesHostIP>] HTTP response headers:
               HTTPHeaderDict({'Content-Type': 'text/plain;
               charset=utf-8' 'X-Content-Type-Options': 'nosniff',
               'Date': '<timestamp>', 'Content-Length': '19'})
    .. INFO .. [<KubernetesHostIP>] HTTP response body: b'404 page
               not found\n'
    .. INFO .. Ending the inventory of a persistent volume.
    .. INFO .. --- Finished discovery for k8s agent ---
    ..
    .. INFO .. Job policy Application Server Inventory completed in
               37 sec(s) and ended with status PARTIAL. id <JobID>
    
    In the agent command log the following will be seen :
    
    [<timestamp>] JOBLOG_SUMMARY pid:914 MainThread jobsummary:
     <CTGGK3001> Starting the inventory of a persistent volume.
    .. updateAgentPodLabels: Updated baas:<KubernetesHostName> add
       label app.kubernetes.io/agent-version=10.1.6.20200716D
    .. api_call: Executing the following Kubernetes API:
       /apis/config.openshift.io/v1/clusteroperators/openshift-
       apiserver Method: GET
    .. getOpenshiftVersion: Openshift version not found. This is
       not openshift.
    .. api_call: Executing the following Kubernetes API: /version
       Method: GET
    .. error: <CTGGK3042> Failed to complete Kubernetes API call.
       Error message: Not Found
    .. executeSppCommand: Application agent is raising an exception
       to the host agent, traceback: 'Traceback (most recent call
       last):
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/main.py",
       line 139, in executeSppCommand
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/
       appdiscoveryx.py", line 398, in execute
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/
       appdiscoveryx.py", line 299, in appSdkInstance
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/
       appdiscoveryx.py", line 69, in appSdkDb
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/KubeCalls.py"
       , line 518, in getVolumeSnapshotClassSnapshotters
       File "/opt/IBM/SPP/agents_irSrH8ZGDIqp9ri/k8s/KubeCalls.py"
       , line 507, in getVolumeSnapshotClassSnapshotters
    .. executeSppCommand: Application agent is raising an exception
       to the host agent, traceback: 'Traceback (most recent call
       last):
    ...
       kubernetes.client.rest.ApiException: (404)
       Reason: Not Found
       HTTP response headers: HTTPHeaderDict({'Content-Type':
            'text/plain; charset=utf-8', 'X-Content-Type-Options':
            'nosniff', 'Date': '<timestamp>', 'Content-Length': '19'
            }) HTTP response body: b'404 page not found\n'
    .. ERROR pid:914 MainThread error: <CTGGK3044> An exception was
             encountered while running application discovery: (404)
             Reason: Not Found
    
    The Kubernetes agent is crashing when it fails to find any
    VolumeSnapshotClass objects.
    Instead of failing the Inventory, the process should report a
    warning when there is no v1alpha1 VolumeSnapshotClass object
    found.
    This can be confirmed by running the following command:
       kubectl get volumesnapshotclass
    A volumesnapshotclass is required to both be able to create
    snapshots and transfer data to the vSnap host during backup.
    This is missing from the requirements listed in the
    documentation.
    Despite the missing requirement, registration of the Kubernetes
    Application Server succeeds.
    The connection test towards the Kubernetes Application Server
    completes successfully without detecting the missing
    volumesnapshotclass.
    
    IBM Spectrum Protect Plus Versions Affected:
    IBM Spectrum Protect Plus 10.1.6
    
    Initial Impact: High
    
    Additional Keywords: SPP, SPPLUS, K8s, inventory
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * IBM Spectrum Protect Plus Container Backup Support level     *
    * 10.1.6 for Kubernetes                                        *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See ERROR DESCRIPTION.                                       *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * Apply fixing level when available. This problem is currently *
    * projected to be fixed IBM Spectrum Protect Plus Container    *
    * Backup Support level 10.1.7.                                 *
    * Note that this is subject to change at the discretion of     *
    * IBM.                                                         *
    ****************************************************************
    

Problem conclusion

  • If there is no available VolumeSnapshotClass object(s) all
    Container Storage Interface (CSI) PersistentVolumeClaim
    object(s) of the Kubernetes cluster will be marked as ineligible
    for backup.
    

Temporary fix

Comments

APAR Information

  • APAR number

    IT34454

  • Reported component name

    SP PLUS

  • Reported component ID

    5737SPLUS

  • Reported release

    A16

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2020-10-06

  • Closed date

    2020-12-04

  • Last modified date

    2020-12-04

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Modules/Macros

  • k8s
    

Fix information

  • Fixed component name

    SP PLUS

  • Fixed component ID

    5737SPLUS

Applicable component levels

[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSNQFQ","label":"IBM Spectrum Protect Plus"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"A16","Line of Business":{"code":"LOB26","label":"Storage"}}]

Document Information

Modified date:
31 January 2024