IBM Support

API Connect Management Subsystem Installation Stalls Due to Missing S3 Backup Secret

Troubleshooting


Problem

During a clean installation of API Connect v10.0.8.1, the Management subsystem fails to complete setup. The installation process stalls with several services remaining in a pending state, including postgresDb, ldap, apim, and others.

kubectl get apic
NAME                                                         READY   STATUS    VERSION    RECONCILED VERSION   MESSAGE                                                                  AGE
managementcluster.management.apiconnect.ibm.com/management   1/16    Warning   10.0.8.1                        Management installation in progress - see status condition for details   18h

 

kubectl get management -o yaml
...
    conditions:
    - lastTransitionTime: "<timestamp>"
      message: 'WAL Archiving not working. Management installation in progress. Not
        all services are ready, next pending services: lur, ldap, apim, websocket-proxy,
        analytics-proxy, ..., postgresDb'
      reason: WALArchiveNotWorking
      status: "True"
      type: Warning
...

Cause

The PostgreSQL database used by the Management subsystem is unable to start due to missing access credentials for the S3 backup secret. This results in errors such as:

kubectl logs pod/management-7cceaa52-db-1 -c postgres

{"level":"info","ts":"<timestamp>","msg":"backup credentials don't yet have access permissions. Will retry reconciliation loop","logging_pod":"management-7cceaa52-db-1"}

{"level":"error","ts":"<timestamp>","msg":"while getting recover credentials","logging_pod":"management-7cceaa52-db-1","error":"while getting secret mgmt-backup-secret-s3: secrets \"mgmt-backup-secret-s3\" is forbidden: User \"system:serviceaccount:apiconnect:management-7cceaa52-db\" cannot get resource \"secrets\" in API group \"\" in the namespace \"apiconnect\": . Opc-Request-Id: <nil>","stacktrace":"github.com/EnterpriseDB/cloud-native-postgres/pkg/management/log.(*logger).Error\n\tpkg/management/log/log.go:125\ngithub.com/EnterpriseDB/cloud-native-postgres/pkg/management/log.Error\n\tpkg/management/log/log.go:163\ngithub.com/EnterpriseDB/cloud-native-postgres/internal/management/controller.(*InstanceReconciler).updateWALRestoreSettingsCache\n\tinternal/management/controller/cache.go:71\ngithub.com/EnterpriseDB/cloud-native-postgres/internal/management/controller.(*InstanceReconciler).updateCacheFromCluster\n\tinternal/management/controller/cache.go:47\ngithub.com/EnterpriseDB/cloud-native-postgres/internal/management/controller.(*InstanceReconciler).Reconcile\n\tinternal/management/controller/instance_controller.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tpkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222"}

{"level":"info","ts":"<timestamp>","logger":"barman-cloud-wal-archive","msg":"2025-10-16 21:03:11,966 [25682] ERROR: Barman cloud WAL archiver exception: An error occurred (403) when calling the HeadBucket operation: Forbidden","pipe":"stderr","logging_pod":"management-7cceaa52-db-1"}

Additionally, WAL archiving fails due to misconfigured or inaccessible backup credentials, further preventing the database from initializing.

Resolving The Problem

To resolve the issue:

1. Verify Backup Configuration

2. Create the Required Secret

  • Manually create the Kubernetes secret mgmt-backup-secret-s3 in the apiconnect namespace with the correct credentials.
  • Ensure the service account used by the management pod has permission to access this secret.

3. Reinstall API Connect

  • If the installation remains blocked after correcting #1 and #2, perform a clean uninstall using the documented steps: Uninstalling API Connect
  • Then proceed with a fresh installation.

Document Location

Worldwide

[{"Type":"MASTER","Line of Business":{"code":"LOB77","label":"Automation Platform"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSMNED","label":"IBM API Connect"},"ARM Category":[{"code":"a8mKe000000CaZWIA0","label":"API Connect-\u003EAPIC Platform - Install\/Upgrade\/Migrate"}],"ARM Case Number":"TS020557134","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"and future releases;10.0.5"}]

Document Information

Modified date:
23 October 2025

UID

ibm17248948

Page Feedback