Backing up IBM Business Automation Workflow on containers

Use the following commands to scale down all deployments and stateful sets:

kubectl scale deploy ibm-cp4a-operator --replicas=0 -n your-namespace
kubectl scale deploy ibm-pfs-operator --replicas=0 -n your-namespace
kubectl scale deploy ibm-content-operator --replicas=0 -n your-namespace

for i in $(kubectl get deploy -n your-namespace -o name | grep bawdeploy); do
  kubectl scale $i --replicas=0 -n your-namespace
done

for i in $(kubectl get sts -n your-namespace -o name | grep bawdeploy); do
  kubectl scale $i --replicas=0 -n your-namespace
done

Export and securely store the final YAML of the Business Automation Workflow CR or take the backup of the latest CR applied.

kubectl get icp4acluster bawdeploy -n your-namespace -o yaml | \
  yq eval 'del(.metadata.uid, .metadata.resourceVersion, 
.metadata.creationTimestamp, 
.metadata.annotations."kubectl.kubernetes.io/last-applied-configuration", 
.metadata.generation, .status)' - > clean-icp4a.yaml

The following example script can back up secrets across all namespaces, excluding default tokens, kube-system. You can customize the script to back up only the critical secrets required.

#!/bin/bash
backup_dir="k8s-backup-secret-$(date +%F_%H-%M-%S)"
mkdir -p "$backup_dir"
 
clean_yaml() {
  yq eval 'del(.metadata.uid, .metadata.resourceVersion, .metadata.selfLink, .metadata.creationTimestamp, .metadata.generation, .metadata.managedFields, .metadata.annotations, .metadata.ownerReferences, .status)' -
}
 
for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
  mkdir -p "$backup_dir/$ns"
  for secret in $(kubectl get secrets -n "$ns" --no-headers=true 2>/dev/null | awk '{print $1}'); do
    [[ "$secret" == *"default-token"* ]] && continue
    echo "---" >> "$backup_dir/$ns/secrets.yaml"
    kubectl get secret "$secret" -n "$ns" -o yaml | clean_yaml >> "$backup_dir/$ns/secrets.yaml"
  done
done

Use the provided script to back up all ConfigMaps across namespaces. Clean metadata by using yq.

#!/bin/bash
backup_dir="k8s-backup-configmap-$(date +%F_%H-%M-%S)"
mkdir -p "$backup_dir"

clean_yaml() {
  yq eval 'del(.metadata.uid, .metadata.resourceVersion, .metadata.selfLink, .metadata.creationTimestamp, .metadata.generation, .metadata.managedFields, .metadata.annotations, .metadata.ownerReferences, .status)' -
}

for ns in $(kubectl get ns -o jsonpath='{.items[*].metadata.name}'); do
  mkdir -p "$backup_dir/$ns"
  for cm in $(kubectl get configmaps -n "$ns" --no-headers=true 2>/dev/null | awk '{print $1}'); do
    echo "---" >> "$backup_dir/$ns/configmaps.yaml"
    kubectl get configmap "$cm" -n "$ns" -o yaml | clean_yaml >> "$backup_dir/$ns/configmaps.yaml"
  done
done

Use the script to back up PVC definitions.

#!/bin/sh  
NS=your-namespace
pvcbackup() {
    kubectl get pvc -n $NS --no-headers=true | while read each
    do
        pvc=`echo $each | awk '{ print $1 }'`

        if [[ "$pvc" == ibm-bts-cnpg*  ]] ; then
          # skip BTS PVCs
          continue 
        fi

        echo "---" >> pvc.yaml
        kubectl get pvc $pvc -n $NS -o yaml \
            | yq eval 'del(.status, .metadata.finalizers, .metadata.resourceVersion, .metadata.uid, .metadata.annotations, .metadata.creationTimestamp, .metadata.selfLink, .metadata.managedFields, .metadata.ownerReferences, .spec.volumeMode, .spec.volumeName)' -  >> pvc.yaml
    
    done
} 
pvcbackup

The following table lists PVC definitions that must be backed up and restored (see the "Back up or replication required" column), along with others that can be optionally backed up.

Do not back up PVCs such as:

• data-iaf-system-elasticsearch-es-data-0
• iaf-system-elasticsearch-es-snap-main-pvc
• ibm-bts-cnpg-*
• user-home-pvc

For more information on other PVCs based on your capabilities, see Persistent volume claims to be backed up .

If your environment relies on dynamic provisioning, it is crucial to make sure that all data stored in your Persistent Volumes (PVs) is properly backed up. This step is vital for accurately restoring your environment. During the restoration process, the backed-up data is used to repopulate the corresponding PVs and to re-create the associated Persistent Volume Claims (PVCs).

Use the script to back up content of your dynamically provisioned PVs.

#!/bin/bash
NS=your-namespace
SRC_PV=<my-storage-class-path>
DEST_BACKUP=backups-pvc-content

mkdir -p "$DEST_BACKUP"

kubectl get pvc -n $NS --no-headers=true | while read line; do
  pvc=$(echo $line | awk '{print $1}')
  pv=$(echo $line | awk '{print $3}')
  [[ "$pvc" == ibm-bts-cnpg* ]] && continue
  src="$SRC_PV/$NS-$pvc-$pv"
  dst="$DEST_BACKUP/$pvc"
  if [ -d "$src" ]; then
    echo "Backing up $pvc from $src to $dst"
    mkdir -p "$dst"
    cp -r "$src/." "$dst/"
  else
    echo "Missing source for PVC: $pvc"
  fi
done

• JDBC drivers
• Customized runtime files (example - fonts)
• Persistent storage and database configuration files

For more information, see the following links:

• Db2 JDBC Driver Versions and Downloads
• Oracle JDBC Frequently Asked Questions
• PostgreSQL JDBC Drivers

• PostgreSQL: Backup and Recovery v1.26.0
• Oracle: Database Backup and Recovery User's Guide

For Db2 deployment, ensure that you perform either an offline or online backup of all relevant databases. This typically includes databases such as GCDDB, BAWTOS, BAWDB, ICNDB, BAWDOCS, BAWDOS, and any others specific to your runtime or authoring environment.

1. For online backup:
   1. Run the following Db2 archival commands for each database.

      mkdir  -p   /home/db2inst1/archive/TOSDB 
      db2 update db cfg for TOSDB using LOGINDEXBUILD  on 
      db2 update db cfg for TOSDB using LOGARCHMETH1   disk:/home/db2inst1/archive/TOSDB   
       
      mkdir  -p   /home/db2inst1/archive/GCDDB 
      db2 update db cfg for GCDDB using LOGINDEXBUILD  on 
      db2 update db cfg for GCDDB using LOGARCHMETH1   disk:/home/db2inst1/archive/GCDDB   
       
      mkdir  -p   /home/db2inst1/archive/AAEDB 
      db2 update db cfg for AAEDB using LOGINDEXBUILD  on 
      db2 update db cfg for AAEDB using LOGARCHMETH1   disk:/home/db2inst1/archive/AAEDB   
       
      mkdir  -p   /home/db2inst1/archive/ICNDB 
      db2 update db cfg for ICNDB using LOGINDEXBUILD  on 
      db2 update db cfg for ICNDB using LOGARCHMETH1   disk:/home/db2inst1/archive/ICNDB   
       
      mkdir  -p   /home/db2inst1/archive/BAWDB 
      db2 update db cfg for BAWDB using LOGINDEXBUILD  on 
      db2 update db cfg for BAWDB using LOGARCHMETH1   disk:/home/db2inst1/archive/BAWDB   
       
      mkdir  -p   /home/db2inst1/archive/DOCSDB 
      db2 update db cfg for DOCSDB using LOGINDEXBUILD  on 
      db2 update db cfg for DOCSDB using LOGARCHMETH1   disk:/home/db2inst1/archive/DOCSDB   
       
      mkdir  -p   /home/db2inst1/archive/DOSDB 
      db2 update db cfg for DOSDB using LOGINDEXBUILD  on 
      db2 update db cfg for DOSDB using LOGARCHMETH1   disk:/home/db2inst1/archive/DOSDB 
      
      mkdir  -p   /home/db2inst1/archive/BASDB 
      db2 update db cfg for BASDB using LOGINDEXBUILD  on 
      db2 update db cfg for BASDB using LOGARCHMETH1   disk:/home/db2inst1/archive/BASDB 
      
      mkdir  -p   /home/db2inst1/archive/APPDB 
      db2 update db cfg for APPDB using LOGINDEXBUILD  on 
      db2 update db cfg for APPDB using LOGARCHMETH1   disk:/home/db2inst1/archive/APPDB 
      
      mkdir  -p   /home/db2inst1/archive/ADPBASE 
      db2 update db cfg for ADPBASE using LOGINDEXBUILD  on 
      db2 update db cfg for ADPBASE using LOGARCHMETH1   disk:/home/db2inst1/archive/ADPBASE 
      
      mkdir  -p   /home/db2inst1/archive/PROJ1 
      db2 update db cfg for PROJ1 using LOGINDEXBUILD  on 
      db2 update db cfg for PROJ1 using LOGARCHMETH1   disk:/home/db2inst1/archive/PROJ1
      
      mkdir  -p   /home/db2inst1/archive/DEVOS1 
      db2 update db cfg for DEVOS1 using LOGINDEXBUILD  on 
      db2 update db cfg for DEVOS1 using LOGARCHMETH1   disk:/home/db2inst1/archive/DEVOS1
      
      mkdir  -p   /home/db2inst1/archive/AEOS 
      db2 update db cfg for AEOS using LOGINDEXBUILD  on 
      db2 update db cfg for AEOS using LOGARCHMETH1   disk:/home/db2inst1/archive/AEOS

   2. Run the following Db2 backup commands for each database.

      mkdir -p /home/db2inst1/backup/2500/offline
      db2 backup db TOSDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db GCDDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db AAEDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db ICNDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db BAWDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db DOCSDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db DOSDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db BASDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db APPDB online to /home/db2inst1/backup/2500/offline 
      db2 backup db ADPBASE online to /home/db2inst1/backup/2500/offline 
      db2 backup db PROJ1 online to /home/db2inst1/backup/2500/offline 
      db2 backup db DEVOS1 online to /home/db2inst1/backup/2500/offline 
      db2 backup db AEOS online to /home/db2inst1/backup/2500/offline

2. Terminate your database connections to prevent errors during the backup.

   db2 force applications all

3. For offline backup, run the following command:

   db2 backup db TOSDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db GCDDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db AAEDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db ICNDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db BAWDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db DOCSDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db DOSDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db BASDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db APPDB to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db ADPBASE to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db PROJ1 to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db DEVOS1 to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024
   db2 backup db AEOS to /home/db2inst1/backup/2500/offline WITH 2 BUFFERS BUFFER 1024

For each Business Automation Workflow instance that is either federated or has full-text search that is enabled, you must back up the following components:

• Indexed data of the BPD runtime
• Saved search definitions
• Reusable queries
  Note: Backing up case instance indexes is not supported. Instead, rebuild case indexes as described in Rebuilding a case index.

1. Backing up indexed data.

   If you are using a containerized environment with full-text search enabled, or a traditional WebSphere® Application Server environment where Business Automation Workflow indexes BPD tasks and process instances into the Federated Data Repository (FDR):

   • Regularly create snapshots of the OpenSearch or Elasticsearch index that is used by the BPD runtime.
   • Restore these snapshots in the backup environment as outlined in the failover support section of Understanding the Federated Data Repository indexing.
   If Process Federation Server is configured to federate an on-premises Business Automation Workflow instance on WebSphere Application Server, and indexing is enabled by using <ibmPfs_bpdIndexer>:
   • Regularly back up the OpenSearch or Elasticsearch index.
   • Follow the steps in Federated Data Repository indexes .
2. Backing up saved searches and reusable queries

   For all Business Automation Workflow instances that are federated or have full-text search that is enabled:

   • Regularly back up saved search definitions and reusable queries.
   • Restore them in the backup environment.

   The Process Federation Server API is available on both standalone and embedded instances. Embedded APIs are exposed on each containerized Business Automation Workflow instance.

   To perform these backups:

   • Saved search definitions: Use the Saved Search Transfer REST API to export and import saved searches. For details, see REST interface for IBM Process Federation Server resources .
   • For reusable queries, you can directly use the Elasticsearch or OpenSearch snapshots API. For more information, see Backing up and restoring reusable search queries .

• OpenSearch: Stores time series data and summaries. For more information on backing up, see Taking and restoring snapshots of OpenSearch data .
• Flink: Holds the state of event processing. For guidance on recovery, see Restarting from a checkpoint or savepoint .
• Manage backup and restore procedures for the Kafka server, which is set up through Cloud Pak for Business Automation.

• IBM Decision Intelligence Client Managed Software: Backing up Decision Intelligence Client Managed Software
• IBM Content Cortex: Backup and recovery
• IBM Business Automation Navigator: Backing up Content Navigator