IT47192: STORAGE PROTECT SERVER DATABASE BACKUP TO CLOUD AND CLOUD DATA T

8.1.25.000-IBM-SPSRV-WindowsX64-340
8.1.25.000-IBM-SPSRV-Linuxx86_64-340
8.1.25.000-IBM-SPSRV-Linuxs390x-340
8.1.25.000-IBM-SPSRV-Linuxppc64le-340
8.1.25.000-IBM-SPSRV-AIX-340

APAR status

• Closed as program error.

Error description

• After upgrade IBM Storage Protect server to v8.1.23 or v8.1.24,
  the database backup to cloud storage runs much slower than
  before.
  
  The customer did server upgrade on 06/24/2024:
  
  
  2023/06/12 18:47:39 8.1.18.000
  
  2024/06/24 18:33:18 8.1.23.000
  
  
  1: Prior to the upgrade,
  - a db backup of ~600GB took ~17mins:
  
  -06/24: 17mins ~600GB
  2024-06-24T09:00:09 NR2017I Administrator AIXADMIN issued
  command: BACKUP DB dev=s3dbbackup type=full nums=18 wait=yes
  (SESSION: 832013)"
  2024-06-24T09:17:18 NR4550I Full database backup (process 2081)
  completed. Total bytes backed up: 612,049,879,040. (SESSION:
  832013, PROCESS: 2081)
  
  
  -A servermon shows cloud upload of speed: ~68MB/s
  
  
  Thread 4009102 AgentThread parent=4009101
  2024-06-22T09:00:13.063-->2024-06-22T09:09:00.987
  Operation Count Tottime Avgtime Mintime Maxtime InstTput Total
  KB
  ---------------------------------------------------------------
  -------------
  
  Cloud Upload 1515 363.119 0.240 0.187 1.055 68312.0 24805376
  
  2: after the upgrade,
  - the same size of db took 1 hour:
  
  
  2024-06-25T09:00:21ANR2017I Administrator AIXADMIN issued
  command: BACKUP DB dev=s3dbbackup type=full nums=18 wait=yes
  (SESSION: 7445)
  
  
  2024-06-25T09:56:03-ANR0985I Process 97 for Database Backup
  running in the FOREGROUND completed with completion state
  SUCCESS at 09:56:03 AM. (SESSION: 7445, PROCESS: 97)"|
  -A servermon shows the cloud upload speed is only 15MB/s
  
  
  Thread 28075 AgentThread parent=28065
  2024-06-25T09:07:29.997-->2024-06-25T09:27:18.162
  Operation Count Tottime Avgtime Mintime Maxtime InstTput Total
  KB
  ---------------------------------------------------------------
  -------------
  Cloud Upload 1135 1176.331 1.036 0.278 2.716 15794.4 185794
  
  Customer/Support diagnostics:
  
  1: "show JVM " command output prior to v8.1.23, it shows "Crypto
  Security Providers: 1. IBMJCEPlus"
  For example:
  
  Crypto Security Providers: 1. IBMJCEPlus 2. IBMJSSE2 3. IBMJCE
  4. IBMJGSSProvider 5. IBMCertPath 6. IBMSASL 7.
  IBMXMLCRYPTO 8. IBMXMLEnc 9. IBMSPNEGO 10. SUN
  
  
  2: "show JVM" command output from v8.1.23 or v8.1.24 are
  missing "Crypto Security Providers: 1. IBMJCEPlus"
  
  For example:
  Crypto Security Providers: 1. IBMJSSE2 2. IBMJCE 3.
  IBMJGSSProvider 4. IBMCertPath 5. IBMSASL 6. IBMXMLCRYPTO 7.
  IBMXMLEnc 8. IBMSPNEGO 9. SUN
  
  
  Note: the missing  IBMJCEPlus Crypto Provider is the reason of
  slowness.
  
  3: ibmjceplus.jar is missing from v8.1.23 and v8.1.24 jre
  package but exists in package prior to v8.1.23:
  
  - version before v8.1.23
  
  
  find /opt -name ibmjceplus.jar -print
  
  
  this shows :
  /opt/tivoli/tsm/jre/fips140-2/lib/ext/ibmjceplus.jar
  /opt/tivoli/tsm/jre/fips140-3/lib/ext/ibmjceplus.jar
  
  -the same find command shows no return from v8.1.23 and v8.1.24
  server
  
  
  Note: The missing ibmjceplus.jar is the cause of missing
  IBMJCEPlus Crypto Provider
  
  Platform/Version Affected:
  IBM  Storage Protect server v8.1.23 and v8.1.24 on all supported
  platforms
  
  Additional Keywords:  IBMJCEPlus ibmjceplus.jar cloud upload
  

Local fix

• IBM Storage Protect Server bundles java 8.0.8.21 in v8.1.23
  package and bundles java 8.0.8.30 in  v8.1.24 package.   The
  java package in v8.1.23 and v8.1.24 packages are missing
  ibmjceplus.jar.  The Java packages from IBM fix central includes
  ibmjceplus.jar
  .
  Steps to fix the issue
  1: Download according java package from IBM FixCentral :
  
  https://www.ibm.com/support/fixcentral/
  select:
  Runtimes for Java Technology
  Version : select Java 8.0.8.21 for v8.1.23 and Java 8.0.8.30 for
  v8.1.24
  Platform: select the proper platform
  
  2: save the download package to the affected IBM storage Protect
  server and extract the package
  3:  schedule a time to halt IBM Storage Protect Server.
  
  
  Copy the following contents from download:
  /java_extract/jre/fips140-2/lib/ext/ibmjceplus.jar
  /java_extract/jre/fips140-3/lib/ext/ibmjceplus.jar
  
  Save them to server location, after copy, the server location
  should have:
  
  /opt/tivoli/tsm/jre./fips140-2/lib/ext/ibmjceplus.jar
  /opt/tivoli/tsm/jre/fips140-3/lib/ext/ibmjceplus.jar
  
  4:restart IBM Storage Manager server
  5:  after server comes back online, verify using "show JVM" ,
  the command should show:
  ...
  Crypto Security Providers: 1. IBMJCEPlus 2. IBMJSSE2 3. IBMJCE
  4. IBMJGSSProvider 5. IBMCertPath 6. IBMSASL 7.
  IBMXMLCRYPTO 8. IBMXMLEnc 9. IBMSPNEGO 10. SUN
  

Problem summary

• ****************************************************************
  * USERS AFFECTED:                                              *
  * IBM Storage Protect Release 8.1.23.000 onwards               *
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  * See error description.                                       *
  ****************************************************************
  * RECOMMENDATION:                                              *
  * Apply fixing level when available. This problem is currently *
  * projected to be fixed in level 8.1.25. Note that this is     *
  * subject to change at the discretion of IBM.                  *
  ****************************************************************
  

Problem conclusion

• This problem was fixed.
  Platforms fixed:  AIX, Linux, and Windows.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  TSM SERVER

• Fixed component ID

  5698ISMSV

Applicable component levels