Upgrading the server side-by-side

Before you begin

• Before you can complete this upgrade procedure, ensure that two systems are available, a system that is running V8.1.3 of the Cloud APM server and a separate system to upgrade to V8.1.4 of the server.

• When you perform the Cloud APM server upgrade, the Db2® server for the Cloud APM server V8.1.4.0 must be at the same version as the Db2 server used by the Cloud APM server V8.1.3. Check the Db2 version that is in use by the Cloud APM server V8.1.3:

  • If Db2 is installed on the same system as the Cloud APM server, enter the following command to check the Db2 version:

    install_dir/db2/V10.5/bin/db2level

    where install_dir is the directory where Db2 is installed.
  • If Db2 is installed on a remote system, ask the Db2 administrator to check the Db2 server version.

  Important:

  The Db2 server for your V8.1.4 Cloud APM server must be at the same version as your V8.1.3 Cloud APM server during the Cloud APM server upgrade.

  If the Db2 version for your V8.1.3 Cloud APM server is Db2 Advanced Enterprise Server Edition V10.5 fix pack 9 (or later), complete the steps in this technote Upgrading your Cloud APM server from V8.1.3 to V8.1.4 when your Db2 server is running V10.5 fix pack 9 or a later fix pack now to continue your Cloud APM server upgrade from V8.1.3 to V8.1.4.

  If you want to use Db2 V11.1 for your V8.1.4 Cloud APM server, you can upgrade your Db2 server to a supported version of Db2 V11.1 after the Cloud APM server upgrade completes. Follow the procedure in Upgrading the Db2 server to Db2 version 11.5.x.

• If LDAP is enabled on your V8.1.3 Cloud APM server to authenticate Cloud APM console users, complete the following steps on your V8.1.3 Cloud APM server before you back up your V8.1.3 data:

  1. Retrieve the value of the realm attribute from the install_dir/wlp/usr/shared/config/ldapRegistry.xml file.
  2. Check the value of the oauthRealm attribute in the install_dir/wlp/usr/shared/config/oauthVariables-onprem.xml file. If the value of oauthRealm attribute does not match the value of the realm attribute in the ldapRegistry.xml file, update the value of the oauthRealm attribute to match the value of the realm attribute.
  3. Complete the following steps to update the install_dir/wlp/usr/servers/apmui/server-oauth2-tai.xml file to add the user from the install_dir/wlp/usr/servers/server1/cscs/conf/cscsRoleAdmin.conf file:
     1. Find the properties line <properties, and identify the systemUser parameter, if it does not exist you will need to add it in the next step. Identify the closing tag /> for the properties line.
     2. Add a new line or edit the existing line before the /> closing tag as follows:

        systemUser="testuser LDAP distinguished name"

        where testuser matches the user string from the cscsRoleAdmin.conf file, for example:

        systemUser="CN=testuser,CN=users,dc=adtest,dc=mycity,dc=mycompany,dc=com"

        Note: Do not include the user:prefix or realm name that was specified in cscsRoleAdmin.conf.
     3. Confirm that the /> closing tag was not deleted, then save and close the file.

• If your V8.1.3 Cloud APM server is connected to a remote MongoDB, install MongoDB V3.2.12. For instructions, see Installing MongoDB V3.2.12 on your remote system.

• If you modified the trust store password for your Cloud APM server V8.1.3, change the password back to the default password before performing the server upgrade. After the upgrade completes, you can change the password back to your custom trust store password. For details, Changing the password for the shared truststore.
• If a custom password is configured for MongoDB on your V8.1.3 Cloud APM server, you must set the MongoDB password back to the default value before running backup.sh. After the upgrade to version 8.1.4.0 is complete, you can set the MongoDB password back to a custom password. For more information, see Default users and passwords.

About this task

These steps assume that the system that is running V8.1.3 of the Cloud APM server is using the local Db2 server that is installed by default with the Cloud APM server. If the V8.1.3 Cloud APM server system that you are using is connected to a remote Db2 server, you must complete the steps in the Upgrading your server when connected to a remote Db2 server topic and complete the steps in this topic. Start with the steps in Upgrading your server when connected to a remote Db2 server. You are referred to the steps in this topic.

The procedure for upgrading the Cloud APM server from V8.1.3 to V8.1.4 on the two systems involves these general steps:

1. Install the Cloud APM V8.1.3 interim fix 16 (8.1.3.0-IBM-IPM-SERVER-IF0016) or later on your V8.1.3 Cloud APM server.
2. Back up your V8.1.3 interim fix 16 or later server data and configuration files with the backup script that is part of the V8.1.3 interim fix 16 server installation.
3. Install your V8.1.4 Cloud APM server and either restore the V8.1.3 Cloud APM server data from step 2 or complete an automatic backup and restore of your V.8.1.3 server data.
4. Shut down your V8.1.3 Cloud APM server.
5. Reassign the V8.1.3 Cloud APM server IP address or both the IP address and host name to the V8.1.4 Cloud APM server.

If you are using a remote Db2 server, the procedure involves these general steps:

1. Install the Cloud APM V8.1.3 interim fix 16 (8.1.3.0-IBM-IPM-SERVER-IF0016) or later on your V8.1.3 Cloud APM server.
2. Back up your V8.1.3 Db2 databases.
3. Set up the new V8.1.4 SCR database.
4. Restore the V8.1.3 Db2 database data to the new V8.1.4 databases.
5. Back up your V8.1.3 Cloud APM server data and configuration files with the backup script that is part of the V8.1.3 server installation if you are completing a manual backup.
6. Install your V8.1.4 Cloud APM server and either restore the V8.1.3 Cloud APM server data from step 5 or complete an automatic backup and restore of your V.8.1.3 server data.
7. Stop your V8.1.3 Cloud APM server.
8. Optionally, rename your new V8.1.4 databases to use the same database names that were used for V8.1.3.
9. Reassign the V8.1.3 Cloud APM server IP address or both the IP address and host name to the V8.1.4 server.

Complete the following steps as a root user:

1. Download the V8.1.4 Cloud APM server installation image from the download site to a staging location of your choosing.
   See Downloading from Passport Advantage.
2. If you plan to configure the agent images, the Hybrid Gateway image, or both during the server upgrade, download the images.
   For more information, see Download instructions.
3. Extract the server installation files for your offering.
4. Install the V8.1.3 patch on your V8.1.3 Cloud APM server:
   1. Download the V8.1.3 interim fix 16 (8.1.3.0-IBM-IPM-SERVER-IF0016) or later patch from IBM Fix Central on the IBM support site.
   2. Copy the 8.1.3.0-IBM-IPM-SERVER-IF0016.tar file to your V8.1.3 Cloud APM server and complete these steps on the V8.1.3 Cloud APM server.
      1. Extract the patch package:

         tar xvf 8.1.3.0-IBM-IPM-SERVER-IF0016.tar

      2. Run the following script to apply the patch:

         apmpatch.sh

5. If the computer system or virtual machine where you are installing the Cloud APM server with a local Db2 server is using LDAP to authenticate the root user or Db2 users for your Cloud APM server, you must create local Db2 users before installing the V8.1.4. Cloud APM server. Complete steps 1 to 7 in Installing on a system using an external directory service.
6. Verify that the default permissions are set correctly. Open a command prompt and enter umask.
   A value of 0022 is returned if the permissions are set correctly. If any other value is returned, set the permissions by entering the following command:

   umask 0022

7. From the directory where you extracted the installation files, install V8.1.4 of the Cloud APM server on the virtual machine or computer system that you are using for the upgrade.
   1. Take note of the installation location of the V8.1.3 server on the existing system because you must install V8.1.4 in the same directory on the upgrade system.
      The installation path was either the default /opt/ibm directory or a directory that you chose.
   2. If your V8.1.3 Cloud APM server has a local Db2 server with custom passwords set for Db2 users itmuser and db2apm, you must change the values of the database passwords in Cloud APM 8.1.4.0 install.properties before running install.sh:

      db2apm.password=my_custom_password
      itmuser.password=my_custom_password

   3. Start the installation script:

      ./install.sh

8. After you start the installation, when you are asked if you are upgrading from an existing installation of the Cloud APM server, enter 1 (yes) to continue with the upgrade.
9. When you are asked if you want to move the data and configuration automatically or manually from the existing system, enter 1 (yes) to accept the default and automatically move the data or 2 (no) to complete a manual migration.
   1. If you entered 2 [manually] to move your data from the existing V8.1.3 server, you must run the backup as user root on this existing server. The backup.sh script is in the ccm directory of the V8.1.3 Cloud APM server installation directory.
      Important:

      • If your MongoDB was installed on a remote system, use the backup file that you created when you upgraded MongoDB instead. For more information, see Installing MongoDB V3.2.12 on your remote system.
      If your existing V8.1.3 server is configured with a non-default user name for the Cloud APM user interface administrator account, you must run the following backup.sh script as user root:

      install_dir/ccm/backup.sh [-f ~/backup813.tar] 
      -u uiadmin_username -p uiadmin_password

      For example, if the Cloud APM UI administrator user name is uiadmin and the non-default password for this user is uiadminpwd, enter the following command:

      install_dir/ccm/backup.sh -u uiadmin -p uiadminpwd

      Note:

      • If you do not want the password to be visible by other users, you can use environment variables to provide the password and the user name by entering the following commands:

        export APMADMIN_USERNAME=uiadmin

        export APMADMIN_PASSWORD=uiadminpwd

        Then, run the backup.sh script as user root by entering the following command:

        install_dir/ccm/backup.sh [-f ~/backup813.tar]

        The Cloud APM UI administrator's user name and password is read from the backup during the restore phase.
      • If your Cloud APM server is connected to a remote Db2 server, you can ignore the warning to run the backup script on the remote Db2 server server.
   2. If you entered 1 [automatically] to move your data from the existing V8.1.3 server, you are prompted to provide or accept default values to set up the SSH connection to the existing V8.1.3 server.
      1. name for the Performance Management UI administrator account or accept the default [apmadmin]
      2. password for the Performance Management UI administrator account
      3. hostname/IP address of the remote server

      After you respond to the prompts, a connection is established and a backup of the existing server data and configuration is created. The time that it takes to create the backup depends on the size of the backup.
10. After the backup is finished, you must enter the root password of the V8.1.4 Cloud APM server so you can copy the backup file to this server.
11. If you are manually migrating your data, copy the backup tar image from step 9 to the server where you are installing the Cloud APM server and enter the path and file name that you created (such as /opt/ibm/backups/backup_20160826_155605.tar) when you are prompted.
12. After you are asked whether you want to configure your agent installation images and Hybrid Gateway installation image (if used) to connect to the server, enter either 1 (yes) to configure the images now or 2 (no) to defer configuration of the agent and Hybrid Gateway images.
    If you entered 1 (yes), you are prompted to confirm the following information:

    • The path to the directory on the server where the agent images and Hybrid Gateway (if used) are stored.
      The agent images and Hybrid Gateway images can be mounted on an NFS partition but must be accessible using the file system.
    • Enter the path to the directory for the configured agent installation images or accept the default install_dir/ccm/depot directory.
    • If you accepted the default directory for storing the configured agent and Hybrid Gateway images, the installer creates the directory install_dir/ccm/depot for storing the configured agent and Hybrid Gateway images. However, if you chose to change the directory, or if the installer fails to create the directory, or the directory is not writable, you are prompted to specify the output directory.

    If you entered 2 (no), this step is skipped.

13. When you are prompted to enter the host name and IP address of the server that will be used in a web browser to log into the Cloud APM console, accept the default values or enter your own values.
    This is the address that users enter to start the Cloud APM console from their web browsers, for example: https://myserver:9443 or http://myserver:8080. You can change the IP address and host name later. See Changing the server IP address and host name.

The V8.1.4 Cloud APM server installation is started.

If the installer detects any agent configuration packages in install_dir/ccm/depot from a previous installation of the Cloud APM server, it warns you that it renamed the old packages and created new agent packages. The old packages are named install_dir/ccm/depot.old.

If the installer detects a keyfiles directory in install_dir from a previous installation of the Cloud APM server, it warns you that it renamed the old keyfiles directory and created a new directory. The old keyfiles directory is named install_dir/keyfiles.old.

A prerequisite scan of your environment starts and takes a few moments to complete. If any requirements are missing, a message directs you to a log file with the reason for the failure, such as insufficient disk space. You must address the failure and start the installation again. A soft prerequisite such as low available memory does not stop the installation but you must enter 1 to continue installing or 2 to stop.

14. When the installation is complete, you can verify that the V8.1.4 upgrade is a success by completing the following steps:
    1. Issue the apm status command from the /usr/bin/ directory on the upgraded V8.1.4 server to view the list of running components. If all the components are running, the upgrade is a success.
    2. If you log in to the Application Performance Dashboard using the same credentials that you used for V8.1.3 and check your applications, groups, and instances for V8.1.4, they are the same as they were for V8.1.3.
15. Shut down the V8.1.3 Cloud APM server system if it is running by entering the following command from any directory:

    shutdown -P now

16. Reassign the V8.1.3 Cloud APM server IP address or both the IP address and host name to the V8.1.4 server. Whether you reassign the IP address only or both the IP address and host name depends on your configuration.
    • If you use the Cloud APM server IP address to connect your agents to your server, complete the steps in Reassigning the server IP address to another server.
    • If you use the Cloud APM server host name to connect your agents to your server, complete the steps in Reassigning the server IP address and host name to another server.
17. Migrate your Hybrid Gateway configuration:
    • To migrate the Hybrid Gateway immediately after a restore, run these commands on the Cloud APM server as the root user:
      1. Copy the restored config.properties file to the ccm directory:

         cp install_dir/ccm/properties/config.properties.restored install_dir/ccm/properties/config.properties

      2. Update the date and time of the config.properties file so that any Hybrid Gateways already running will reload their configuration:

         touch config.properties

      3. Copy the updated config.properties file to the Central Configuration Services component:

         cp install_dir/ccm/properties/config.properties install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/config.properties

    • If any configurations were done on the upgraded system after the restore, complete these steps:
      1. Copy these two lines from theinstall_dir/ccm/properties/config.properties.restored file:

         com.ibm.tivoli.ccm.encryption\:key= 
         com.ibm.tivoli.ccm.gaian.connect\:gaianReq=

      2. Replace the same lines in install_dir/ccm/properties/config.properties with the lines that you just copied.
      3. Copy config.properties to the Central Configuration Services component:

         cp install_dir/ccm/properties/config.properties install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/config.properties

    To verify that the config.properties file in install_dir/wlp/usr/servers/min/dropins/CentralConfigurationServer.war/common/ was replaced, check that the modified date and time are current.

Results

The Cloud APM server upgrade to V8.1.4 is complete. You can now access the latest functions for your agents and other components.

What to do next

• If you are using the default out of the box certificates for accessing the Cloud APM console, you must complete the steps in this technote V8.1.4 Application Performance Management UI certificates are expiring in upgraded environments to update the default certificates to prevent them from expiring in April 2019.
• Before you use the Cloud APM console that you upgraded, clear your web browser cache and restart your browser. Clearing the cache avoids display issues that new capabilities in this update introduced to some of the user interfaces.
• If you want to use the old agent configuration packages from a previous installation for agent installations, complete these steps:
  1. Go to the install_dir/ccm directory.
  2. Delete the agentconfig file.
  3. Change the name of the agentconfig.old file to agentconfig.
• If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server, you must change clientAuthentication to true. Copy the <ssl> xml element that contains the enabledCiphers attribute from the install_dir/wlp/usr/servers/min/server.xml file to the install_dir/wlp/usr/servers/min/user-exit.xml file if it does not already exist in the user-exit.xml file. Then add this clientAuthentication="true" line after the enabledCiphers line in the user-exit.xml file. Remove the <ssl> xml element from the server.xml file. The following code example shows you where to add the clientAuthentication="true" line in the user-exit.xml.

  <ssl
  id="defaultSSLConfig"
  sslProtocol="TLSv1.2"
  enabledCiphers="TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
  clientAuthentication="true"
  serverKeyAlias="server_key"
  clientKeyAlias="IBM_Tivoli_Monitoring_Certificate"
  keyStoreRef="defaultKeyStore"/>

• If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server and used the default certificates, change the communication protocol that the Cloud APM server agents use to HTTPS. For instructions, see Configuring the communications protocol for server agents.
• If you configured HTTPS communication between the Cloud APM server and agents in your V8.1.3 Cloud APM server and did not use the default certificates, complete these steps:
  1. Encode the (xor) server keystore password that you used when you created certificates for the V8.1.3 Cloud APM server:

     /opt/ibm/wlp/bin/securityUtility encode

  2. Copy the <keyStore> xml element from the install_dir/ibm/wlp/usr/servers/min/server.xml to the install_dir/wlp/usr/servers/min/user-exit.xml file if it does not already exist in the user-exit.xml file. Then in the user-exit.xml file, replace the value of the password attribute with the newly encoded password from the step 1. Remove the <keyStore> xml element from the server.xml file.
  3. Go to the install_dir directory.
  4. Delete the keyfiles directory.
  5. Change the name of the keyfiles.old directory to keyfiles.
  6. Update the certificates that are used by the monitoring agents to connect to the Cloud APM server to use the new keystore. For instructions, see Configuring certificates between the server and agents for HTTPS communication.
  7. Update the communication protocol and certificates that are used by the Cloud APM server agents. For instructions, see Configuring the communications protocol for server agents.
• If the system where you installed the Cloud APM server is using LDAP to authenticate the root user or Db2 users, and you updated the passwords for the itmuser and the Db2 instances users when following the procedure referenced in step 5, then complete step 9 in the Installing on a system using an external directory service topic.
• Reconfigure and reinstall the reports by completing the steps in Configuring the reports installation image and Installing reports.
• If you change the host name or IP address of the Cloud APM server and Db2 is installed locally and previously integrated with other products (such as Tivoli Common Reporting), complete these steps:
  1. Complete steps 6, 7, and 8 in Enabling single sign-on between Cloud APM and Tivoli Common Reporting.
  2. Uncatalog the old server node and databases for Tivoli Common Reporting:

     db2 list db|node directory

     (db2 list db2 directory or db2 list node directory)

     db2 uncatalog node node_alias

     db2 uncatalog db db_alias

  3. Recatalog the server node and databases for Tivoli Common Reporting by completing the steps in Configuring an ODBC connection.
• Most V8.1.3 Cloud APM agents are compatible with the V8.1.4 Cloud APM server. However, you must upgrade the following agents after you upgrade the Cloud APM server to version 8.1.4.0:
  • If you are using the Synthetic Playback agent, you must upgrade the agent by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers. Then, you must apply the latest Cloud APM 8.1.4.0 server interim fix that is available from Fix Central.
  • The Monitoring Agent for WebSphere Applications must be upgraded by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers.
  • The Monitoring Agent for MongoDB must be upgraded by using the latest agent package on Passport Advantage. To view the Passport Advantage part numbers for the agent packages, see: Part numbers.