Setting up a test staging environment with production data

Edit online

Learn how to use the server rename feature to prepare a staging environment with actual production data. A staging environment is a test sandbox that is isolated from the production environment. It can be used to try out new features or functions with real data without impacting the production database.

Before you begin

To perform a server rename, you must obtain a feature key file from IBM Software Support. When you contact IBM Support, mention that you are requesting a server rename feature key file. The key file is named ImportURLMappings.activate. Copy the file to the JazzInstallDir/server/conf directory for the applications that you will rename. The key is only needed to execute the repotools importURLMappings command.
Before you proceed with the rename, check that the required version of the ELM software is installed and that your environment meets the criteria for a server rename. See Software version requirements and Supported scenarios for using server rename for details.
Decide whether you are going to move the deployment to new hardware or keep the same hardware (rename-in-place). In the following steps, your current server environment is called the source environment. The server environment that you are moving to is called the target environment. If you are performing a rename-in-place, the target environment refers to a new location on the same server, or set of servers in a distributed deployment.
Review the topics in the Impact of server rename on the IBM Engineering Lifecycle Management section to understand the actions you need to take when the Jazz® Team Server and ELM applications are renamed.
Review the Impact of server rename on Lifecycle Query Engine and the Link Index Provider topic.
Before you proceed with the rename, review the post-rename steps for Report Builder.
Decide whether you are going to keep the existing application server or migrate to a new one. If you are planning to move to a new application server, see one of the following topics as appropriate to your environment: Switching from Apache Tomcat to WebSphere Application Server or Changing WebSphere Application Server installations and instances.
Decide whether you are going to keep the existing database or move to a new one. If you are planning to move to a new database, see one of the following topics as appropriate to your environment: Moving the IBM Engineering Lifecycle Management (ELM) databases to a different database server or Changing the IBM Engineering Lifecycle Management (ELM) databases to a different database vendor.
Setting up server rename can take several attempts. If your setup takes more than one attempt, you will need to restore your environment to a clean state between attempts. For instructions on restoring to a clean state, see Recovering from common problems.

About this task

The server rename feature uses a mapping file to determine the URLs to be renamed. A repotools command is provided to generate an initial mapping file for you. The mapping file contains source-target pairs for the Jazz Team Server and all ELM applications, as well as any other URLs contributed by applications. See Mapping file for server rename for details about the mapping file. See Topologies and mapping files for setting up a test staging environment for sample topology diagrams and mapping files.

When you are planning a staging environment, you need to be especially careful not to contaminate production data or vice versa. The staging environment must be isolated with no connections to any part of the production environment, including the production database. To isolate the staging environment, complete several steps:

Add entries to the hosts file, if it is allowed in your environment
Create mappings to mask the servers that are not present in your staging environment. In the mapping, be sure to include any additional ELM servers or non-ELM products that might be connected to the production server. For details, see step 3.d. and Dummy mappings for staging environments.

For example, you might have a Change and Configuration Management (CCM) application on one production server with links to CCM on another production server, and links to Quality Management (QM) on still another production server. Furthermore, you might have integrations with non-ELM products, such as Rational® ClearQuest®. In this case, first mask out the URLs of the additional CCM and QM servers. Also, ensure that the URL for Rational ClearQuest is masked out or otherwise mapped to a staged Rational ClearQuest gateway server.

In addition, if you are using IBM HTTP Server, reconfigure the plugin-cfg.xml file to point to the target environment. For details, see step 4.

Important: Do not connect a Engineering Workflow Management client to both the production and staging repositories. Always use a staging workspace for connecting to the staging server.

Procedure

Prepare for the rename by following the steps that are described in the topic Preparing the mapping file.
Be sure to create dummy mappings for any of the affected URLs in the mapping file. This practice avoids cross-linking between the staging and production environments. See Dummy mappings for staging environments for details.

The result of the preparatory phase is a mapping file that is generated on the original, source Jazz Team Server. After you carefully review and edit the mapping file, copy it to the JazzInstallDir\server directory on the target Jazz Team Server in the staging environment.
Back up the production environment and copy text indexes and application configuration files to the target staging server. For distributed systems, go to the appropriate server to copy the files.
Note: If you are performing a rename-in-place and not moving to new hardware, you are copying the environment from one installation to a second installation on the same system.

Because you need to stop the production servers while you perform these steps, be sure to choose a convenient time to perform the backup, preferably right before you set up the staging environment. If any changes are made to the production environment before you proceed to the rename, you need to regenerate the mapping file before you import it into the staging server. While the servers are down, users are unable to create or traverse links from any external systems that are integrated with the ELM deployment about to be renamed.
1. Stop the Jazz Team Server and any distributed ELM applications that are registered with the Jazz Team Server in the production environment.
  In addition, stop any other applications that will be affected by the server rename, such as the Jazz Build Engine, test tool adapters that are running and configured to use a Quality Management server that is to be renamed, or synchronizers that integrated with an ELM application about to be renamed. See Impact of server rename on integrated products for a complete list of integrated products that would be effected by a server rename.
2. Back up all of the databases in the ELM production environment, including the Jazz Team Server database; the databases for the Change and Configuration Management, Quality Management, Global Configuration Management, IBM Engineering Lifecycle Optimization - Engineering Insights, Data Collection Component, Lifecycle Query Engine, Link Index Provider, and Report Builder applications; and the data warehouse database. Copy the databases from the production to the staging environment by following the steps in Moving the CLM database.
  
  Note: The Report Builder database is in the server/conf/rs/db folder.
3. Copy the JFS/text indexes from the source production server to the target staging server.
  The following examples for a Linux® server assume that the drives for the staging computers are mounted on the network. If it is not possible to mount the drives for the staging computers on the network, use other file transfer methods to ensure that the files are copied.
```
cp -R SourceJazzInstallDir/server/conf/jts/indices TargetJazzInstallDir/server/conf/jts
cp -R SourceJazzInstallDir/server/conf/ccm/indices TargetJazzInstallDir/server/conf/ccm
cp -R SourceJazzInstallDir/server/conf/dcc/indices TargetJazzInstallDir/server/conf/dcc
cp -R SourceJazzInstallDir/server/conf/gc/indices TargetJazzInstallDir/server/conf/gc
cp -R SourceJazzInstallDir/server/conf/qm/indices TargetJazzInstallDir/server/conf/qm
cp -R SourceJazzInstallDir/server/conf/rm/indices TargetJazzInstallDir/server/conf/rm
cp -R SourceJazzInstallDir/server/conf/relm/indices TargetJazzInstallDir/server/conf/relm
cp -R SourceJazzInstallDir/server/conf/lqe/indexTdb TargetJazzInstallDir/server/conf/lqe
cp -R SourceJazzInstallDir/server/conf/lqe/textIndex TargetJazzInstallDir/server/conf/lqe
cp -R SourceJazzInstallDir/server/conf/lqe/shapeTdb TargetJazzInstallDir/server/conf/lqe
cp -R SourceJazzInstallDir/server/conf/lqe/shapeText TargetJazzInstallDir/server/conf/lqe
cp -R SourceJazzInstallDir/server/conf/ldx/indexTdb TargetJazzInstallDir/server/conf/ldx
cp -R SourceJazzInstallDir/server/conf/ldx/textIndex TargetJazzInstallDir/server/conf/ldx
```
4. Copy the application configuration files from the source production server to the target staging server. (For distributed systems, go to the appropriate server to copy the files.) As with the previous step, the next examples are for a Linux server and assume that the drives for the staging computers are mounted on the network.
```
cp SourceJazzInstallDir/server/conf/jts/teamserver*.properties  TargetJazzInstallDir/server/conf/jts
cp SourceJazzInstallDir/server/conf/ccm/teamserver*.properties  TargetJazzInstallDir/server/conf/ccm
cp SourceJazzInstallDir/server/conf/dcc/teamserver*.properties  TargetJazzInstallDir/server/conf/dcc
cp SourceJazzInstallDir/server/conf/gc/teamserver*.properties  TargetJazzInstallDir/server/conf/gc
cp SourceJazzInstallDir/server/conf/qm/teamserver*.properties  TargetJazzInstallDir/server/conf/qm
cp SourceJazzInstallDir/server/conf/relm/teamserver*.properties  TargetJazzInstallDir/server/conf/relm
cp SourceJazzInstallDir/server/conf/rm/teamserver.properties  TargetJazzInstallDir/server/conf/rm
cp SourceJazzInstallDir/server/conf/rs/app.properties TargetJazzInstallDir/server/conf/rs
cp SourceJazzInstallDir/server/conf/lqe/dbconnection.properties TargetJazzInstallDir/server/conf/lqe
cp SourceJazzInstallDir/server/conf/lqe/lqe.key TargetJazzInstallDir/server/conf/lqe
cp SourceJazzInstallDir/server/conf/lqe/lqe.node.id TargetJazzInstallDir/server/conf/lqe
cp SourceJazzInstallDir/server/conf/lqe/lqe.properties TargetJazzInstallDir/server/conf/lqe
cp SourceJazzInstallDir/server/conf/ldx/dbconnection.properties TargetJazzInstallDir/server/conf/ldx
cp SourceJazzInstallDir/server/conf/ldx/lqe.key TargetJazzInstallDir/server/conf/ldx
cp SourceJazzInstallDir/server/conf/ldx/lqe.node.id TargetJazzInstallDir/server/conf/ldx
cp SourceJazzInstallDir/server/conf/ldx/lqe.properties TargetJazzInstallDir/server/conf/ldx
```
5. Copy the mapping file to the TargetJazzInstallDir\server directory on the staging server. See Preparing the mapping file for details about the mapping file.
  Note: If any configuration changes are made to the ELM deployment before you perform the next step, you will need to regenerate the mapping file.
6. Restart all of the ELM servers, and allow users to continue to use the production environment. The remaining steps apply to the staging test environment.
Perform the following steps in the staging test environment:
1. Optional: Differentiate the staging servers from the production servers by uploading a theme.
  Learn more about themes: Because the staging and production servers have the same data, it can be difficult to distinguish between the two. To help differentiate the staging servers from the production servers, you can use a theme on the staging servers to make it easy to see which server you are working on. To upload a new theme on the Jazz Team Server, type https://fully.qualified.hostname:9443/jts/admin, click Servers, and then click Themes. To upload a new theme for Change and Configuration Management, type https://fully.qualified.hostname:9443/ccm/admin and click Themes. To upload a new theme for Quality Management, type https://fully.qualified.hostname:9443/qm/admin and click Themes. In all cases, click Browse and select a themes.zip file. You can find sample themes at Uploading a new theme. For more information about themes, see Configuring themes.
2. Restore the database backups from the production environment to the staging environment. A best practice is to use a separate database server in the staging environment to avoid stressing the production database server.
3. Replace the database connection parameters in the JazzInstallDir/server/conf/applicationName/teamserver.properties files so that they point to the connection parameters for the staging copy of the database in the test environment. These parameters include the entries for the jts database, the ccm and qm application databases, the data warehouse database, and updated passwords.
  Attention: This step is important to complete before you continue with the next steps in the task. Before you run the repotools-jts -importURLMappings command, it is important that you verify the teamserver.properties files are pointing to the staging copy of the database in the test environment. Otherwise, if the teamserver.properties files specify connection parameters to the database in the production environment, running the -importURLMappings command will corrupt the production database.
  
  Important: You must update only the database connection parameters in the teamserver.properties files. Do not update any other entries in these teamserver.properties files as these properties are reserved and modified by the server rename process. For example, the public URI property: com.ibm.team.repository.server.webapp.url
4. If it is allowed in your environment, update the hosts file to prevent access to the production system URLs and to prevent accidental edits to artifacts in the production server.
  The goal here is to resolve the host names to a false IP address that is not being used by any of the ELM servers in the production environment. Thus, if a server in the staging environment tries to establish a connection to a production server, the connection will fail.
  
  Make a list of all of the distinct host names that appear in your mapping file. These host names might be source URLs or affected URLs. For each host name, create an entry in the hosts file that maps the host name to a false IP address. You need to edit the hosts file for all of the servers in your staging environment, including third-party servers and client workstations. The following host file shows sample entries for your staging server:
```
 # 10.10.10.10 refers to an IP address that does not host a clm application. 
  10.10.10.10   ProductionServerP1
  10.10.10.10   ProductionServerP2
  10.10.10.10   ProductionServerP3
```
5. Open a command line window in the JazzInstallDir\server directory and import the mappings file into the target Jazz Team Server by using the repotools-jts -importURLMappings command.
  Attention: Before you run the repotools-jts -importURLMappings command, it is important that you verify the teamserver.properties files are pointing to the staging copy of the database in the test environment. Otherwise, if the teamserver.properties files specify connection parameters to the database in the production environment, running the -importURLMappings command will corrupt the production database.
  - If you have an all-in-one deployment, import the mappings file by using the repotools-jts -importURLMappings command as follows:
    repotools-jts.bat -importURLMappings fromFile="C:\mappings.txt"
    
    ./repotools-jts.sh -importURLMappings fromFile="opt/mappings.txt"
    
    The rename begins offline on the Jazz Team Server, before the server is restarted.
  - If you have a distributed deployment and are allowed to map network drives, map a network drive from the Jazz Team Server host to each of the application hosts. Then, create a file (for example, serverConfFile.txt) that contains a list of the remote server/conf directories in your deployment in the following format:
```
 # Remote CCM server
   x:/JazzTeamServer/server/conf
   # Remote QM server
   y:/JazzTeamServer/server/conf
   # Remote RM server
   z:/JazzTeamServer/server/conf
   # Remote DCC server
   w:/JazzTeamServer/server/conf
   # Remote GC server
   v:/JazzTeamServer/server/conf
   # Remote RELM server
   u:/JazzTeamServer/server/conf
   # Remote RS server
   t:/JazzTeamServer/server/conf
```
    Finally, proceed with the repotools-jts -importURLMappings command and add the serverConfFile= parameter as shown next.
    
    repotools-jts.bat -importURLMappings fromFile="C:\mappings.txt" serverConfFile="C:\serverConf.txt"
    
    ./repotools-jts.sh -importURLMappings fromFile="/opt/mappings.txt" serverConfFile="/opt/serverConf.txt"
  - If you have a distributed CLM deployment and are not allowed to remap network drives, proceed with the repotools-jts -importURLMappings command as shown next.
    repotools-jts.bat -importURLMappings fromFile="C:\mappings.txt"
    
    ./repotools-jts.sh -importURLMappings fromFile="opt/mappings.txt"
    
    Then, copy the server/conf/jts/.mappingEvent file to the remote application configuration directories (server/conf/application_name), that is, ccm, dcc, gc, qm, relm, rm, and rs. You must copy the .mappingEvent file after you import the mapping file but before you start the server.
    
    The .mappingEvent file contains information that the applications need to contact the Jazz Team Server at its new location. The .mappingEvent file contents are the same for a Jazz Team Server and its registered applications.
  Verify that the rename is successful by checking the console output and the server/repotools-jts_importURLMappings.log file. If any errors are displayed, or if you realize that you made a mistake in your mapping file, see Troubleshooting server rename to pinpoint and fix the problem.
  
  Note: Rename servers that are running on z/OS® by using sample JCL member HLQ.SBLZSAMP(BLZRENAM). See the comment header of BLZRENAM for detailed customization instructions.
If you are using IBM HTTP Server (IHS), reconfigure the plugin-cfg.xml file to point to the target environment. Otherwise, unless the target system is isolated from the network, there is a risk of being redirected to the production servers that are configured in this file.
Start the Jazz Team Server (unless you already started the server in step 8) and any distributed ELM applications that are registered with the server.
Log in to the Jazz Team Server at https://new host:port/jts/serverRenameStatus. Logging in starts the renaming process. The ELM applications will synchronize with the Jazz Team Server to apply the URL mappings and update their data warehouse data. This process generally takes about 5 minutes for a small data set and up to 30 minutes or more for a large data set. When the rename is complete, you can verify the rename and take any corrective action that is necessary. See Verifying URLs and links after a server rename for details.
Before you complete the verification process, ensure that you performed any additional product-specific verifications that are described in Completing the server rename verification process. When you are confident that the renamed data is correct, click the I have verified the server rename ... check box and click Finish.
The Jazz Team Server and all registered applications exit read-only mode and normal product use can resume.
Note:
- If you introduced dummy mappings to protect the production environment, errors are displayed about these URLs in the Applications and Friends section of the Application Diagnostics page on the staged server.
- If you have multiple linked deployments, do not complete the wizard until you validate the data in all deployments.
If your deployment includes the Global Configuration Management application or the Link Index Provider application and you move either application to a different Jazz Team Server, or you rename the server, you must update the application property on the server Administration page.
1. Log in to the Administration page of the Jazz Team Server. Point your web browser to the following URL: https://fully qualified hostname:9443/jts/admin
  
  Remember: The fully qualified hostname is the host name along with the DNS domain reference of the machine on which the Jazz Team Server is installed.
2. On the Administration page, click the Server tab, and then in the left pane, under Configuration, click Advanced Properties.
3. On the Advanced Properties page, complete the following steps.
  1. If your deployment includes the Global Configuration Management application, go to the Global Configuration SDK section and update the Global Configuration Provider URL property with the URL of the renamed server.
  2. If your deployment includes the Link Index Provider application, go to the Links Component section and update the Link index provider URL property with the URL of the renamed server.
4. At the top of the page, click Save.
If you are including other components in your staging environment, see Impact of server rename on the IBM Engineering Lifecycle Management and Impact of server rename on integrated products.