Preparing to migrate from v5 Public Cloud

Plan the migration, deploy API Connect v10, download v5 Public Cloud data, and set up the migration utility.

Procedure

Complete the following steps:

Plan your migration:
1. Review all of the migration topics to ensure you understand the requirements, the general process, and important considerations.
2. Plan your new v10 deployment.
  
  When moving from v5 to v10, use the opportunity to review your current resources, such as user registries, provider organizations, and so on. You might want to look at the existing data in v5 Public Cloud and remove or consolidate data before running the extract. This will help ensure only objects that are still needed in v10 will be migrated, and that your data will require less clean-up in the new deployment.
3. Plan to do run several test iterations of migration.
  
  You can use the migration utility's --dry-run option to perform iterations of the migration push without actually uploading data to v10. The test runs offer an opportunity to test for errors and warnings that need to be resolved before you perform the actual data push.
4. Prepare a pilot migration of your APIs.
  
  API Connect v10 on-premises supports both a v5-compatible DataPower Gateway and the newer DataPower API Gateway, which has different requirements.
  
  To ensure a smooth migration, convert 10-to-20 of your most commonly used v5 APIs to meet the requirements of the API Gateway. Use one or more test runs to ensure that APIs do not generate errors during migration. Then, migrate the converted APIs and test them in v10 to verify that they can be used in the new environment, and to determine the appropriate migration utility settings before you convert your remaining APIs.
Install and configure API Connect v10.
Configure v10 with usage of the incoming v5 data in mind. Before you push your v5 data to v10, your v10 system must be up and running, with all required gateway services, provider organizations, user registries, and so on, ready to receive the data. In particular, ensure that your v10 deployment meets the following requirements for migration:
- Understand the requirements for the Gateway in v10:
  API Connect v10 supports two types of gateway services: DataPower Gateway (v5-compatible) and DataPower API Gateway (the DataPower API Gateway type is not available in API Connect v5). When you register your gateway service on your v10 system, you must select a gateway service type (see API Connect gateway types).
  
  If you select the API Gateway, then you must complete the following tasks:
  - Ensure that xml-manager is enabled in the default domain of all gateways.
  - Your migration process must include a step to convert the v5 API Definitions from the DataPower Gateway format to the DataPower API Gateway format. For information on converting the v5 APIs, see Converting APIs to DataPower API Gateway.
  - Review the OAuth Security Definitions of APIs that use OAuth.
    To ensure that APIs that use OAuth security definitions are mapped to the corresponding OAuth Provider APIs in v10, make sure that either the Authorization URL or the Token URL defined in OAuth Security Definitions of APIs that use OAuth correctly point to the base path of the OAuth Provider API. Note that the migration utility uses logic to map Consumer APIs to Provider APIs, and adds x-ibm-oauth-provider in the OAuth Security Definition of the consumer API.
- Prepare a provider organization in v10 that will own the v5 Public Cloud assets:
  In v5 public cloud, there is a single provider organization; all of your data migrates to a single provider organization in v10. If the target provider organization in v10 uses the same name as in v5 public cloud, data is migrated automatically. If the names don't match, you can map the names while preparing your data for migration (see Mapping v5 resource names to v10).
- Prepare user registries in v10:
  Configure user registries to match the user registries in v5 Public Cloud. The migration process migrates users and sends email to users about their accounts.
  
  Note: Portal Delegated User Registries are not supported in v10 because they are replaced with equivalent registries that are managed by the Management subsystem. If you used the PDUR feature in v5 Public Cloud, create matching registries in v10 so that you can migrate users to the new registries. For information on mapping a PDUR registry to a v10 registry, see Mapping v5 PDUR registries to v10.
- Create the Portal services that you want to use with the Catalogs that you plan to migrate:
  You will configure the catalogs to the Portals after the migration data is pushed to the v10 system.
Set up the AMU migration utility on a secure computer where you will process the data from V5 Public Cloud before importing it into V10.

The V5 Public Cloud migration archive file must be downloaded to the same computer, so that you can use the AMU to push the V5 Public Cloud data to your v10 deployment.

Note: When running the migration utility commands on Windows, open a command window with Administrator privileges, and omit the "./" from the commands.
1. Install Node 18.20.2 (or greater).
  
  To use the migration utility for converting API definitions for the API Gateway, you must have Node version 18.20.2 (or greater) installed.
2. Download and extract the newest version of the AMU for API Connect 10.0.5 from IBM Fix Central.
  Note: If you do not already have entitlement to use IBM Fix Central, contact IBM Support to request access.
  
  When you unpack the data from v5 Public Cloud using the AMU tool unpack command, it is stored in a directory called cloud within the same folder structure where the AMU is stored. When you extract the AMU, keep the following considerations in mind:
  - Keep the path to the AMU short, especially on Windows.
    Windows has path length limits that can interfere with large extracts. To avoid this issue, keep the path for your extracted data as short as possible (short folder names with as flat a folder structure as you can manage).
  - Do not include a folder named cloud in the path where you extract the AMU.
    When you unpack the data from v5 Public Cloud, it is stored in a folder called cloud within the same folder structure where the AMU is stored. If a folder named cloud already exists, the conflict will cause an error during the unpack process.
3. Start the AMU:
  1. In the folder where you extracted the AMU, navigate to the appropriate subfolder for your operating system (Mac, Linux, or Windows).
  2. Open a command window where you can run AMU commands.
    Note: On Windows: open a command window with Administrator privileges.
  3. Verify that the utility is available by running the apicm command to view the list of available commands; for example:
```
./apicm
```
    Note: On Windows: omit the "./" from the command.
  Tip: Use the apicm <command-name> --help command to review the command options and their purpose. To see the list of commands, run: apicm help.
Get owner access to the v5 Public Cloud provider organization.

Only the provider organization's owner can download the data for migration. If you will migrate data but are not the owner of the provider organization, contact the current owner and have that person complete the following steps to make you the new owner.

Attention: Only the current owner of the provider organization can transfer ownership to another user.
1. Log in to v5 Public Cloud as the owner of the provider organization.
2. Click Admin > Members.
3. In the list of members, locate the row containing the name of the person who will become the new owner of the provider organization.
4. On the row containing the selected user, click > Transfer Organization Owner.
5. At the confirmation message, click OK.
Notes:
- The previous owner of the provider organization loses access to the Admin > Members page.
- The previous owner is removed from the provider organization unless they also owned a catalog (limitation in v5). As the new owner, you can add that person as a member of the provider organization.
- Transferring ownership of the provider organization does not transfer ownership of any catalogs or spaces. The previous provider organization owner still retains ownership of all catalogs and spaces that they previously owned.
Download your v5 Public Cloud data.
Complete this step on the secure computer where you have the AMU migration utility installed.
1. Log in to V5 Public Cloud as the owner of the provider organization.
  
  Only the owner of the provider organization can migrate data.
2. Open the Dashboard, locate the "API Connect Public Cloud instances will expire soon" message and click Download.
3. At the "Select the type of API Connect environment to which the assets will be migrated" message, select API Connect v10 on-premise installation as your migration target, and click Next.
4. At the "Please enter API Connect Service Instance URL from AWS Subscriptions and Services Instances View Details Page" message, type or paste the URL for accessing your instance of API Connect Enterprise as a Service, and click Continue.
  
  You can obtain the URL from the link to your instance that is provided on the IBM Automation management console's Subscriptions page.
5. At the "Please enter an encryption password then press the download button which will start preparing the data" message, create an encryption password:
  The encryption password that you create here is required for extracting the downloaded data. If you lose the password you cannot retrieve it; however you can download the data again with a new password.
  1. In the Encryption Password field, type a new password.
  2. In the Re-enter Encryption Password field, type the new password again to confirm it.
6. At the "Assets Download" message, click the Download button to start the data extraction from v5 Public Cloud.
  
  It might take a few minutes for the assets to be prepared and the button to be enabled.
7. At the "Downloading is in progress" message, click OK.
  
  The extracted data is downloaded as an archive file called <ProviderOrgName>-extract.tgz. Proceed to the next step to extract the data.
Decompress the archive file containing the v5 Public Cloud data.
Run the following command to decompress the <ProviderOrgName>-extract.tgz that you downloaded from v5 Public Cloud:
```
tar xvf "<ProviderOrgName>-extract.tgz"
```
The decompress process creates the following files:
- extract-backup.ecr contains your v5 Public Cloud data. If you did not use the PDUR (Portal Delegated User Registry) feature in v5, this file also contains your registry information.
- (PDUR only) pdurexport.ecr If you used PDUR feature in v5 Public Cloud, then registries were managed by the Portal subsystem and this file contains the registry information from the v5 Developer Portals.
Unzip the decompressed files using the encryption password that you created when you downloaded the archive.
1. Unzip the extract-backup.ecr file by running the following command and replacing with your encryption password:
```
unzip -P "<encryption_password>" extract-backup.ecr
```
  This step creates the extract-backup.tgz file.
2. If available, unzip the pdurexport.ecr file by running the following command and replacing with your encryption password:
  The pdurexport.ecr file is only available if you used the PDUR feature in v5.
```
unzip -P "<encryption_password>" pdurexport.ecr
```
  This step creates the pdurexport.tgz file.

Use the AMU to unpack the extract-backup.tgz and pdurexport.tgz files.

The AMU must be run in a command window with administrative privileges on Windows. You might prefer to run the migration utility from a Linux VM on your Windows computer.

Run the following command to unpack the files (if you do not have a pdurexport.tgz file, omit it from the command):

./apicm archive:unpack ./extract-backup.tgz ./pdurexport.tgz

The output from this command looks like the following example:

    INFO[2022-06-20T13:03:16-07:00] Saving copy of: saveErrorLog to: logs/archive-unpack20220620T13-03-16error.log 
    INFO[2022-06-20T13:03:16-07:00] Saving copy of: saveInfoLog to: logs/archive-unpack20220620T13-03-16info.log 
    INFO[2022-06-20T13:03:16-07:00] Disabled PDUR errors as no PDUR archive was passed in 
    INFO[2022-06-20T13:03:16-07:00] unpacking v5 archive ./extract-backup.tgz    
    INFO[2022-06-20T13:03:23-07:00] Writing unpacked content to /Users/single_porg/osx/cloud 
    .......
    .......
    INFO[2022-06-20T13:03:39-07:00] Completed unpacking of provider_org_name organization contents. 
    INFO[2022-06-20T13:03:39-07:00] Completed unpacking archive ./extract-backup.tgz. 
      Admin Org Stats:
      - Availability Zones: 1
      ......
      Provider Org Stats:
      - Provider Organizations: 1
      ........

The unpack process produces YAML files containing your v5 data formatted for use in v10 and stores them in a folder structure stemming from the cloud folder.

Example output from the unpacked migration data:

cloud
└── provider-orgs
    ├── <providerOrg>
│   │   ├── catalogs
│   │   │   ├── develop
│   │   │   │   ├── catalog-settings.yml
│   │   │   │   ├── catalog.yml
│   │   │   │   ├── configured-gateway-services
│   │   │   │   │   └── g-w-1-8f82
│   │   │   │   │       └── configured-gateway-service.yml
│   │   │   │   ├── members
│   │   │   │   │   └── testuser2-example-com-b9ed.yml
│   │   │   │   ├── products
│   │   │   │   │   ├── account-1.0.0.api.meta.yml
│   │   │   │   │   ├── account-1.0.0.api.yml
│   │   │   │   └── roles
│   │   │   │       ├── owner.yml
│   │   │   │       └── viewer.yml
│   │   │   ├── my-catalog
│   │   │   │   ├── catalog-settings.yml
│   │   │   │   ├── catalog.yml
│   │   │   │   ├── configured-gateway-services
│   │   │   │   │   └── g-w-1-8f82
│   │   │   │   │       └── configured-gateway-service.yml
│   │   │   │   ├── members
│   │   │   │   │   └── testuser2-example-com-b9ed.yml
│   │   │   │   ├── products
│   │   │   │   │   ├── ibm-apim-banka-1.0.0.api.meta.yml
│   │   │   │   │   ├── ibm-apim-banka-1.0.0.api.yml
│   │   │   │   └── roles
│   │   │   │       ├── owner.yml
│   │   │   │       └── viewer.yml
│   │   │   └── sandbox
│   │   │       ├── catalog-settings.yml
│   │   │       ├── catalog.yml
│   │   │       ├── configured-gateway-services
│   │   │       │   └── g-w-1-8f82
│   │   │       │       └── configured-gateway-service.yml
│   │   │       ├── members
│   │   │       │   └── testuser2-example-com-b9ed.yml
│   │   │       └── roles
│   │   │           ├── admin-catalog.yml
│   │   │           ├── deployment-manager-catalog-94d9.yml
│   │   │           ├── developer-catalog.yml
│   │   │           ├── owner.yml
│   │   │           ├── prodmgr-catalog.yml
│   │   │           └── viewer.yml

Troubleshooting:

Path: The unpack command might return an error message stating that the cloud must be empty, as in the following example:
```
ERRO[2022-06-20T13:02:00-07:00] AMUAU000E: The migration tool failed to initialize.
    Reason: The cloud directory /Users/single_porg/osx/cloud must be empty to run the unpack command
```
The error indicates that the location where you extracted the files already contained a folder named cloud, which caused a conflict. Correct the error by extracting the files to a folder that does not already contain a subfolder named cloud.
Invalid characters: The unpack command ensures that catalog names and API properties contain only valid characters; invalid characters are replaced with "-".
Validation issues: The unpack command issues warning messages for APIs and Products that fail validation against the OpenAPI specification.
Correct the APIs and Products before attempting to migrate them into v10.
The unpack command might return error messages related to your PDUR archive missing some information, as in the following example:
```
ERRO[3414] PDUR backup data for IDP PortalDelegatedIdentityProvider-123b81fbe4b06de411580e2e 
(123b81fbe4b06de411580e2f) not provided: not found 
ERRO[3414] PDUR backup data for IDP PortalDelegatedIdentityProvider-123b820de4b06de411580e32 
(123b820ee4b06de411580e33) not provided: not found 
```
If your v5 Public Cloud instance used the PDUR feature, this error indicates that the AMU could not find all of the specified Portal user registries when it unpacked thepdurexport.tgz archive. To resolve this problem, download your data from V5 Public Cloud again, and then re-run the unpack command.

If you are not migrating PDUR data and your command did not include the pdurexport.tgz file, this error indicates that you do in fact have a PDUR registry, and that you should have included the pdurexport.tgz file in the unpack command. To resolve this problem, download your data from v5 Public Cloud again and re-run the unpack command with the pdurexport.tgz file included.
Any other Warnings or Errors that occur during archive:unpack process should be investigated for remediation (see Migration troubleshooting).