Migration mapping files

You can use mapping files to map the name of a v5 object to a corresponding v10 object that has a different name.

The API Connect migration process allows you to make modifications to the configuration information after it has been extracted from the source v5 system and before it is uploaded to the target v10 system.

The names of some structures may not match exactly between v5 and v10. Since the names of some high-level objects may be intentionally different in the v10 environment, you can substitute values so that references to v5 object names (user registries, gateway services etc.) can be bound to the newly created objects in v10.

Note: If the names are preserved exactly between the v5 and v10 environments, you do not need to map any names.

What is a mapping file?

What does a mapping file do?
Mapping files are simple yaml files that may be placed at any level in the directory structure produced by the apicm archive:unpack command where there is a yaml file describing a configuration object. A mapping file contains yaml key: value pairs that provide the values to apply in place of the original configuration values.
  • You can use mapping files to override any values within the .yml files.
  • Common mappings can be specified in mapping.yml.
  • Server specific mappings can be provided in mapping.<servername/ip>.yml. For example: mapping.hostname.com.yml for hostname.com server.
What kinds of settings can I map?
You can override any values within a .yml files. The most common mappings for v5 migrations are:
  • Gateway service names
  • Provider organization names
  • User registry names
What does a mapping file contain?
A mapping entry consists of two or three lines:
  • The first line of the mapping entry should be the file name you want to override, without the .yml extension. For example, when overriding a value in provider-org.yml, the first line of the mapping is provider-org.
  • Additional lines have key: value pairs that override key: value pairs found in the main .yml file. The most common usage is for overriding the name. To do this include the name: setting, followed by the new value for v10. For example, if you have a v5 provider organization myv5porg, and you want to map it to myv10porg, the second line is name: myv10porg.
    provider-org:
      name: myv10porg
  • There are a few exceptions for some additional fields with special meaning in the mapping file for gateway services. See Mapping a gateway service.
Where do I create mapping files
Examine the /cloud directory hierarchy that you created when you ran apicm archive:unpack. Locate the directory that has the .yml file with the v5 value you want to overwrite. In that same directory, create a new file named mapping.yml. For a detailed example, see the Mapping tutorial.

Mapping a gateway service

You can use a mapping file to tell the migration program the name of the target gateway service in v10 to use when migrating from v5 gateway service data to v10 gateway service data. You might want to map a v5 gateway service name to a v10 gateway service if the names do not match, or if the number of gateway services does not match.

Note: You can map a v5 gateway service to a v10 DataPower API Gateway. In this case, the mapping definition requires a gateway-type field. See the Mapping tutorial.

After you unpack the v5 data, you can find each of v5 gateway services in cloud/admin-org/availability-zones/availability-zone-default/gateway-services/.

For example, the following cloud/admin-org/availability-zones/availability-zone-default/gateway-services/ hierarchy shows entries for three v5 gateways named: gateway-2536, gateway-2537, and gateway-2538.

Figure 1. Example admin-org with entries for three v5 gateway services
admin-org
├── availability-zones
│   └── availability-zone-default
│       ├── availability-zone.yml
│       └── gateway-services
│           ├── gateway-2536
│           │   ├── gateway-service.yml
│           ├── gateway-2537
│           │   ├── gateway-service.yml
│           └── gateway-2538
│               ├── gateway-service.yml

Mapping a migrated OAuth shared secret key

Information about migrated OAuth shared secret keys is contained in your gateway-service.yml file. When you map a gateway to a new name, you must ensure that the OAuth shared secret key is associated with mapped gateway name.

Note: Make sure you only migrate one OAuth shared secret key per gateway service. You will get an error if you attempt to map more than one to the same gateway service.
  1. Determine if your gateway-service.yml includes information for an OAuth Shared Secret Key (oauth_shared_secret:). For example:
    api_version: 2.0.0
    metadata:
      v5_created_at: 2018-12-01 17:43:03.065 +0000 UTC
      v5_created_by: admin
      v5_id: 5c02c827e4b0e816d3874747
      v5_updated_at: 2018-12-04 15:59:03.282 +0000 UTC
      v5_updated_by: admin
    name: gw3-cf31
    oauth_shared_secret: 0xF2DB3AE67A7988C06FF1EBA5C2B51F04ABC5D3D1192B4C994CA00C79FAB5ADAD
    type: gateway_service
  2. If oauth_shared_secret: is found, add migrate_oauth_shared_secret: true to the mapping.yml. For example:
    
    gateway-service:
      name: gateway-10-1 
      migrate_oauth_shared_secret: true
     

Mapping SNI settings

You can include Server Name Indication (SNI) settings in the mapping file. When enabled, these settings, which include the host and TLS Server Profile pairs, are included in the migration of gateway services.

Note: You can migrate only one set of SNI settings for a v10 gateway service. If you are mapping multiple v5 gateway services to a single v10 gateway service, only the first set of SNI settings is migrated and warnings are generated for the additional mappings.
  1. Determine whether your gateway-service.yml file contains SNI settings to migrate, as in the following example.
    api_version: 2.0.0
    gateway_service_type: datapower-gateway
    metadata:
      v5_created_at: 2022-02-02 20:07:29.566 +0000 UTC
      v5_created_by: admin
      v5_id: 61fae481e4b0b16dad38691a
      v5_updated_at: 2022-02-02 20:07:29.566 +0000 UTC
      v5_updated_by: admin
    name: gateway-service6-fabd
    oauth_shared_secret: 0x014D004DC0A32EF091C17FEFCF10A621C36A92CE88BCE0F8FC2463B096D72D99
    sni:
    - host: internal.apic.ibm.com
      tls_server_profile_url: idurl://../../../../tls-server-profiles/gatewaytlsprofile-1.0.0.yml
    - host: partner.apic.ibm.com
      tls_server_profile_url: idurl://../../../../tls-server-profiles/default-ssl-profile-1.0.0.yml
    type: gateway_service
  2. If SNI settings are found, add migrate_sni: true to the mapping.yml. For example:
    gateway-service:
      name: gateway-10-1 
      migrate_sni: true

Mapping tutorial

You create and bring up a new gateway service in v10 called gateway-10-1. You want the data (e.g Catalogs) associated with the v5 gateway gateway-2536 to be migrated to the v10 gateway named gateway-10-1.

  1. Go to the /cloud hierarchy and locate the gateway-2536/ directory.
    
    │       └── gateway-services
    │           ├── gateway-2536
    │           │   └── gateway-service.yml
  2. Open the gateway-service.yml and verify that the name: entry specifies the v5 gateway service that you want to map.
    
    api_version: 2.0.0 
    v5_id: 5c0icef4e45816d32290fb
    name: gateway-2536
    type: gateway-service
  3. To map name: gateway-2536 to a new value, use a text editor to create a new file named mapping.yml.
    1. On a new line, enter gateway-service:.
      How do I know what to enter on the first line?
      Enter the name of the file that you are overriding, without the .yml suffix. In this case, you are overwriting gateway-service.yml, so you enter gateway-service:.
    2. On the next line, indent two spaces (as required for valid yaml), and enter name: gateway-10-1.
      
      gateway-service:
        name: gateway-10-1 
      
      How do I know which name to use when mapping the v10 gateway-service?

      In the v10 Cloud Manager UI, go to the Topology page. In the Default Availability Zones table, locate the Service column and click the gateway service entry. On the Edit DataPower Gateway Service page, in the Gateway Details section, locate the value in the Name field. Use this value.

      Note: Do not use the value in the Title field. The value in the Name field is sluggified to contain only characters that are supported in v10, and thus might differ from the value in the Title field.

    3. If the new v10 gateway service gateway-10-1 is a DataPower API Gateway, add a third line to specify the gateway type:
      
      gateway-service:
        name: gateway-10-1 
        gateway-type: datapower-api-gateway
       

      If the new v10 gateway service gateway-10-1 is a v5-compabile gateway, do not specify gateway-type. When gateway-type is not specified, the mapping defaults to v5-compatible.

    4. The mapping is complete. Save the file in the same directory.

      The following diagram shows the location of the new mapping.yml file in the cloud/ hierarchy for admin-org. Note that it is in the same directory as the gateway-service.yml file for the v5 service it applies to (gateway-2536).

      Figure 2. Example admin-org with mapping.yml for gateway-2536 gateway service entry
      admin-org
      ├── availability-zones
      │   └── availability-zone-default
      │       ├── availability-zone.yml
      │       └── gateway-services
      │           ├── gateway-2536
      │           │   ├── gateway-service.yml
      │           │   └── mapping.yml
      │           ├── gateway-2537
      │           │   ├── gateway-service.yml
      │           └── gateway-2538
      │               ├── gateway-service.yml

    When the mapping utility (apicm archive:push) runs, it automatically uses the new name: value found in mapping.yml to overwrite the v5 value found in gateway-service.yml. Note that the gateway service on v10 must be up and running

Do I have to edit the existing gateway-service.yml?
No, just create a new mapping.yml file and add it to the same directory as gateway-service.yml.
What if I want to map several v5 gateway-services to one v10 gateway-service?
If you have several v5 gateway services, and you want to map each of them to the same v10 gateway service, gateway-10-1, you can copy your newly edited mapping.yml into the directory for each gateway service. Under each gateway entry, place a copy of your new mapping.yml.

The following diagram shows the locations of the three copies of mapping.yml files (one each for gateway-2536, gateway-2537, and gateway-2538) in the cloud/ hierarchy for admin-org.

Figure 3. Example admin-org with mapping.yml files for three v5 gateway services
admin-org
├── availability-zones
│   └── availability-zone-default
│       ├── availability-zone.yml
│       └── gateway-services
│           ├── gateway-2536
│           │   ├── gateway-service.yml
│           │   └── mapping.yml
│           ├── gateway-2537
│           │   ├── gateway-service.yml
│           │   └── mapping.yml
│           └── gateway-2538
│               ├── gateway-service.yml
│               └── mapping.yml

Mapping a provider organization

The apicm: archive:unpack command produces a directory structure that includes a subdirectory for each provider organization in the original system. A provider organization named onlinebanking would be represented by a directory name cloud/provider-orgs/onlinebanking, containing a file named provider-org.yml:

api_version: 2.0.0
name: onlinebanking
org_type: provider
state: enabled
title: OnlineBanking
summary: migrated onlinebanking
metadata:
  v5_id: 5c0ebfdae4b007df531ffe40
...

To override the organization name (indicated by the name property), create a file named mapping.yaml in the same directory as the provider-org.yml file, and include the new name to apply to the organization. Example mapping.yml:

provider-org:
  name: digitalbanking

Here, the result of the override is to cause a provider organization named digitalbanking to be used as the target provider organization for the contents originally extracted from the onlinebanking organization.