Using Config Sync to replicate catalog data

Requirements

For API Connect Config Sync to run successfully, the following requirements that must be satisfied:

Target cluster configuration

The target API Connect cluster must be configured correctly, including:

• Topology setup
• Provider organization creation
• Catalog creation

Replicating APIs and Products

If you disable the product and API syncing using the SyncProductsAndAPIs environment variable, then you must manually replicate the APIs and products from the source catalog to the target catalog, either through manual procedures or CI/CD pipelines. For more information about disabling API and Product syncing, see the SyncProductsAndAPIs section.

Note: If you need to use Product and API sync feature, ensure that both the source and target API Connect clusters run version 10.0.8.3 or later.

Target catalog settings

The target catalog settings must be configured to match the configuration of the source catalog, including:

• User registries
• OAuth providers
• TLS client profiles
• Lifecycle approvals
• Application lifecycle
• Other relevant catalog settings

If products and APIs syncing is enabled, you must set the catalog_settings.config_sync_target_catalog property to true on the target catalog. This can be done using the toolkit or REST API.

To update the catalog settings using the toolkit, run the following command:

./apic catalog-settings:update -s <server_url> -o <org_name> -c <catalog_name> <catalog_setting_file>

For example:

./apic catalog-settings:update -s https://api.example.com -o test_org -c test_catalog catalog_setting_file.txt

The catalog_setting_file.txt should contain the following JSON:

{
  "config_sync_target_catalog": true
}

Note: Login to the target cluster and run the command there, not in the source.

User registries and OAuth Providers

• Compatible onboarding user registries must be configured in the target catalog with the same name and type as in the source catalog. Ensure that the target configured must be same as the source.
• If this is not possible, use the USER_REGISTRY_NAME_MAPPING variable.
• All OAuth providers must be configured on the target catalog and spaces.
• All API user registries must be configured on the target catalog and spaces.
• All TLS client profiles must be configured on the target catalog and spaces.

Connectivity and permissions

The Config Sync utility uses the API Connect Provider REST API, so you must have connectivity to both the source and target clusters with sufficient provider organization user permissions.

Reserved metadata key

The metadata.source_id key is reserved for use by Config Sync

Resource mapping

Config Sync uses the metadata field of a resource to track which resources in the source catalog map to which resources in the target catalog. In addition to using the metadata field, API Connect Config Sync also uses the name field of some resources to determine a match. This is typically done for resources that are not synchronized by the utility but are needed in the target catalog for the synchronization to be successful. Examples of such resources include:

• User Registries
• Consumer Org Groups
• Roles
• Published Products (where both the name and version fields are used for product matching)

By using a combination of the metadata and name fields, Config Sync can accurately map resources between the source and target catalogs, ensuring a successful synchronization.

Tracking resource mappings

To track which resources in the source catalog map to which resources in the target catalog, Config Sync uses the metadata field of a particular resource. Specifically, when a new resource is created in the target catalog, the metadata field of that resource is set to contain a key named source_id with a value equal to the ID of the corresponding resource in the source catalog that is being synchronized.

This process enables Config Sync to maintain a record of the relationships between resources in the source and target catalogs, ensuring that changes are accurately propagated and that resources are correctly matched across catalogs.

Limitations

API Connect Config Sync has the following limitations:

• Config Sync does not back up data; it merely replicates the contents of a source catalog to a target catalog. You must still take periodic backups to ensure the safety of your data.
• A reverse sync can be performed only if the original source cluster and the original target cluster are at the same VRM (Version.Release.Milestone) level. For example, API Connect 10.0.8.1 and API Connect 10.0.8.0 are at the same VRM level even though they are at different fix pack levels. A reverse sync to a different version, release, or milestone is not supported.
• Source and target deployments operate independently of each other:
  • API rate limiting on Plans is counted independently in the source catalog and the target catalog.
  • OIDC tokens are not replicated. OIDC tokens that are generated in the source cluster are not valid within the target cluster.
  • Analytics data is tracked independently in the source cluster and the target cluster. Analytics data is not replicated or aggregated across clusters.
  • Developer Portal configuration is not replicated. If you want to enable consumer organization access in the target catalog, you must replicate your Developer Portal site for consistency (restore the site with UUID/URL update where needed).
• Config Sync operates on a single source-target catalog pair at a time.

  If you want to sync multiple different source-target catalog pairs, create a separate cronjob for each pair.

Supported versions of API Connect

Config Sync can synchronize consumer-side resources across different versions of API Connect clusters. The target cluster version must always be greater or equal to the source cluster version.

Table 1 shows the supported version combinations of API Connect SaaS:

Consumption reporting

The analytics subsystem records the number of resources that are processed during a config sync. You can see the total number of resources that were processed for a config sync at provider organization and catalog scope in the consumption dashboard of the API Manager UI.

For users on call volume licensing, the amount of data that is synchronized is converted to API calls and is included in your call volume license.