Migrating a CICS provider API from IBM z/OS Connect OpenAPI 2 to OpenAPI 3

Learn how to migrate your existing APIs running on a z/OS Connect Server (OpenAPI 2) to a z/OS Connect Server (OpenAPI 3) without changing the exposed API.

IBM® z/OS Connect OpenAPI 3 support is available from z/OS Connect V3.0.55. The OpenAPI 3 support is a new solution that is stand-alone and functions differently to the z/OS Connect OpenAPI 2 software. The core of the OpenAPI 3 solution is designed to deliver a simpler API creation process to be contract first to support new requirements, and provide a more flexible management.

You might already have APIs on your z/OS Connect OpenAPI 2 servers. And in the long term, you might not want to manage both OpenAPI variants of z/OS Connect Servers. Since z/OS Connect OpenAPI 3 works differently than z/OS Connect OpenAPI 2, migrating from OpenAPI 2 to OpenAPI 3 is, for the moment, a manual task.

This task is a manual migration process to move your existing APIs running on a z/OS Connect Server (OpenAPI 2) to a z/OS Connect Server (OpenAPI 3) without changing the exposed API.

Throughout this task, the terms OpenAPI 2, OpenAPI 2.0, OpenAPI 3 and OpenAPI 3.0 are used:
  • OpenAPI 2 and OpenAPI 3 generally refer to the z/OS Connect support and the resources that are related to z/OS Connect.
  • OpenAPI 2.0 and OpenAPI 3.0 generally refer to the specifications and the API description document.

Identify the APIs to migrate first

To help you decide which APIs to prioritize, consider the following requirements:

  1. APIs that must present an OpenAPI 3.0 description.
  2. APIs that need to have a more customized JSON payload (for example the JSON schema structure is different from the copybook).
  3. APIs that require more advanced transformation processing than what was supported at the OpenAPI 2 level. For more information on JSONata powerful mapping capabilities, see What is JSONata?.
  4. APIs or API operations that need a more granular authorization control.

Migration process

The following steps are using the Catalog API that is provided as a server template by z/OS Connect (sampleCicsIpicCatalogManager).

  1. Retrieve the existing OpenAPI specification 2 document and convert it to OpenAPI specification 3.0.
    Tip: Start from the existing API description so that the exposed interface is kept as-is.
    1. To retrieve the API OpenAPI 2.0 description, send an HTTP request from a web browser to the z/OS Connect server: HTTP GET https://hostname:httpsPort/apiBasePath/api-docs.

      For example, for the CICS Catalog API, the HTTP request is
      HTTP GET https://hostname:httpsPort/catalogManager/api-docs
      If the authorization interceptor is enabled, the user needs one of the following roles: reader, operator, admin. The z/OS Connect Server provides a Swagger 2.0 description for APIs in JSON format. Swagger 2.0 is now best known as OpenAPI Specification 2.0.
      Figure 1. Retrieve the swagger document through a web browser
      Retrieve the swagger document through a web browser
    2. Save the OpenAPI 2.0 description as a file on your computer.
    3. Convert the OpenAPI 2.0 description to an OpenAPI 3.0 description.
      The OpenAPI specifications are widely used standards, so various tools are available to convert from one version to another. For example:

      To learn more about the OpenAPI specification 3.0, see Creating a simple OpenAPI 3 document or the Creating an API Document in z/OS Connect video series.

      Figure 2. Convert the OpenAPI 2.0 document into OpenAPI 3.0 with Swagger Editor online
      Convert the OpenAPI 2.0 document into OpenAPI 3.0 with Swagger Editor online
      Figure 3. The resulting OpenAPI 3.0 document
      The resulting OpenAPI 3.0 document
    4. Save the OpenAPI 3.0 file on your computer.
    5. Update the OpenAPI 3.0 file to make minor changes. If the API base path is in the servers: array, change the url value to / (slash) and prefix the base path to all the defined paths. To quickly update all the paths, find and replace all ' /' by ' /catalogManager/'.
      Note: The number of spaces before the slash character might vary.
      Warning: YAML is whitespace-sensitive. The indentation for an item needs to be consistent with other items at the same nested level. Tabs should be avoided as they can cause parsing issues.
      For example, if the Swagger Editor generates an OpenAPI 3.0 file that starts with:
      openapi: 3.0.1
      info:
        title: catalog
        description: API for the CICS catalog manager sample application
        version: 1.0.0
      servers:
      - url: /catalogManager
      paths:
        /items:
          get:
            tags:
            - catalog
            operationId: getInquireCatalog
      Then update it to move the API base path, in this case, /catalogManager, to the paths:
      openapi: 3.0.1
      info:
        title: catalog
        description: API for the CICS catalog manager sample application
        version: 1.0.0
      servers:
      - url: /
      paths:
        /catalogManager/items:
          get:
            tags:
            - catalog
            operationId: getInquireCatalog
      The servers array in the OpenAPI 3.0 specification replaces the concept of a unique basePath in the OpenAPI 2.0 specification. This modification allows more flexibility as the servers array can include multiple hosts, ports, and base paths. However, the z/OS Connect Designer does not take the servers array into account.
  2. Prepare and start z/OS Connect Designer.

    In this example, z/OS Connect Designer runs in a Docker container but the z/OS Connect Designer can run on any Open Container Initiative (OCI) compliant container platform, such as Podman. For more information, see z/OS Connect Designer runs in any Open Container Initiative (OCI) compliant container platform.

    Follow the tasks listed in Developing APIs with the IBM z/OS Connect Designer, to:
    • Download and verify the z/OS Connect Designer image and its signature.
    • Prepare the workspace and the connection to the CICS region for the z/OS Connect Designer.
    • Start z/OS Connect Designer.
    • Import the Catalog API OpenAPI 3 document that was generated in Step 1.

    The Preparing a IBM z/OS Connect Designer workspace task and the related tasks show a CICS configuration file that does not use TLS between the Designer and the CICS region.

    All the configurations for the zosconnect_cicsIpicConnection and zosconnect_cicsConnectionGroup elements available in z/OS Connect OpenAPI 2 are also available in OpenAPI 3. So, if the IPIC connection requires an APPLID to be provided or a TLS handshake, the CICS configuration file needs to be updated to establish the appropriate connection.

    For example, if you have the following configuration in your existing z/OS Connect OpenAPI 2 server:
    <zosconnect_cicsIpicConnection id="cicsConn" 
        host="localhost" 
        port="1234"
        zosConnectApplid="ZOSCONN"
        zosConnectNetworkid="ZOSNTWK"
        sslCertsRef="IPICSSLConfig"/>
     
     <ssl id="IPICSSLConfig"
        keyStoreRef="zosConnectRing"
        trustStoreRef="zosConnectRing"/>
    
     <keyStore id="zosConnectRing"
        location="safkeyring:///Keyring.ZOSCONN"
        type="JCERACFKS"
        password="password"    
        fileBased="false"
        readOnly="true"/>
    You can create the following cics.xml file:
    <?xml version="1.0" encoding="UTF-8" ?>
    <server description="IPIC connection to CICS">
        <featureManager>
            <feature>zosconnect:cics-1.0</feature>
        </featureManager>
    
        <zosconnect_cicsIpicConnection id="cicsConn" 
            host="${CICS_HOST}"
            port="${CICS_PORT}"
            zosConnectApplid="${ZOSCONNECT_APPLID}"
            zosConnectNetworkid="${ZOSCONNECT_NETID}"
            sslCertsRef="IPICSSLConfig"/>
        
        <ssl id="IPICSSLConfig"
            keyStoreRef="defaultKeyStore"
            trustStoreRef="defaultTrustStore"/>
        
        <keyStore id="IPICKeyStore"
            location="serverKey.p12"
            password="{aes}APu1HaGpBzYbH+g8xVbuqUgJPz1ydNS9VbpAq1hsFdGt"
            type="PKCS12"/>
        
    </server>
    

    CICS_HOST, CICS_PORT, ZOSCONNECT_APPLID, ZOSCONNECT_NETID are environment variables that need to be defined in the docker-compose.yaml file for z/OS Connect Designer.

    Note: Since the z/OS Connect Designer does not run on z/OS, it cannot use RACF key rings as keystore. Thus, the keyStore type needs to be changed to PKCS12 (or JKS).
    Note: By default, the z/OS Connect Designer is not configured with a user registry, so it cannot enforce authentication. If the CICS region requires a user identity, use authDataRef to refer to a set of static credentials, or authenticate the user with a client certificate. It is best practice to use an encrypted password for the referred zosconnect_authData component, see Create the CICS tutorial workspace using Docker Desktop for an example.
    Figure 4. Example contents of a docker-compose.yaml file
    version: "3.2"
     services:
        zosConnect:
            image: icr.io/zosconnect/ibm-zcon-designer:3.0.81
            environment:
                - CICS_HOST=myHost
                - CICS_PORT=1234
                - ZOSCONNECT_NETID=ZOSNTWK
                - ZOSCONNECT_APPLID=ZCEEDSGN
                - HTTP_PORT=9080
                - HTTPS_PORT=9443
            ports:
                - "9443:9443"
                - "9080:9080"
            volumes:
                - ./apiProject:/workspace/project
                - ./logs/:/logs/
                - ./certs:/config/resources/security/:ro
    Put the p12 keystores in the certs folder. The p12 keystores related to IPIC connections can be obtained from RACF with the RACDCERT EXPORT command, if the certificates are stored in RACF. The keytool utility can then be used to put multiple certificates from different keystores into a single keystore.
    Note: The cics.xml and docker-compose.yaml examples show the use of TLS between the z/OS Connect Designer and the CICS region. If you don’t use TLS, the ssl and keyStore elements are not necessary.

    After you start your z/OS Connect Designer and import the OpenAPI 3.0 document you created in Step 1, you can continue to the next step.

    Figure 5. z/OS Connect Designer
    z/OS Connect Designer information page
  3. Create the z/OS Assets.
    1. Click A plus icon in the z/OS Assets section to add a z/OS Asset.
      Note: A z/OS Asset is a program or resource, which is defined with its connectivity and data structures, which is similar to defining a service in z/OS Connect (OpenAPI 2).
      Figure 6. Add a z/OS Asset
      Add a z/OS Asset
    2. Select Add new z/OS Asset and click Next.
    3. Choose the appropriate z/OS Asset type, complete the information and click Next. The related IBM z/OS Connect Server OpenAPI 2 service project can be used to identify the values to set.
      Figure 7. Define a CICS program asset
      Define a CICS program asset

      Open the related z/OS Connect service project and the appropriate service.properties file.

      Browse the Definition and Configuration tabs to find the configurations.

      Figure 8. Inquire Catalog Service Project Editor: Definition
      Inquire Catalog Service Project Editor: Definition

      Figure 9. Inquire Catalog Service Project Editor: Configuration
      Inquire Catalog Service Project Editor: Configuration

      The connection reference might differ if you specified a different id value in the z/OS Connect Designer configuration files.

      Note: Bidirectional text conversion is not supported by the CICS z/OS Asset provider for z/OS Connect OpenAPI 3.
    4. Import the request and response data structures.

      Use the same data structures that were imported in the z/OS Connect OpenAPI 2 service project, and check the selection box to include the structures.

      Figure 10. Add z/OS Asset - import data structure
      Add z/OS Asset - import data structure
    5. Assign a name to the z/OS Asset.
      If it’s relevant, you can use the name of the z/OS Connect OpenAPI 2 service. For more information about z/OS Assets, see Creating and using z/OS Asset.
      Note: If the OpenAPI 2 API uses different services, you might need to define extra z/OS Assets per target program, or with different data structures.
  4. Create the request mapping for each operation.

    The API operations are marked as incomplete with a An amber exclamation mark. Each operation needs to be associated with a z/OS Asset and mappings need to be created for the request and the responses.

    1. Select an API operation and click the A plus icon in the Operation Viewer to add an existing z/OS Asset.
      Figure 11. Add an existing z/OS Asset
      Add an existing z/OS Asset
    2. Define the mapping for each request structure field by assigning values from the HTTP request (JSON fields, query parameters, path parameters, HTTP headers). The Edit mapping can be obtained by selecting the z/OS Asset, and then in the z/OS Asset window by going into Request and Edit mapping.

      To facilitate the creation of the mapping, the Insert a mapping icon and Insert a function icon to the side of the selected field lists the available fields and select functions as shown in Create mappings.

      Figure 12. Create mappings with z/OS Connect Designer
      Create mappings with z/OS Connect Designer
      Note: Some fields might not require a mapping, in which case, they can be left blank.

      In the z/OS Connect OpenAPI 2 API project, the default-mapping behavior is to pass over all fields unless they have been removed. For requests, all the fields are exposed as JSON fields and are passed to the backend program. And for responses, all the fields returned by the backend program are sent back to the caller.

      z/OS Connect OpenAPI 3 uses a contract-first approach so the previous default behavior is no longer applicable. All the necessary fields need to be explicitly mapped in order to be passed to the backend program, otherwise they are ignored. For suggestions of common mappings and how to convert a mapping from API Toolkit (OpenAPI 2) to z/OS Connect Designer (OpenAPI 3), see Table 1.
      Table 1. Common mapping comparisons between z/OS Connect API Toolkit (OpenAPI 2) and z/OS Connect Designer (OpenAPI 3)
      z/OS Connect API Toolkit (OpenAPI 2)
      z/OS Connect Designer (OpenAPI 3)
      Assign a hardcoded value (for example 01INQC).
      Image to show hard coded value of 01INQC in z/OS Connect Designer

      Assign a default value (for example 0) if the request parameter is not provided.
      Image to show startItemID in a boolean expression in z/OS Connect Designer
      If startItemID exists then the condition is true, so the assigned value is the first option. If the condition is false then the assigned value is the hardcoded second option.
      Direct mapping.
      Image to show startItemID in a boolean expression in z/OS Connect Designer

      Table 2 and Table 3 provide examples of how to convert a mapping from API Toolkit to z/OS Connect Designer.

      Table 2. Request mappings
      z/OS Connect API Toolkit (OpenAPI 2)
      z/OS Connect Designer (OpenAPI 3)
      BOOLEAN to FIELD
       
      Boolean to field with API Toolkit Boolean to field with z/OS Connect Designer

      This example converts the boolean field ca_new_user to 1 when it’s true, and 0 when it’s false.

      ARRAY COUNTER to FIELD
       
      Array counter to field with API Toolkit Array counter to field with z/OS Connect Designer

      The $count function returns the number of items in the CA-CAT-ITEM array. The value can then be assigned to the relevant counter field.

      Table 3. Response mappings
      z/OS Connect API Toolkit (OpenAPI 2)
      z/OS Connect Designer (OpenAPI 3)
      FIELD to BOOLEAN
       
      Field to boolean with API Toolkit Field to boolean with z/OS Connect Designer

      The $boolean function casts the argument to a boolean. An empty string is cast to false. A non-empty string is cast to true. The number 0 is cast to false. A non-zero number is cast to true.

      Field to boolean with API Toolkit Field to boolean with z/OS Connect Designer

      This example tests the value of CA-CHARGE-DEPT. If it is equal to the character 1, then the returned value is true, otherwise the returned value is false.

      Field to boolean with API Toolkit Field to boolean with z/OS Connect Designer

      This example tests the value of CA-CHARGE-DEPT. If it is equal or lower than the numerical value 100, then the returned value is true, otherwise the returned value is false.

      DATE support
       
      Date support with API Toolkit Date support with z/OS Connect Designer

      This example uses the $substring function to extract the month, day and year from the host date format (here, MM/dd/yyyy). The string concatenation forms a date in ISO 8601 format (yyyy-mm-dd). Note that the example mapping is on a single line.

  5. Create the response mapping for each return code.
    1. Replicate the HTTP return code conditions from the OpenAPI 2 API.
      Figure 13. API Toolkit Response HTTP return codes
      API Toolkit Response HTTP return codes

      Use the Responses node in the z/OS Connect Designer to define which response is returned by the API. You can set conditions for each response within an API operation. For more information about how to define response mappings, see Defining responses.

      Tip: Configure an error response as the default response code. The API document might not have a 500 response defined. For more information about configuring error responses, see Handling errors in response mappings.
      Warning: If a default HTTP 500 is used, API consumers may observe a different behavior compared to the existing OpenAPI 2 APIs.
    2. Add response mappings for each return code.

      There are many ways to map data in z/OS Connect Designer from the OpenAPI 3 API to the z/OS Asset. See Table 3 for examples. For more information about data mapping and JSONata in z/OS Connect Designer, see Mapping your API and z/OS Assets. For sample JSONata expressions that illustrate its powerful mapping capabilities within z/OS Connect Designer, see Sample JSONata expressions.

  6. Test the API project.

    Test your API project with z/OS Connect Designer by using the Open Liberty REST Client. The z/OS Connect Designer contains a z/OS Connect Server for iteratively developing and testing your API. For more information about testing an API with z/OS Connect Designer, see Testing an API with z/OS Connect Designer.

Security aspects

Consider security when the API or API requester is deployed to the server runtime. For more information, see Securing IBM z/OS Connect resources. For specific information about configuring security for connections to Systems of Record, see API provider security for z/OS Assets.

Note: With IBM z/OS Connect OpenAPI 3, security is not enabled by default.

To enable authentication and authorization, enable the appSecurity-2.0 feature and verify that the authorization roles are specified in the OpenAPI 3.0 document for API provider. An invoke authorization role is already added to each API requester WAR file.

Restriction:
  1. OpenAPI 3 APIs do not support the OpenAPI 2 authorization interceptor and its associated server.xml configuration.
  2. OpenAPI 3 APIs do not support the OpenAPI 2 requireAuth and requireSecure attributes.
  3. The OpenAPI 3 API requester does not support the OpenAPI 2 API requester identity assertion. However, it is now possible to pass credentials, for example a PassTicket, from the z/OS application.