Adding tables from VSAM or IMS data sources to Data Gate for watsonx

The process of adding tables is similar for all supported data sources. However, some of the characteristics of VSAM and IMS data sources require your attention.

Before you begin

Before you can add a table to be synchronized by Data Gate for watsonx, complete the following steps:

  1. Using the Classic Data Architect, map your VSAM or IMS data to a relational table. For this purpose:
    1. Connect the Classic Data Architect to your z/OS® system. This comprises the following steps:
      1. Import the z/OS certificate into the keystore of your Classic Data Architect.
      2. Open the physical data model and add the classic objects, which can be VSAM or IMS tables.
      3. Use the appropriate COBOL copybook for the generation of table mappings and extra statements, such as grant statements.

      VSAM example:

      CREATE TABLE SCHEMA.EMPLOYEE DBTYPE VSAM
      DS "SYS5.CLASICDC.V113.VSAM.EMPLOYEE"
      (
      
      ENAME SOURCE DEFINITION
      DATAMAP OFFSET 0 LENGTH 20
      DATATYPE C
      USE AS CHAR(20),
      PHONE SOURCE DEFINITION
      DATAMAP OFFSET 20 LENGTH 4
      DATATYPE UF
      USE AS INTEGER,
      MAILID SOURCE DEFINITION
      DATAMAP OFFSET 25 LENGTH 6
      DATATYPE C
      USE AS CHAR(6),
      SALARY SOURCE DEFINITION
      DATAMAP OFFSET 31 LENGTH 4
      DATATYPE P
      USE AS DECIMAL(7 , 2),
      JOBID SOURCE DEFINITION
      DATAMAP OFFSET 35 LENGTH 4
      DATATYPE D
      USE AS REAL,
      EMPID SOURCE DEFINITION
      DATAMAP OFFSET 39 LENGTH 4
      DATATYPE UF
      USE AS INTEGER,
      DEPTID SOURCE DEFINITION
      DATAMAP OFFSET 43 LENGTH 2
      DATATYPE UH
      USE AS SMALLINT,
      DEPARTMENT SOURCE DEFINITION
      DATAMAP OFFSET 47 LENGTH 15
      DATATYPE C
      USE AS CHAR(15));
  2. Based on the table you created in CDC with the help of the Classic Data Architect, Data Gate for watsonx creates a schema and a table. An error is displayed if:
    • The table exists.
    • The schema does not exist in the catalog that is associated with the target bucket.
    Important:
    • VSAM or IMS-based source tables names are in uppercase, but the automatically created Iceberg schemas and tables are in lowercase.
    • The first column of the target table includes a timestamp timestamp not null column for internal use.
    • Refer to the following table to understand the automatic mapping.
    • Do not insert blanks in the automatically generated schema, table, or column names.
    • The following information pertains to the synchronization of entry-sequenced datasets (ESDS) in a VSAM source:

      For a successful synchronization, the rows of an ESDS need a unique key. If you do not define such a key (replication key), the key is made up of all the column values of a row. This means that the likelihood of duplicate keys increases. If duplicate keys exist, only the last row with that key is synchronized with the target.

      It is crucial that a predefined key exists or that such a key can be derived from the column values of a row. If such a key cannot be identified, and rows with identical column values are found, the rows are synchronized only if they occur within different units of work. If they occur within the same unit of work, some of the duplicate rows might not be synchronized.

    To identify the resulting target data type, see the following table:

    Source type in Classic Data Architect Target type
    BINARY VARBINARY
    CHAR(N) VARCHAR, VARCHAR(N)
    DECIMAL DECIMAL
    DECIMAL(P,S) DECIMAL(P,S)
    DOUBLEPRECISION DOUBLEPRECISION
    FLOAT DOUBLE
    GRAPHIC VARCHAR
    INTEGER INTEGER
    LONG VARCHAR VARCHAR
    LONG VARGRAPHIC VARCHAR
    REAL REAL
    SMALLINT INTEGER
    VARBINARY VARBINARY
    VARCHAR VARCHAR
    VARGRAPHIC VARCHAR
Important: Before you can add a table from the instance dashboard, you must stop synchronization for all tables.

Procedure

  1. Make sure that the Cloud Pak for Data perspective is selected. If necessary, click Icon (button) to change the perspective to change the perspective.
  2. On the side bar, click Instances.
    You see the Service instances page. All instances are listed in the Name column.
  3. Locate the row representing the instance that you want to view. Click the Open instance icon icon in that row. You might have to scroll to the right.
    You see the dashboard of your instance.
  4. Make sure that synchronization has been stopped for all tables. If not, stop synchronization for all tables on the Overview tab.
  5. From the Table tab of the dashboard, click the Add table button, which is on top of the tables list on the right.
  6. You might want to use the search function to narrow the choice first, especially if the list is long.
    1. Selecting Schema or Table from the drop-down list under Search and select tables for synchronization, you determine what to search for (schemas or tables).
    2. Type the names of schemas or tables in this field, either fully or partially, to display just the objects starting with or containing the search string. The names of schemas tables that have already been selected are not listed.
  7. Select schemas. On the left, you see a list of the table schemas in the connected Db2 subsystems or data sharing groups. Select one or more of the check boxes in front of the schema names.
    The tables of the selected schemas are listed in the box to the right of the schema list. By default, all tables that belong to the chosen schema are selected.
  8. If needed, reduce the number of selected tables by clearing some of the check boxes in front of the table names.
  9. Click Continue.
    You see a summary of your table selection:
    Total schemas
    The total number of schemas involved in your selection.
    Total tables
    The total number of tables you selected.
  10. Click Finish to confirm.
    Important: The process might take several minutes to complete. Do not close the Add tables page during that time.
    Attention: While the synchronization process is running, do not manually change the received data in the storage object from the watsonx.data user interface. Doing so breaks the synchronization process and compromises the integrity of the data.