Upgrading from legacy IBM Optim environments

Edit online
Use the nextgen-upgrade utility to upgrade existing test data workflows and connections from a legacy IBM® Optim distributed environment to IBM Optim Test Data Management.

The nextgen-upgrade utility is a standalone Go binary (Linux compatible) that extracts data directly from a legacy IBM InfoSphere® Optim Test Data Management Solution database, processes the extracted data, and generates pipeline JSON definitions that are compatible with IBM Optim Test Data Management. The generated definitions can be imported into the new platform and viewed in the flow builder canvas UI.

The binary is available for download from the following location: 2.0.0 Upgrade.

The utility supports the following workflows:

  • Extract-creation workflow
  • Extract-insertion workflow

High-level upgrade flow

The upgrade process consists of three phases:

Data extraction

The utility connects directly to the legacy OptimDB database (DB2, Oracle, or SQL Server) and exports OPTIMDIR tables to JSON files.

Data processing

The utility reads the exported JSON files, parses and transforms the legacy object definitions, and generates NextGen Optim pipeline JSON files. You can run the utility in assessment mode to review upgrade readiness before generating pipelines.

Pipeline import

The utility imports the generated pipeline JSON files into a running NextGen Optim instance by using its REST API, resolving database connection references automatically.

Supported upgrade artifacts

The nextgen-upgrade utility processes the following object types from the legacy database:
Object type Description
Access definitions Table and relationship selections with SQL predicates and column criteria
Column maps Source-to-target column mappings, including supported masking expressions
Table maps Source-to-target table mappings
Relationships Referential and primary relationship definitions
Extract requests Extract workflow definitions
Insert requests Insert workflow definitions (extract-based only)
Database aliases Source and target database connection profiles
Important: Not all legacy IBM InfoSphere Optim Test Data Management Solution object types are supported. Unrecognized object types are identified by name and reported as unparsed in the upgrade summary.

Limitations

The following limitations apply when upgrading from a legacy environment:

  • Only the distributed platform is supported.
  • Insert requests are processed for extract-based flows only. Archive-insert flows are not supported.
  • The following masking expressions are supported:
    TRANS CCN ('={n|r} [SOURCE-COLUMN]')

TRANS COL ('{UNIQUE|HASH}, [SOURCE=["colname"]], [LENGTH={n|max}], [TRIM=(char)\u]')

TRANS EML ('[=i|n|l|u] [sourcecol | "domain"] ["userpfx"]')

TRANS FILL ('repchar=<c>, length={n|match|max}')

TRANS NID ('SWI=<country_code> [,FMT=<format>] [,MTD={mask|random}] [,SRC=<col>]')

TRANS SSN ('={n|-} [SOURCE-COLUMN]')

TRANS PRO ('=AFF[, MTD=REP][, ALGO={DEF|FPE}][, CASE={UPPER|LOWER}][, LENGTH={n|MAX}][, REMOVE=<chars>]')

AGE([nY][,nM][,nW][,nD][,SF='<source-date-format>'][,{SC|SRCCOL}='<source-column>']) 

LOOKUP([src-col | SRCSEARCH=(src-col1[, src-col2, ...]),] [DEST=(dest-col1[, dest-col2, ...]),] [dbalias.]creator.table({lkp-search-col | LKPSEARCH=(lkp-col1[, lkp-col2, ...])}, {value-col | VALUES=(value-col1[, value-col2, ...])})[, PRESERVE=(NOT_FOUND)])

HASH_LOOKUP([src-col,] [TRIM=([char...] [\u]),] DEST=(dest-col1[, dest-col2, ...]), [dbalias.]creator.table(lkp-search-col, {value-col | VALUES=(value-col1[, value-col2, ...])}))
  • The following column map expression types are supported in column maps:
    • String literal
    • Numeric constant
    • NULL
    • Boolean constant
    • Hexadecimal literal
    • Date or time literal
    • Substring function
    • Random function
    • Sequential function
    • Concatenated expressions
    • Numeric expressions
  • Passwords are not extracted from the legacy database. Database connections are created with placeholder values that must be updated with actual credentials before use.