Migrating DataStage jobs

Edit online

You migrate DataStage® jobs by using ISX files that contain the job information. You can create these ISX files in one of two ways: by using the system command-line interface (CLI) or the graphical user interface (GUI) of the DataStage service.

To migrate a DataStage job, complete the following steps:

  1. Create and export an ISX file by using one of the following methods:
    Option Instructions
    ISTOOL Use ISTOOL to create an ISX file and export the file. For instructions, see Export command for InfoSphere DataStage and QualityStage assets and How to use ISTOOL for EXPORT IMPORT Information Server Components.
    MettleCI Use MettleCI, which is a third-party service, to convert a server job design into an equivalent parallel job design, then create an ISX file and export the file to your system. For more information, see MettleCLI docs.
    InfoSphere Information Server Manager GUI client Use the Information Server Manager GUI client to export the ISX file. For detailed instructions, see Exporting assets
    Note: Make sure that the ISX file export includes any dependencies, such as parameter sets and table definitions.
  2. Import the ISX file:
    1. Open an existing project or create a new one.
    2. From the Assets tab of the project, click New asset, then select the DataStage tile.
    3. Click the Local file tab, then upload an ISX file from your local computer.
    4. Click Create.
  3. If you have connections that use JDBC JAR files, reload the JAR files to IBM Cloud Pak® for Data. See Reloading JAR files for connections that use JDBC JAR files.
  4. If the export did not pick up the specific values for the parameters of hostname or port number from the original job, you must manually specify the information in the job settings. See Specifying hostname and port number information.
  5. If necessary, update job settings for any missing properties. See Updating job settings for missing properties.
  6. If necessary, update credential information. See Updating credential information.

Reloading JAR files for connections that use JDBC JAR files

If you have connections that use JDBC JAR files, you must reload the JAR files to IBM Cloud Pak for Data.
To upload the JAR files, log in as a Cloud Pak for Data administrator with the Administer platform permission, then complete the following steps:
  1. Locate a JDBC connection in a project, then open it.
  2. Click Upload new file to upload JDBC drivers JAR files.

Next, edit the Generic JDBC connection to use the JAR files. Go to Add to project > connection. Select the Generic JDBC connection. For more information, see Generic JDBC connection (Watson Studio and Watson Knowledge Catalog.

Specifying hostname and port number information

In jobs in the traditional version of DataStage, data connection information might be specified in the form of a parameter reference.

When you import one of these jobs into the modern version of DataStage, the connection information is not recognized, and the import process produces warnings. To address the warnings, manually specify the information in the DataStage job settings. Then, recompile the job.

The following table shows the connections that might have this issue, the default values of the parameters, and what fields in traditional DataStage that the parameter information is derived from.

Note: Some traditional connection names map to multiple modern connection names and some modern connection names map to multiple traditional connection names.
Table 1.
Connection name in traditional DataStage Connection name in modern DataStage Field that the original value was derived from in traditional DataStage Parameter that receives the parameter reference Default value Notes
Sybase Enterprise SAP ASE Server host port = 5000  
Sybase OC  SAP ASE Server host port = 5000  
Db2® Db2 (optimized) Instance advanced.host advanced.port = 50000  
Db2 Enterprise Db2 (optimized) Server advanced.host advanced.port = 50000  
Oracle Oracle (optimized) Server oracle_service_name oracle_db_port = 1522  
Oracle Enterprise Oracle (optimized) Server oracle_service_name oracle_db_port = 1522  
Hive Apache Hive Server host
  • port = 10000
  • database = default
  
Netezza® Enterprise Netezza (optimized) Database database
  • hostname = <hostname>
  • port = 5480
  
FTP Enterprise FTP URI host
  • connection_mode = SSH
  • port = 22
 The file_name property will also be set to a default in the associated flow stage. The default value will be <file_name>.

In addition to manually updating the connection properties and parameter value, you must also change this usage property within the stage, then compile the job.
Azure Storage Connector Microsoft Azure File Storage accountName, accountKey N/A https;AccountName=accountName;
AccountKey=accountKey		 If either accountName or accountKey are parameter references, the connection string is replaced with a parameter reference for that field.

Updating job settings for missing properties

When you import certain jobs from the traditional version of DataStage into the modern version, some properties not specified in the original job. If the unspecified properties are host, database, or database type, DataStage automatically parameterizes the properties. For example, host is parameterized to #ConnProp_<connectorName>_host#.

A special case for the parameter naming is ODBCConnectorPx and PxODBC, which may insert dsn_type into the name similar to ConnProp_[connectorName]_[dsn_type]_hostname. This pattern applies to JDBC, also (ConnProp_JDBC_[protocol]_[property]). If the string portion protocol is readable it will be carried over, otherwise it be cut from the string and take the form ConnProp_JDBC_[property]. This case applies if dsn_type or protocol are available. This scenario also applies to ODBC.

The following values have defaults:
Apache Cassandra
local_datacenter = #ConnProp_Cassandra_local_datacenter# will have a default value of "datacenter1"
Apache Hive
database = #ConnProp_Hive_database# will have a default value of "default"
DRS as Oracle
service_name = #ConnProp_Oracle_service_name# will have a default value of "orcl"
Informix® CLI
database = #ConnProp_Informix_database# will have a default value of "default"
Informix Enterprise
database = #ConnProp_Informix_database# will have a default value of "default"
Informix Load
database = #ConnProp_Informix_database# will have a default value of "default"
Informix XPS Load
database = #ConnProp_Informix_database# will have a default value of "default"
Oracle
service_name = #ConnProp_Oracle_service_name# will have a default value of "orcl"
Oracle Enterprise
service_name = #ConnProp_Oracle_service_name# will have a default value of "orcl"

For Db2, the field names have periods in them. These periods are dropped when the names are converted into parameter references.

If the missing property is port, the import process produces warnings. To address the warnings, manually specify the information in the DataStage job settings. Then, recompile the job.

The following table shows the connections that might have missing properties, the properties that cannot be retrieved, the default value that will be given, and the field that the original value was derived from in traditional DataStage.

Note: Some traditional connection names map to multiple modern connection names and some modern connection names map to multiple traditional connection names.
Table 2.
Connection name in traditional DataStage Connection name in modern DataStage Missing properties that are automatically given a default parameter Missing properties that are automatically given a default value Notes
ODBC Connector ODBC
  • hostname
  • database
  • port = 0
  • dsn_type = DB2
  
ODBC Enterprise Connector ODBC Enterprise
  • hostname
  • database
  • port = 0
  • dsn_type = DB2
 If dsn_type is not in the list of allowed values, the default is DB2.
JDBC Connector Generic JDBC jdbc_driver   You must edit the connection for this property and ensure that the driver JAR file is in the runtime pod.
DRS Connector ODBC, Oracle, Db2 (optimized) (not available in the modern version of DataStage) advanced.hostname advanced.port = 0
  • The connection, by default, will be migrated as a Db2 connector.
  • If you use a parameter for Database type, DRS is migrated as Db2 by default.
Informix Load Informix
  • host
  • password
  • username
 port = 1526  
Informix Enterprise (if remote server = true) Informix
  • host
  • server
 port = 1526  
Informix Enterprise (If remote server = false) Informix
  • username
  • password
  • host
  • server
 port = 1526  
Informix CLI Informix
  • host
  • username
  • password
 port = 1526  
Informix XPS Load Informix host port = 1526  

Updating credential information

For the following stages, you must replace some properties after the import process is finished:
  • Google BigQuery
  • Google Cloud Storage
  • Google Cloud Pub/Sub
After the import process is finished, specify the correct values for the following fields :
  • client_id
  • client_secret
  • access_ token
  • refresh_token