Generating a job by using a template
You can use a job template to generate a job that automates and simplifies data movement, taking data from a source X and moving it to a target Y.
Rather than having to craft a job by using the DataStage® canvas and palette, and then dragging connectors and stages to build a job, you can use the new job template to follow a simple series of steps and generate a parallel job. You can then use this parallel job to move the data. Also, if you are an administrator you can define target rule sets that define best practices and have users apply those target rule sets to their job templates and eventual generated jobs.
Administrator
From the Projects dashboard, an administrator can open the rule sets dashboard. From this dashboard, the administrator creates and manages the following types of target rule sets:
- Target connector rule set
- For a supported connector type, the target connector rule set specifies basic connector properties that can be used in a job template.
- Target naming rule set
- The target naming rule set defines the target asset name prefix or suffix, target column name prefix or suffix, and define more columns that can be used in a job template.
User
- Source assets
-
You identify one of the following source types:
- Connection
- For a connection, you select an existing supported connection, schema, one or more tables, and which columns from the source are to be carried forward.
- Local file
- For local files, you can drag files from their local system or select them through a file browser. The .csv or .txt files are then quickly processed to identify the column metadata.
- Remote directory or pattern
- For a remote directory or pattern, you can specify a fixed directory (for example:
/data/sales/) or a pattern (for example:
/data/sales/2020*.csv). You can also specify whether to include matches that
are found in subdirectories. You can then select one or more files that matched the remote directory
or pattern.
Click the lock icon if you need to specify that the list of files to be processed is unknown. This situation occurs because each time the generated job is run different files might be in the remote directory. If you use this option, the first file found will be used as the column metadata for all other files when the job is generated.
- Target
- You identify a target type, connection, or remote directory. You can optionally apply a target connector rule set, as well as a target naming rule set. For a connection, you select an existing supported connection and schema.
- Mapping
- You can review the source asset to target mappings. You can modify the table action, file action, write mode, target asset name, and target column metadata.
- Summary
- You can review a textual description of the prior steps.
Examples
- Db2® as a source and a target
- Local file source and remote directory target
- Source and target remote directory