Job designs
Design jobs that extract data from various sources, transform it, and deliver it to target systems in the required formats.
The following examples show some of the ways that you can design jobs by using IBM® InfoSphere® DataStage®. The key elements of a job design are the stages and the links between the stages. A stage describes a data source, a processing step, or a target system. The stage also defines the processing logic that moves the data from the input links to the output links. Because the stages are flexible and configurable, you can link multiple stages together to satisfy complex business requirements.
These job designs include only a sample of the available stages. The stages that you use depend on your requirements and environment.
- Extract and load data
You can develop jobs to move data between various sources and in various formats. These examples show how you can extract data, identify changes between files, and load data to a database or a data warehouse. - Transform data
You can transform data to satisfy both simple and complex data integration tasks. These examples show how you can join data, summarize data, use a pivot table to reorder data, and identify changes from a previous file. - Enrich data
These examples show how you can enrich customer data with reference data and how you can apply business rules to determine an appropriate course of action. - Cleanse data
You can develop jobs to standardize, verify, and validate data. These examples show how you can standardize customer data, verify international addresses, identify duplicate records, and apply data rules by integrating stages from IBM InfoSphere QualityStage® and IBM InfoSphere Information Analyzer. - Real-time processing
You can develop jobs for real-time data integration. These examples show how you can respond to a web service request and verify data between message queues. - Big data processing
You can develop jobs that exchange data with big data sources. These examples show how you can access files on the Hadoop Distributed File System (HDFS) and augment data with Hadoop-based analytics. - Combine jobs in a sequence job
You can combine jobs in a sequence job to develop larger processing patterns.
