Getting started with grid project design
When you design grid jobs, you edit the master template using the IBM® InfoSphere® DataStage® Administrator client. The master template defines the static resources that are available in your grid. You can then further define resources for individual projects by generating a project template.
The following list defines terms used in this section:
- Master template
- The master template defines the static resources that are available on your grid. There is only one master template for each grid (master_config.apt). Only the administrator can edit the master template.
- Project template
- A project template represents a subset of the static grid resources that have been assigned to a particular project. The administrator defines which static resources to use for the project, and then uses the InfoSphere DataStage Administrator client to generate a project template. The project template contents are populated to the Grid job properties page. Immediately before job runtime, the Administrator client generates a job configuration file. For examples and more detail about templates, see Configuration file templates.
- Static resources
- Static resources consist of fixed-name servers such as database servers, SAN servers, SAS servers, and remote storage disks.
- Dynamic resources
- Dynamic resources consist of compute nodes that are in the default compute node pool and that are assigned dynamically at job runtime. A default compute node entry ($HOSTNAME) is included in the master template to represent a generic compute node. If selected by the administrator for a project, the default compute node entry is included in the project template. After the requested resources are allocated, the engine uses the default compute node entry and any user-defined job requirements to dynamically generate the configuration file.
Grid resources for a job are resolved as described in the following overview procedure. As indicated in the steps, some actions require the administrator-level permission, some require only the user-level permission.
- The administrator defines static resources in the grid in a master template.
- The administrator defines which grid resources are enabled for a particular project by selecting which resources to include in the project template.
- The user defines resource requirements for particular jobs within that project.
- The user defines resource constraints for particular stages within that job.
- After the job is invoked but immediately before it runs, the resource manager calls InfoSphere DataStage to dynamically generate a configuration file.
By this method, InfoSphere DataStage can assure that jobs run immediately when enough resources of the correct types are available, instead of waiting for a fixed resource that might be tied up on another long-running job.