Next-Gen DataStage

With Databand, you can track the execution of your Next-Gen DataStage jobs. Databand scans your Next-Gen DataStage projects every few seconds and reports on collected metadata from any jobs that were run. With the metadata collected, you can use the Databand alerting system to notify your data team about the health of your jobs and the quality of your inputs and outputs.

Creating a Next-Gen DataStage integration in Databand

Complete the following steps to begin monitoring your Next-Gen DataStage projects in Databand:

  1. Click Integrations in the navigation menu.

  2. Click Add integration.

  3. Select Next-Gen DataStage as the integration type.

  4. Choose whether you are an on-prem user or a cloud user.

  5. Follow the additional steps in the section that applies to your user type:

On-prem users

  1. Provide the following details in the integration configuration wizard and then proceed to step 7:

    Integration name
    This input becomes the name of your Next-Gen DataStage integration in Databand.
    Hostname
    The URL you use to access your on-prem installation of Next-Gen DataStage.
    Select authentication method
    Basic authentication
    Username and password
    The credentials that you use to access your on-prem installation of Next-Gen DataStage.
    Advanced settings
    Number of threads
    The number of concurrent threads to use on the Next-Gen DataStage API client. The default value is 2.

Cloud users

  1. Provide the following details in the integration configuration wizard and then proceed to step 7:

    Integration name
    This input becomes the name of your Next-Gen DataStage integration in Databand.
    API key
    Databand uses the API key to authenticate with your Next-Gen DataStage project. To generate a new API key for your user identity, follow the steps that are outlined in IBM Documentation.
    Advanced settings
    Number of threads
    The number of concurrent threads to use on the Next-Gen DataStage API client. The default value is 2.

All users

  1. After you provide the required parameters, click Next. The configuration information that you entered is validated before you can proceed to the next screen. If any issues are detected, you are shown an error message that explains why the connection failed.

  2. After your connection is validated, you are taken to a list of all Next-Gen DataStage projects that you can access based on the provided authentication information. This list includes any projects that you created and any projects that are shared with you. Select the projects that you want the integration to monitor. When you select a project, all runs for any current jobs and any new jobs that are created in the future are monitored.

    A Next-Gen DataStage project can be monitored by only a single integration in Databand. If any of your projects are already part of another Databand integration, you cannot add them to your integration.

  3. Optional: The account ID is the default source name for a Next-Gen DataStage project and all of its jobs. If you want to change the source name, click the pencil icon next to the account ID, enter a new source name, and then click the checkmark to save.

    The source name applies to all instances of that account ID throughout Databand. When you modify the source name, any existing pipelines that are monitored under that account ID, and any future pipelines that are monitored under that account ID also change.

  4. Click Save.

After these steps are completed, the next time a job runs in your Next-Gen DataStage project, you can see it on the Databand Pipelines page. The name of your job in Next-Gen DataStage becomes the pipeline name in Databand.

Editing an existing Next-Gen DataStage integration

  1. Click Integrations in the navigation menu.
  2. Locate your Next-Gen DataStage integration in the list of integrations.
  3. Hover over the row for your integration, then click Edit at the end of the row in the Actions column.
  4. Make the necessary changes to your integration configuration, your selected projects, or both, and then click Save.

Known issues

  • Next-Gen DataStage version 4.7.4 is only supported on Databand version 1.0.58 and later.

Supported data sources

Databand supports accurate parsing of dataset paths for the following data sources:

  • Db2
  • Oracle
  • Hive
  • Impala
  • ODBC
  • SQL Server
  • File