What's new and changed in Orchestration Pipelines

Orchestration Pipelines updates can include new features and fixes. Releases are listed in reverse chronological order so that the latest release is at the beginning of the topic.

You can see a list of the new features for the platform and all of the services at What's new in IBM Software Hub.

IBM Cloud Pak for Data Version 5.2.0

A new version of Orchestration Pipelines was released in June 2025.

This release includes the following changes:

New features
This release of Orchestration Pipelines includes the following features:
Secure your projects by using the secure proxy setting
You can now secure your projects by turning on the secure proxy setting. When secure proxy is enabled, users can only select assets or job-related inputs from the current project. This is a project-level setting.
New pipeline nodes
You can now use the following new nodes in your pipelines:
  • Run Spark Application Job: Use this new node to run Spark applications.
  • Run Structured Ingestion Job: Use this new node to run structured ingestion jobs.

Both of the new pipeline nodes require IBM® watsonx.data™ to be installed. Without watsonx.data, the new nodes will not be visible in the node palette.

Quickly move pipelines between folders for easier pipeline management
You can now quickly and easily move a pipeline to a different folder within the same project by editing its location from the Pipeline information panel.
Track the status of pipelines that use the DataStage optimized runner
You can now see the compile status of pipelines that use the DataStage optimized runner by viewing the Project assets tab. Pipelines are only compiled after you start a pipeline run. Possible statuses include stale, not compiled, or successfully compiled.
Updates
The following updates were introduced in this release:
Performance improvements during job runs
Job node logs can now only be viewed once the node is in a finished state. They will not be polled during the run.
New DataStage Super Operator role
Users with the DataStage Super Operator role can view pipelines in read-only mode, define pipeline job run configurations, and run pipelines and their associated jobs. DataStage Super Operator users cannot edit or create pipelines.
Additional node name restrictions
There are new restrictions and rules about naming nodes:
  • Node names can only contain alphanumeric characters and underscores.
  • Job node names that are automatically updated when a job is selected will have any special character sequences replaced with an underscore.
  • Any existing nodes will retain their names, but if they are edited, the new validation requirements will be enforced.
Expression usability enhancements
The following new enhancements have been added for expression usability:
  • Expressions referencing a node (node output or loop info) can now use the node name if it adheres to the new validation requirements.
  • If a node ID is used, it will automatically be converted to the node name if the node name meets the new validation requirements, ensuring backward compatibility.
  • Existing expressions will be automatically updated if any node references have a node name that meets the new validation requirements.
  • Expressions referencing a node will automatically be updated whenever the node name is updated.

All these usability enhancements apply only to the Pipelines UI. Expressions used in the API or pipeline JSON must still use node IDs.

Customer-reported issues fixed in this release
For a list of customer-reported issues that were fixed in this release, see the Fix List for IBM Cloud Pak® for Data on the IBM Support website.