5 Things to Know about a Data Refinery from IBM® DataWorks™
MandyChessell 1200004PHV Visits (12010)
This post is written by Mandy Chessell - IBM Distinguished Engineer, Master Inventor, Chief Architect for Information Solutions.
Here are five things to know about a Data Refinery from IBM® DataWorks™:
This includes moving data between data stores and verifying that a data structure is correctly populated with valid values. For example Joe is a developer building a Bluemix application that manages travel offers. How does Joe ensure that the new daily offers are copied into the application each morning?
2. Integrating and refining data takes both interactive services and long running activities.
The data refinery services are REST APIs. Some are simple requests. However long running data refinery services need an additional asynchronous mechanism called an activity. An activity’s function is defined by an activity pattern. There is a catalog of activity patterns. Each activity pattern defines a type of activity and the parameters required to call it. In our example, Joe selects the data load activity pattern. He fills out the template with details of the source and target data store. The completed activity pattern template is called an activity.
3. An activity is reusable.
It can be stored for later use. For example, Joe could create activities for each of the data sources that the Travel Offers application needs and they can be stored ready for use by the program code.
4. An activity executes when it is passed to the data refinery services.
The data refinery creates an instance of an activity that performs the actions defined in the activity. This is called an Activity Run. Joe’s activity requests that the daily travel offers are uploaded to the Travel Offers Bluemix application.
5. Activities can be scheduled or called on demand.
Joe uses the Bluemix scheduler to ensure that the data load activity runs every day. He also adds a button to the Travel Offers administrator’s page to run the data load activity on request. The administrator can also request a list of the activity runs, showing both the currently executing and completed activity runs.