Action Required Service Updates

Data Refinery and Profiling Changes in Watson Studio and Watson Knowledge Catalog

Share this post:

Two changes related to Watson Studio and Watson Knowledge Catalog

We’d like to announce two changes related to Watson Studio and Watson Knowledge Catalog that will take effect on May 17, 2019:

  1. Shaping data and running data refinery flows will use enhanced spark environments that add flexibility and change the capacity unit hours needed.
  2. Data profiling and classification will be available from Watson Knowledge Catalog only.

Enhanced spark environments to shape data and run data refinery flows

Data refinery provides data preparation and visualization capabilities in both Watson Studio and Watson Knowledge Catalog. You use data refinery to shape your data and run data flows to create new datasets for your analytical projects. We use Capacity Unit-Hours as the unit of measure for the amount of processing consumed to run data refinery flows. Each Watson Studio or Watson Knowledge Catalog plan includes a set amount of Capacity Unit-Hours (CUH). Overage is charged as per-plan when the account has used up the included amount of CUHs.

Current case

Currently, data refinery flows use a dedicated Spark cluster that requires 6 Capacity Units per hour. In addition, there is a minimum charge of 0.96 Capacity Unit-Hours each time you initiate a data flow job.

Proposed changes

Effective May 17, 2019, data refinery flows will run using the default Spark environment and require only 1.5 Capacity Units per hour, which is lower than the current rate and provides greater value. We are also removing the minimum charge related to running data flows. However, shaping data in the interactive UI will start consuming 0.5 Capacity Units per hour. Note that this is a new charge. Here are some data refinery usage examples and their associated CUH consumption:

If you need additional processing capacity for big data flows, you can select another custom Spark environment. The associated capacity unit hours can vary based on the performance specified. Learn more about custom Spark environments here. A user will also get an option to select a non-distributed R runtime for data flow runs. This runtime will consume 0.5 capacity units per hour.

These changes provide three key benefits:

  • Reduced rates when running data refinery flows using the default Spark environment
  • Greater flexibility in selecting the appropriate Spark environment for the job
  • A pricing model that reflects the actual usage of data refinery services

Data profiling and classification available only with Watson Knowledge Catalog

Data profiling and classification processes samples of data assets to extract statistics and insights on their content. It includes whether the data contain personal or sensitive information, such as names, emails, national identifications, and credit card numbers. You can access the data profile from the Profile tab when viewing the data asset in a project or catalog.

Currently, you can run profiling and classification if you have either Watson Studio or the Watson Knowledge Catalog. Effective May 17, 2019, you must have Watson Knowledge Catalog in order to run profiling or access the profile of a data asset from a project or a catalog.

There is no change to the current number of capacity units needed to run profiling jobs at 6 Capacity Units per hour. A minimum charge of 0.96 Capacity Unit-Hours applies each time you run a profiling job. The consumed capacity units will be deducted from the set amount included with the provisioned Watson Knowledge Catalog plan.

How to access profiling

Note that the sign-up process for Watson Studio includes the no-cost Lite plan of Watson Knowledge Catalog by default. Therefore, unless you have explicitly removed the Watson Knowledge Catalog app from the account or during the sign-up process, you can continue to access profiling services from the included Watson Knowledge Catalog Lite plan.

If you only have Watson Studio and want continued access to data profiling, you can add Watson Knowledge Catalog to your account at any time.

For both of these updates, there are no changes in price metrics for CUH (0.50 USD/Capacity Unit-Hour). For the most up-to-date pricing and plans, please refer to the catalog pages for Watson Studio and Watson Knowledge Catalog.

Watson Studio - Offering Management

More Action Required Service Updates stories
April 29, 2019

Transforming Customer Experiences with AI Services (Part 1)

This is an experience from a recent customer engagement on transcribing customer conversations using IBM Watson AI services.

Continue reading

April 26, 2019

Updated Tutorial: Database-Driven Chatbot

The tutorial on how to build a database-driven chatbot has been updated. It's now simpler to deploy and and offers more options—Slack, Facebook Messenger, Wordpress, and more.

Continue reading

March 27, 2019

Db2 Warehouse Flex Comes to Amazon Web Services (AWS)

In a strategic escalation in our approach to cloud data warehousing, we’re bringing Db2 Warehouse Flex to Amazon Web Services as a fully managed, scalable, and elastic cloud data warehouse.

Continue reading