April 2, 2019 By Snehal Gawas 3 min read

Two changes related to Watson Studio and Watson Knowledge Catalog

We’d like to announce two changes related to Watson Studio and Watson Knowledge Catalog that will take effect on May 17, 2019 (1) and July 12, 2019 (2) respectively:

  1. Shaping data and running data refinery flows will use enhanced spark environments that add flexibility and change the capacity unit hours needed.

  2. Data profiling and classification will be available from Watson Knowledge Catalog only.

Enhanced spark environments to shape data and run data refinery flows

Data refinery provides data preparation and visualization capabilities in both Watson Studio and Watson Knowledge Catalog. You use data refinery to shape your data and run data flows to create new datasets for your analytical projects. We use Capacity Unit-Hours as the unit of measure for the amount of processing consumed to run data refinery flows. Each Watson Studio or Watson Knowledge Catalog plan includes a set amount of Capacity Unit-Hours (CUH). Overage is charged as per-plan when the account has used up the included amount of CUHs.

Current case

Currently, data refinery flows use a dedicated Spark cluster that requires 6 Capacity Units per hour. In addition, there is a minimum charge of 0.96 Capacity Unit-Hours each time you initiate a data flow job.

Proposed changes

Effective May 17, 2019, data refinery flows will run using the default Spark environment and require only 1.5 Capacity Units per hour, which is lower than the current rate and provides greater value. We are also removing the minimum charge related to running data flows.

However, effective July 12, 2019, shaping data in the interactive UI will start consuming Capacity Units per hour. Note that this is a new charge. Here are some data refinery usage examples and their associated CUH consumption:

If you need additional processing capacity for big data flows, you can select another custom Spark environment. The associated capacity unit hours can vary based on the performance specified. Learn more about custom Spark environments here. Effective July 12, 2019, a user will also get an option to select a non-distributed R runtime for data flow runs. 

These changes provide three key benefits:

  • Reduced rates when running data refinery flows using the default Spark environment

  • Greater flexibility in selecting the appropriate Spark environment for the job

  • A pricing model that reflects the actual usage of data refinery services

Data profiling and classification available only with Watson Knowledge Catalog

Data profiling and classification processes samples of data assets to extract statistics and insights on their content. It includes whether the data contain personal or sensitive information, such as names, emails, national identifications, and credit card numbers. You can access the data profile from the Profile tab when viewing the data asset in a project or catalog.

Currently, you can run profiling and classification if you have either Watson Studio or the Watson Knowledge Catalog. Effective July 12, 2019, you must have Watson Knowledge Catalog in order to run profiling or access the profile of a data asset from a project or a catalog.

Please note that the profile and classification feature (available on the preview of data in the project or catalog) is different than the profile tab in the Data Refinery tool. There are no changes in the profile tab of the Data Refinery tool.

There is no change to the current number of capacity units needed to run profiling jobs at 6 Capacity Units per hour. A minimum charge of 0.96 Capacity Unit-Hours applies each time you run a profiling job. The consumed capacity units will be deducted from the set amount included with the provisioned Watson Knowledge Catalog plan.

How to access profiling and classification

Note that the sign-up process for Watson Studio includes the no-cost Lite plan of Watson Knowledge Catalog by default. Therefore, unless you have explicitly removed the Watson Knowledge Catalog app from the account or during the sign-up process, you can continue to access profiling services from the included Watson Knowledge Catalog Lite plan.

If you only have Watson Studio and want continued access to data profiling, you can add Watson Knowledge Catalog to your account at any time.

For both of these updates, there are no changes in price metrics for CUH (0.50 USD/Capacity Unit-Hour). For the most up-to-date pricing and plans, please refer to the catalog pages for Watson Studio and Watson Knowledge Catalog.

More from Analytics

A new era in BI: Overcoming low adoption to make smart decisions accessible for all

5 min read - Organizations today are both empowered and overwhelmed by data. This paradox lies at the heart of modern business strategy: while there's an unprecedented amount of data available, unlocking actionable insights requires more than access to numbers. The push to enhance productivity, use resources wisely, and boost sustainability through data-driven decision-making is stronger than ever. Yet, the low adoption rates of business intelligence (BI) tools present a significant hurdle. According to Gartner, although the number of employees that use analytics and…

VeloxCon 2024: Innovation in data management

3 min read - VeloxCon 2024, the premier developer conference that is dedicated to the Velox open-source project, brought together industry leaders, engineers, and enthusiasts to explore the latest advancements and collaborative efforts shaping the future of data management. Hosted by IBM® in partnership with Meta, VeloxCon showcased the latest innovation in Velox including project roadmap, Prestissimo (Presto-on-Velox), Gluten (Spark-on-Velox), hardware acceleration, and much more. An overview of Velox Velox is a unified execution engine that is built and open-sourced by Meta, aimed at…

How the Recording Academy uses IBM watsonx to enhance the fan experience at the GRAMMYs®

3 min read - Through the GRAMMYs®, the Recording Academy® seeks to recognize excellence in the recording arts and sciences and ensure that music remains an indelible part of our culture. When the world’s top recording stars cross the red carpet at the 66th Annual GRAMMY Awards, IBM will be there once again. This year, the business challenge facing the GRAMMYs paralleled those of other iconic cultural sports and entertainment events: in today’s highly fragmented media landscape, creating cultural impact means driving captivating content…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters