April 2, 2019 By Snehal Gawas 3 min read

Two changes related to Watson Studio and Watson Knowledge Catalog

We’d like to announce two changes related to Watson Studio and Watson Knowledge Catalog that will take effect on May 17, 2019 (1) and July 12, 2019 (2) respectively:

  1. Shaping data and running data refinery flows will use enhanced spark environments that add flexibility and change the capacity unit hours needed.

  2. Data profiling and classification will be available from Watson Knowledge Catalog only.

Enhanced spark environments to shape data and run data refinery flows

Data refinery provides data preparation and visualization capabilities in both Watson Studio and Watson Knowledge Catalog. You use data refinery to shape your data and run data flows to create new datasets for your analytical projects. We use Capacity Unit-Hours as the unit of measure for the amount of processing consumed to run data refinery flows. Each Watson Studio or Watson Knowledge Catalog plan includes a set amount of Capacity Unit-Hours (CUH). Overage is charged as per-plan when the account has used up the included amount of CUHs.

Current case

Currently, data refinery flows use a dedicated Spark cluster that requires 6 Capacity Units per hour. In addition, there is a minimum charge of 0.96 Capacity Unit-Hours each time you initiate a data flow job.

Proposed changes

Effective May 17, 2019, data refinery flows will run using the default Spark environment and require only 1.5 Capacity Units per hour, which is lower than the current rate and provides greater value. We are also removing the minimum charge related to running data flows.

However, effective July 12, 2019, shaping data in the interactive UI will start consuming Capacity Units per hour. Note that this is a new charge. Here are some data refinery usage examples and their associated CUH consumption:

If you need additional processing capacity for big data flows, you can select another custom Spark environment. The associated capacity unit hours can vary based on the performance specified. Learn more about custom Spark environments here. Effective July 12, 2019, a user will also get an option to select a non-distributed R runtime for data flow runs. 

These changes provide three key benefits:

  • Reduced rates when running data refinery flows using the default Spark environment

  • Greater flexibility in selecting the appropriate Spark environment for the job

  • A pricing model that reflects the actual usage of data refinery services

Data profiling and classification available only with Watson Knowledge Catalog

Data profiling and classification processes samples of data assets to extract statistics and insights on their content. It includes whether the data contain personal or sensitive information, such as names, emails, national identifications, and credit card numbers. You can access the data profile from the Profile tab when viewing the data asset in a project or catalog.

Currently, you can run profiling and classification if you have either Watson Studio or the Watson Knowledge Catalog. Effective July 12, 2019, you must have Watson Knowledge Catalog in order to run profiling or access the profile of a data asset from a project or a catalog.

Please note that the profile and classification feature (available on the preview of data in the project or catalog) is different than the profile tab in the Data Refinery tool. There are no changes in the profile tab of the Data Refinery tool.

There is no change to the current number of capacity units needed to run profiling jobs at 6 Capacity Units per hour. A minimum charge of 0.96 Capacity Unit-Hours applies each time you run a profiling job. The consumed capacity units will be deducted from the set amount included with the provisioned Watson Knowledge Catalog plan.

How to access profiling and classification

Note that the sign-up process for Watson Studio includes the no-cost Lite plan of Watson Knowledge Catalog by default. Therefore, unless you have explicitly removed the Watson Knowledge Catalog app from the account or during the sign-up process, you can continue to access profiling services from the included Watson Knowledge Catalog Lite plan.

If you only have Watson Studio and want continued access to data profiling, you can add Watson Knowledge Catalog to your account at any time.

For both of these updates, there are no changes in price metrics for CUH (0.50 USD/Capacity Unit-Hour). For the most up-to-date pricing and plans, please refer to the catalog pages for Watson Studio and Watson Knowledge Catalog.

More from Analytics

In preview now: IBM watsonx BI Assistant is your AI-powered business analyst and advisor

3 min read - The business intelligence (BI) software market is projected to surge to USD 27.9 billion by 2027, yet only 30% of employees use these tools for decision-making. This gap between investment and usage highlights a significant missed opportunity. The primary hurdle in adopting BI tools is their complexity. Traditional BI tools, while powerful, are often too complex and slow for effective decision-making. Business decision-makers need insights tailored to their specific business contexts, not complex dashboards that are difficult to navigate. Organizations…

IBM unveils Data Product Hub to enable organization-wide data sharing and discovery

2 min read - Today, IBM announces Data Product Hub, a data sharing solution which will be generally available in June 2024 to help accelerate enterprises’ data-driven outcomes by streamlining data sharing between internal data producers and data consumers. Often, organizations want to derive value from their data but are hindered by it being inaccessible, sprawled across different sources and tools, and hard to interpret and consume. Current approaches to managing data requests require manual data transformation and delivery, which can be time-consuming and…

A new era in BI: Overcoming low adoption to make smart decisions accessible for all

5 min read - Organizations today are both empowered and overwhelmed by data. This paradox lies at the heart of modern business strategy: while there's an unprecedented amount of data available, unlocking actionable insights requires more than access to numbers. The push to enhance productivity, use resources wisely, and boost sustainability through data-driven decision-making is stronger than ever. Yet, the low adoption rates of business intelligence (BI) tools present a significant hurdle. According to Gartner, although the number of employees that use analytics and…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters