What's new and changed in Watson Discovery

Installing or upgrading Watson Discovery

Cloud Pak for Data Version 4.6.5

A new version of Watson Discovery was released in April 2023 with Cloud Pak for Data 4.6.5.

Operand version: 4.6.5

This release includes the following changes:

New features

The 4.6.5 release of Watson Discovery includes features and updates.

Manage the data in a collection

You can now access a Manage data page for a collection. From the new page, you can see a list of the documents in your collection and get a quick view of information about the documents. You can also delete documents from a collection with just a few clicks. For more information, see Excluding content from query results in the Watson Discovery documentation on IBM® Cloud.

You now have more control over the data that is crawled by the database connector

When you connect to a database as an external data source, you can now specify the column from which to extract data. If you don't specify the column, a column with text or with a single large object is chosen to be crawled. You can also specify the MIME type of the data in the column that you want to crawl. For details, see the Database data source topic in the Watson Discovery documentation on IBM Cloud.

Several security patches were applied

• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache Lucene
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in LibTIFF
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Jettison
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in XStream
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in F5 NGINX
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache Commons
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Golang Go
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Jettison
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Pypa Setuptools
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Node.js ua-parser-js
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Node.js http-cache-semantics
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache CXF
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache James MIME4J
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in cryptography
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Pallets Werkzeug
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in PostgreSQL
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache Log4j
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in FasterXML jackson-databind
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in VMware Tanzu Spring Framework
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Java
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Spring Framework
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Apache Thrift
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Liquibase
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Netty
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Golang Go

Cloud Pak for Data Version 4.6.3

A new version of Watson Discovery was released in February 2023 with Cloud Pak for Data 4.6.3.

Operand version: 4.6.3

This release includes the following changes:

New features

Version 4.6.3 of the Watson Discovery service includes various fixes.

Several security patches were applied

• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Golang Go
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in Go
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by a vulnerability in JSON Web Token
• Security Bulletin: IBM Watson Discovery Cartridge for IBM Cloud Pak for Data affected by vulnerability in LibTIFF

Cloud Pak for Data Version 4.6.2

A new version of Watson Discovery was released in January 2023 with Cloud Pak for Data 4.6.2.

Operand version: 4.6.2

This release includes the following changes:

New features

The 4.6.2 release of Watson Discovery includes features and updates.

Updated built-in enrichments

The version of the built-in Watson natural language processing enrichments was updated.

Use custom entity models in other projects

Now you can use the models that you build with the entity extractor tool in other projects. You can download any custom entity model and upload it as a machine learning model enrichment that you can apply to your collections.

Use a Watson Knowledge Studio corpus as the starting point for entity extractor model training

You can import a corpus of documents that were annotated in Watson Knowledge Studio to use as the training data for an entity extractor in Watson Discovery.

When you import the ZIP file that you exported from Watson Knowledge Studio, any entities that are annotated in the ground truth are added as entity types to the entity extractor workspace.

Entity subtypes, relations, and custom dictionaries that are associated with the corpus in Watson Knowledge Studio are not represented in Watson Discovery. The annotated documents are stored with the entity extractor workspace, not as a new collection in the project. You can continue to annotate the imported documents when you customize the entity extractor model.

For details, see Entity extractor in the Watson Discovery product documentation on IBM Cloud.

Cloud Pak for Data Version 4.6.0

A new version of Watson Discovery was released in November 2022 with Cloud Pak for Data 4.6.0.

Operand version: 4.6.0

This release includes the following changes:

New features

Version 4.6.0 of the Watson Discovery service includes the following features and updates:

Build a custom type system based on key terms in your data with the Entity extractor

Now you can build a custom type system into Watson Discovery that can recognize, tag, and retrieve important information from your unstructured data. With the new Entity extractor tool, you label examples of significant terms in a sample of your business data. The tool uses your examples to generate a machine learning model for you. When you apply the model to new data, it finds and tags information that is uniquely significant to your business. For details, see Entity extractor in the Watson Discovery product documentation on IBM Cloud.

Visualize the enrichments found in your documents

When you view the passage from a search result, the document preview shows a representation of the original document where the search result was found. For most document types, you can now open an advanced view of the document. The advanced view provides a useful summary, including the total number of occurrences of any enrichments that are detected in the document. You can also select one of the enrichments to highlight every occurrence of the element in the document.

New and updated external data source connectors

You can use the following new and updated data source connectors:

Box

Now you can use the App Access Only access level when you set up the Watson Discovery connector to crawl files that are stored in a Box data store. If you don't want the crawler to have access to all of your enterprise data, you can configure it with more targeted access control so that it can crawl only content that is designated for use by the app. For details, see Box in the Watson Discovery documentation on IBM Cloud.

Windows File System

The connector now supports Microsoft Windows Server 2022. The Windows Agent Service that you use to install the connector on your file system runs on a 64-bit version of Windows. To use the new agent service, you must uninstall and delete the prior version of the agent service. For details on migrating to the new connector, see Windows File System in the Watson Discovery documentation on IBM Cloud.

API improvements

The following APIs are now available from a Cloud Pak for Data instance:

• The Get collection method now returns the smart_document_understanding field, which specifies whether an SDU model is enabled for the collection and indicates the model type. For more information, see the API reference documentation for Watson Discovery on IBM Cloud.
• The Query method now includes the similar parameter, which you can use to find documents that are similar to documents of interest to you. For more information, see the API reference documentation.