Updating projects and metadata with a new embedding model after upgrading

Migrating projects that are enabled for natural language queries

In projects that are enabled for natural language queries, you must repeat the onboarding of data assets. After you identify the projects that are enabled for natural language queries, complete the following steps for each of these projects:

1. Open the project.
2. Go to the Manage tab and open the Data intelligence settings.
3. Go to the Natural language queries section and disable natural language queries by clicking Disable.
4. In the confirmation window, click Disable to stop vectorization of metadata in this project.
5. In the Natural language queries section, re-enable natural language queries by clicking Enable. You might need to refresh the page before you can do that.
6. In the confirmation window, click Enable to start a new onboarding job and resume vectorization with the new model.
7. Wait for the onboarding job to complete successfully before you proceed to rerun any metadata enrichment or unstructured data curation flows. To monitor the job:
   1. Go to the Jobs tab in the project and look for the new onboarding job.
   2. To view the job details, click the job name.
   3. Wait for the job run to show the status Completed.

Updating assets that stem from unstructured data processing

Reprocess all entity tables that were created through unstructured data curation or Unstructured Data Integration flows as derivative data assets to trigger computation of embeddings with the new model. Complete the following steps for each project thats contain entity assets:

1. Open the project and go to the Assets page.

2. To identify the derivative data assets to update, you have several options:

   • Depending on the overall content of your project and the naming of derivative assets, you might be able to identify such assets in the data asset list. To view the list, select Data > Data assets.

   • Identify derivative data assets from document sets. Go to Data > Document sets, open each document set, and check the list of derivatives in the Output section.

3. Create a metadata enrichment asset with the profiling, data search, and metadata expansion options.

   Submit a POST request for /v2/metadata_enrichment/metadata_enrichment_area?project_id=$PROJECT_ID&enrichImmediate=false with the following request body:

   {
     "name": "$NAME",
     "job": {
         "name": "$JOB_NAME"
       },
     "objective": {
       "enrichment_options": {
           "structured": {
          "semantic_expansion": true,
          "analyze_quality": false,
          "assign_terms": false,
          "data_search": true,
          "profile": true,
          "analyze_relationships": false
           }
         },
       "governance_scope": [
           {
             "type": "category",
             "id": "$ANY_CATEGORY_ID"
           }
         ],
       "sampling": {
           "structured": {
             "method": "top",
             "analysis_method": "fixed",
             "sample_size": {
               "name": "BASIC",
               "options": {
                 "row_number": 1000,
                 "classify_value_number": 100
               }
             }
           }
         },
       "datascope_of_reruns": "all"
     }
   

   The category ID must be a valid UUID.

4. After the metadata enrichment asset is created, open it.

5. Edit the metadata enrichment:

   1. Click Edit enrichment, click the Select scope step and add all assets that you identified as derivative data assets.
   2. Click the Review step, the click Save.

6. Click Start full enrichment and wait for the job to finish.

Learn more

• IBM Knowledge Catalog API: Create a metadata enrichment area asset