Trouble indexing and summarizing certain documents

The Content Assistant does not perform vector indexing and document summarization for certain documents in an object store. These documents remain unprocessed even though other documents in the same object store are indexed and summarized successfully.

Symptoms

You installed the Content Assistant add-on for an object store. You created document class subscriptions for vector indexing and watsonx summary generation. You notice the following symptoms:

Some documents are successfully vector indexed and watsonx summaries are also generated for those documents.
Vector indexing and watsonx summary generation fail for certain other documents in the object store.

Causes

The issue can occur because of one of the following reasons:

Table 1. Problem cause and description
Problem cause	Problem description
Insufficient queue sweep permissions	When you run a queue sweep for vector indexing and document summarization, the sweep assumes the identity of the object store administrator who installed the Content Assistant add-on for the object store. However, this object store administrator may not be part of user groups and roles that include `READ` and `VIEW_CONTENT` access to documents which the sweep processes. Note: The problem occurs with Content Assistant deployments that use Content Platform Engine V5.7 and earlier only.
Maximum indexing limit for subscription	Your Content Assistant subscription has reached the maximum index size limitation. In this case, vector indexing fails for new documents that are added to the object store.
Maximum summarization limit for subscription	Your Content Assistant subscription has reached the maximum number of document summaries. In this case, watsonx document summary generation fails for new documents that are added to the object store.
Missing text extraction configuration for parent class	Persistent text extraction is not configured for the parent document class for the failing documents. This causes both vector indexing and document summarization to fail for the documents that belong to the parent class.
Missing vector indexing configuration for parent class	A vector indexing subscription is not configured for the parent document class for the failing documents.
Unsupported document content type	Some documents can include only image-based content, or may be PDF files that contain only images. Text cannot be extracted from such documents and hence vector indexing and watsonx summary generation fails for those documents. For more information, see the topic Known issues and limitations.

Resolving the problem

The following table presents the resolution steps that apply to your environment based on the problem cause:

Table 2. Resolution steps
Problem cause	Resolution steps
Insufficient queue sweep permissions	Set the following environmental variable to resolve the issue with sweep permissions: `-Dcom.filenet.engine.sweep.RelaxedQueueSecurity={76e57717-2d30-4198-8396-f88ec21e1daa},{8a920c77-ca2b-407f-9610-3a5ee8fd54a4}` Note: The suggested resolution is applicable for Content Assistant deployments that use Content Platform Engine V5.7 and earlier only.
Maximum indexing limit for subscription	Contact your IBM Sales representative to purchase an Indexed Data Pack add-on for your subscription.
Maximum summarization limit for subscription	Contact your IBM Sales representative to purchase a Document Summaries add-on for your subscription.
Missing text extraction configuration for parent class	Configure persistent text extraction for the parent document class. For more information, see the topic Configuring text extraction for document classes.
Missing vector indexing configuration for parent class	Create a vector indexing subscription for the parent document class. For more information, see the topic Creating a document class subscription for vector indexing.