After the initial training of the model, you can review the results for accuracy and
update the model as needed.
About this task
To create the model, you specified field name location, field value location and format,
and other characteristics of the field values. Training the model incorporates these parameter and
characteristics. When the training is complete, you can examine the results to see whether the model
has accurately captured the fields and values that you created for each document type. If the model
has issues, you can update the model with accurate values.Before Automation Document Processing
can compute the accuracy of the training results, it must first know what the correct values for the
extracted fields are. These correct values are known as the ground truth values. When
you initially enter the Review the trained model screens, all of the training
info is in a Verify results section, and no accuracy metrics are available.
At this point, the sample documents are in the Needs verification state. You
must review each training sample of each document type and correct any missing or incorrect values
to establish the ground truth for that sample. After you enter the correct values for all of the
fields, you can use the Submit button to move the sample document to the
Verified state.
After all sample documents are in the
Verified state, Automation Document Processing can calculate and display
accuracy statistics for each document type, and for the overall model.
Note: It is not possible
to enter ground truth values for tables. Table fields are not considered when computing accuracy
statistics.
Procedure
To refine the trained model:
- From the main page in the Designer, on Extraction model, click
Open.
- Click Review the trained model.
- On the Last training results tab, click a document type to see the
results for samples of that type.
- For each sample that needs additional verification, in the Action
column, click Start.
- In the verify view, check that the extracted values for the fields match the values that
you find in the displayed sample document.
- For any field value that needs to be corrected, click the edit icon, add the correct
value.
Type in text updates, or select Boolean settings from the drop down choice list,
and click Save.
- After you verify, edit, and save any changes for every field, click
Submit to finalize the test sample.
- Repeat the verification for the remaining samples in the list.
After you
complete the verification for all the samples in a type, the document type is listed in the
Results ready table. The model includes the updated accuracy and confidence
for the document type.
- Use the Results ready information to further refine the
model.
Documents that are listed in the
Results ready have already
been verified.
In the
Results ready table, click a document type to see
more information about the samples and fields for that type.
- Click Sample results, then click View results to
see the results for a particular sample document and how the sample compares to the model.
- Click Field results, then click View results to
see the value and confidence for the field from each of the samples in the document type. You can
filter to show only samples with issues. You can also filter to group fields by confidence
levels.
What to do next
After you make any adjustments to your document types and fields, or add or remove any
sample documents, you must retrain the field extraction model. After a model is retrained, the
ground truth values are reset. You must review the values for each sample again, taking care to add
values for any new fields, and any new sample documents. If there are no new sample documents or
fields, and you have already entered values for all fields prior to retraining, then you can use the
Verify all button to mark all samples for a document type as verified
again.