Example: Using Visual Inspector with PowerAI Vision

In this fictional scenario, you want to use a PowerAI Vision deep learning model with Visual Inspector to determine whether an assembly has been performed correctly.

Before you begin

Before using Visual Inspector with PowerAI Vision, you must have the following:

A trained model with Core ML enabled. For instructions to train a model, see Training a model.
A device such as an iPad, iPod, or iPhone at iOS version 13.0 or later with Visual Inspector installed.
To use Visual Inspector on a fixed / mounted device, the device must be in Single app mode and there must be a connection to an MQTT broker or a TCP/IP based socket server.

Set up the PowerAI Vision project

Visual Inspector needs access to a PowerAI Vision project. This project must contain a data set for Visual Inspector to write to and the trained model you will be using.

If necessary, create a new project. To create a new project, log on to PowerAI Vision and click Projects, then click +. For our example, we will name the project "Assembly".
Note: Each data set and trained model can belong to only one project. Therefore, if you already have a project that contains the relevant data set and model, do not create a new project.
If necessary, add the data set and trained model to the project by clicking + in the project details page. Visual Inspector captured images can be uploaded to the existing data set, or you can create a new data set for those results (specified in the inspection that is set up on the app).
For this example, our data set is called "Assembly_data_set" and the trained model is called "Assembly_model".

Configure Visual Inspector

For this example, we will only set up the required values; all others are left with defaults. For more information about configuring the app, see Setting up.

Select I already have an account to turn off demo mode and start configuring the app. Click the Settings icon (gear) in the upper left corner.
Specify the following:
Global settings
Handheld Mode

Disable this setting.

MQTT broker

Specify the MQTT broker to use. The broker must conform to the full MQTT specification.

PowerAI Vision Server

When specifying the PowerAI Vision server, for Base URL, you must include the server API. For example: https://xxx/powerai-vision/api/.
Trigger Config
An external system can invoke the device using a socket based connection. The device accepts a trigger message that is similar to MQTT trigger format, except that it is comma separated. For example:
```
"date","ref","triggerString"
```
Visual Inspector responds with PASS or FAIL over the socket connection, depending on the inspection results.
The default values can be used for all other settings.

Create an inspection

For more details, see Creating inspections

Specify the following required values. Everything else can be left with the default settings:
Project

This is the PowerAI Vision project to use. Ours is called "Assembly".

Model

The model to use for inspections. It must be part of the project. Our model is called "Assembly_model".

Data Set

The data set within the project to upload images to. Our data set is called "Assembly_data_set".
Optionally create more inspections. For example, for various sites.

Capture photos for labeling and deep learning

Once mounted and activated, the device will capture images automatically and send them for deep learning.

In Visual Inspector, select the inspection you are using, then click Capture and choose Inspect > Auto Capture.
Images are stored in the data set specified in the inspection. Inspection results can be reviewed on the dashboard.

View results on the dashboard

Click Dashboard on the main screen. The summary information is on the main page. Select any statistic for full details. Select a photo to see whether it passed or failed the inspection.