TensorRT model conversion

You can enable or disable NVIDIA TensorRT model conversion and configure the precision by which different model types are converted.

Configure TensorRT model conversion

To configure the TensorRT model conversion, edit the vision-edge-properties file in the following directory location:

<installation root directory>/volume/run/var/config/vision-edge-properties

From this directory location, you can complete the following conversion configurations:

Enable or disable a TensorRT conversion
Configure the precision at which different model types are converted.

Parameters to enable TensorRT model conversion

The following table details the parameters that control whether or not TensorRT model conversion is enabled for the different model types.

Table 1. Parameters to enable TensorRT model conversion
Parameter	`TRUE/FALSE`	Disable TensorRT model conversion
`DLE_DISABLE_TENSORRT`	`TRUE`	All model types. The value of this parameter overrides any model-specific settings.
`DLE_DISABLE_TENSORRT_GOOGLENET`	`TRUE`	GoogleNet models only.
`DLE_DISABLE_TENSORRT_SSD`	`TRUE`	SSD models only.
`DLE_DISABLE_TENSORRT_YOLOV3`	`TRUE`	YOLOv3 models only.
`DLE_DISABLE_TENSORRT_TINY_YOLOV3`	TRUE	Tiny YOLOv3 models only.

TensorRT model conversion precision

Models that are converted to 16-bit reduced precision run more efficiently and take up less GPU memory than models that maintain 32-bit precision. A model converted to 16-bit reduced precision can result in a small loss in accuracy.

The following table details the conversion values TensorRT model conversion.

Note: You must restart Maximo® Visual Inspection Edge for changes to take effect.

TensorRT model conversion values

Table 2. TensorRT model conversion values
Value	Description
`fp16`	Models are converted to 16-bit floating point, reduced precision.
`pf32`	Models maintain 32-bit floating point precision.

Parameters for TensorRT model precision

The following table details the parameters that determine the TensorRT model precision that is used during TensorRT model conversion.

Table 3. Parameters for TensorRT model conversion
Parameter	Model type that is set for TensorRT model precision
`DLE_TENSORRT_PRECISION_GOOGLENET fp16/fp32`	GoogLeNet
`DLE_TENSORRT_PRECISION_SSD fp16/fp32`	SSD
`DLE_TENSORRT_PRECISION_YOLOV3 fp16/fp32`	YOLOv3
`DLE_TENSORRT_PRECISION_TINY_YOLOV3 fp16/fp32`	Tiny YOLOv3