Create an image dataset for object detection

Create a dataset from images for object detection. Depending on the storage format specified, this dataset can be used for Caffe or TensorFlow models.

About this task

Before creating an LMDB dataset for the purposes of object detection, make sure that your training data resides on the shared file system. The training data must be in one folder which contains two sub folders, one for .jpg images named JPEGImages and one for annotations named Annotations.

Each image must have a corresponding annotation of the same name, for example: 01_01.jpg resides in the /training-images/JPEGImages folder and 01_01.xml resides in the /training-images/Annotations folder.

Example annotation contents for 01_01.xml:
<annotation>
    <folder>JPEGImages</folder>
    <filename>./JPEGImages/01_01.jpg</filename>
    <path>/JPEGImages/</path>
    <source>
        <database>Unknown</database>
    </source>
    <size>
        <width>2000</width>
        <height>2000</height>
        <depth>3</depth>
    </size>
    <segmented>0</segmented>
    <object>
        <name>mitoses</name>
        <pose>Unspecified</pose>
        <truncated>0</truncated>
        <difficult>0</difficult>
        <bndbox>
            <xmin>935</xmin>
            <ymin>880</ymin>
            <xmax>1015</xmax>
            <ymax>960</ymax>
        </bndbox>
    </object>
</annotation>

There are open source tools that can help with creating this relationship, for example, LabelImg.

Procedure

  1. From the cluster management console, select Workload > Spark > Deep Learning.
  2. Select the Datasets tab.
  3. Click New.
  4. Create a dataset from Images for Object Detection.
  5. Provide a dataset name.
  6. Specify a Spark instance group.
  7. Provide a training folder. The full absolute path to the training folder must be provided. The training folder must contain two folders, one for .jpg images named JPEGImages and one for annotations named Annotations.
  8. Provide the percentage of training images for validation. A percentage of images are used for validation from the training folder.
  9. Provide the percentage of training images for testing. A percentage of images are used for testing from the training folder.
  10. Specify a split algorithm.
  11. To double the number of images in the dataset by creating a resided copy of each existing image, enable the option. If enabled specify the following options.
    • Specify the resized image height.
    • Specify the resized image width.
  12. Click Create.

Results

The dataset is created once it is in Created state. If creation failed, see the driver and executor logs in the Spark Applications tab.

What to do next

To view details about the dataset, click the dataset name. To use the dataset in a training run, either create a training model or start a training run.