Creating and working with data sets

Before you can work with videos or images, you need to create a data set. A data set is a group of images, videos, or both that you will use to train a deployable model.

1. Log in to PowerAI Vision.
2. Click Data Sets in the navigation bar to open the Data Sets page. There are several ways to create a new data set:
   • To create an empty data set, click Create new data set.
   • If you have a previously exported data set, click Import .zip file.
   • If you want to copy an existing data set, select the data set and click Duplicate.
   File considerations:

   • Videos
     • You can play only the following video types in the PowerAI Vision video player:
       • Ogg Vorbis (.ogg)
       • VP8 or VP9 (.webm)
       • H.264 encoded videos with MP4 format (.mp4)
     • Before importing videos for use with action detection models, it is recommended that you prepare them as follows:
       • Cut out long periods of background video without any actions.
       • Transcode videos with FPS greater than 30 down to 30 FPS
       • Crop the video so that actions should take up a large part of the frame.
   • Images
     • PowerAI Vision has limited support for Pascal VOC annotations. Annotations for multiple files residing in a common XML file are not supported. In other words, each annotation XML file can only contain annotations for a single image, identified by the filename attribute.

       If you have a single XML annotation file containing annotations for multiple images in the data set to be imported, the annotations need to be split out into separate XML files before PowerAI Vision can import the annotations successfully.

     • PowerAI Vision supports importing COCO data sets with the following limitations:
       Only "object detection" annotations are supported. You can review the annotation format on the COCO data format page. When you import images with COCO annotations, PowerAI Vision only keeps the information it will use, as follows:

       • PowerAI Vision extracts the information from the images, categories, and annotations lists and ignores everything else.
       • Unused annotations are not saved. For example, if there is annotation information for clock, but no image is tagged with a clock, then the clock object (called category in COCO) is not saved.
       • For COCO annotations that use the RLE format, the RLE is not saved. Only the bounding box is used.

       Note: Images without tags are saved.
3. Click the data set you just created to open it. Add images and videos by using Import file or by dragging them to the + area. If you do not follow these considerations, your upload will fail and a message will be shown on the screen. For details about why the upload failed, click the bell icon at the top of the page to open the Notifications center.
   Upload considerations:

   • You can select multiple image or video files, or a single .zip file that contains images and videos, but you cannot upload a folder that contains images or videos.
   • If you import a .zip file into an existing data set, the .zip file cannot contain a directory structure.
   • You cannot navigate away from the PowerAI Vision page or refresh until the upload completes. You can navigate to different pages within PowerAI Vision during the upload.
   • There is a 24 GB size limit per upload session. This limit applies to a single .zip file or a set of files. You can, however upload 24 GB of files, then upload more after the original upload completes.