Uploading data files and previewing the data

On the home page, you can upload files for variable analysis and attribute analysis. Variable analysis and attribute analysis each require an event data file and an IBM® Quality Early Warning System parameter file.

About this task

When a file is uploaded, the system identifies the file type. The file types include; Variable Event file, IoT Variable Event file, Variable Parameter file, Attribute Event file, IoT Attribute Event file, and Attribute Parameter file. The IoT Variable Event file and the IoT Attribute Event file contain data that is collected from the Watson IoT Platform. The data is generated by the system automatically. When you upload a file, it is parsed to a file type according to the contents of the file.

Variable event files and attribute event files are used to identify variables and attributes. When they are identified, variable types and attribute types appear in the Object list. (Parameter files cannot be used to identify variables and attributes. Variables and attributes in parameter files must also be in event files to be correctly identified.)

The variable event file can contain dimensional data and must also contain variable values and one timestamp column. The timestamp column must use one of the following time formats:
  • dd/MM/yyyy HH:mm:ss
  • yyyy/MM/dd H:mm:ss
  • yyyy/M/d H:mm:ss
  • d/M/yyyy H:mm:ss
  • dd.MM.yyyy HH:mm:ss
  • MM-dd-yyyy HH:mm:ss tt, where tt = AM or PM
  • M/d/yyyy H:mm:ss tt, where tt = AM or PM
  • yyyy-MM-dd HH:mm:ss
The attribute event file can contain dimensional data and must also contain ATTRIBUTE_NAME, DATE or TIMESTAMP, PRODUCED_QTY, TESTED_QTY, and FAILED_QTY columns. If both the DATE and TIMESTAMP column exist, the DATE column is used and the TIMESTAMP column is not. The DATE column must use one of the following date formats:
  • dd/MM/yyyy
  • yyyy/MM/dd
  • yyyy/M/d
  • d/M/yyyy(
    Note: M/d/yyyy format is assumed if no day is higher than 12
    )
  • dd.MM.yyyy
  • MM-dd-yyyy
  • M/d/yyyy
  • yyyy-MM-dd
The variable parameter file can contain dimensional data as well as the following required and optional columns (not case-sensitive):
  • QEWS_ACCEPT_LEVEL (required)
  • QEWS_SIGMA (required)
  • QEWS_TARGET (required)
  • QEWS_UNACCEPT_LEVEL (required)
  • QEWS_VARIABLE or QEWS_VARIABLE_NAME (required)
  • QEWS_FALSE_ALARM_RATE (optional)
  • QEWS_NUMBER_OF_POINTS (optional)
  • QEWS_RUNTIME (optional)
  • QEWS_TYPE_OF_CONTROL (optional)
The attribute parameter file can contain dimensional data as well as the following required and optional columns (not case sensitive):
  • QEWS_ATTRIBUTE_NAME (required)
  • QEWS_LAM0 (required)
  • QEWS_LAM1 (required)
  • QEWS_NUMBER_OF_VINTAGES (optional)
  • QEWS_PROB (optional)
  • QEWS_RUNDATE (optional)
Important:
  • File names must not contain double pound symbols (##), less-than-greater-than symbols (<>), or square brackets ([]).
  • Dimension column names must not contain special characters or spaces.
  • Values in dimension columns must not contain colons (:), semicolons (;), or commas (,).
  • Values in the ATTRIBUTE_NAME column must not begin or end with a pound symbol (#) and must not contain double pound symbols (##) or semicolons (;).
  • The QEWS_RUNTIME column of parametric parameter must use the yyyy-MM-dd HH:mm:ss time format.
  • The QEWS_RUNDATE column of attribute parameter must use the yyyy-MM-dd date format.
  • The column headings ORG_ID, DEVICE_TYPE, DEVICE_ID, and EVENT_TYPE are used by IoT event files. These columns are always identified as dimensions.
  • Files must not contain duplicate column names.
  • In each file, date and time formats must be consistent, otherwise the file parser fails.
  • During file upload, the file parser identifies file types by using the following rules. If the file contains a QEWS_ATTRIBUTE_NAME column, then it is an attribute parameter file. If the file contains a QEWS_VARIABLE_NAME or QEWS_VARIABLE column, then it is a variable parameter file. If the file contains an ATTRIBUTE_NAME column, then it is an attribute event file. Otherwise, it is a variable event file.

Procedure

  1. Select My Data > Files > Browse for file, select the file, and select Open.
  2. In the Progress pane, view the status of the upload.
  3. In the Uploaded Files section, view the files.
    A file cannot be deleted while it is being parsed by the system. When the parsing is done, hover over the tags in each file card to see the object type, dimensions, and other information. (If the parsing fails, then an error message appears. Delete the file, correct the problem in the file, and upload again.)
  4. To preview the data in the file, click Preview. The first 10 rows of the data are shown.