Lesson 3: Configuring import modes

Using data movement functionality, you can import tables to Hadoop in various ways. When importing, you can replace target tables, append to a target table or even work in incremental way, so that only the records inserted to source NPS table after the last import are transferred. The import behavior is defined with the nz.fq.append.mode property in the configuration XML file.

About this task

The nz.fq.append.mode property specifies how data is transferred to the specified HDFS location. You can select one of the following values:

create This value causes the import to create a new table. Import fails if a specified table already exists on Hadoop.
append This value causes the import to append the data to the existing table, or to create a table if it is not present.
overwrite This value causes the import to create a new table that replaces a file that is present at the specified path.
incremental In this mode, any data that was added after the last data movement is appended to the file at the specified path.
Important: If any deletions or updates were made in the table on NPS after the last import, these changes cannot be propagated to Hadoop with incremental import. To prevent data inconsistency, the import operation stops and the following error message is displayed:
```
Error message: Some records have been deleted or updated after last import. Full import 
is required to have consistent data. To run incremental import anyway please set 
nz.fq.ignore.deleted to true.
```
You can use the nz.fq.ignore.deleted flag to ignore such error messages and import data anyway.

Default value for the nz.fq.append.mode property is create.