Extract error prevention
Data that fails to meet the criteria and specified format cannot be processed. Any large volume of data is likely to have unforeseen characteristics and should be reviewed for accuracy before implementation.
Data verification checklist
Be sure to perform the following series of checks to ensure that the quality, size, and format of the data are what is expected and agreed upon.| Step | Description | Pass (Yes/No) |
|---|---|---|
| Readable | Are the files readable? If they are zipped, can they be extracted and read in a plain text viewer? | |
| File count | What is the record count of each file? | |
| Primary ID | Is the primary identifier unique throughout the source file? | |
| Date Format | Is the date format for all date fields valid? | |
| Name Format | Is the name format divided into first, middle, and last? | |
| Non-printable characters | Does the file contain printable characters only? In other words, it does not include any non-printable characters. | |
| End of line terminator | Does the end of line terminator use the correct CRLF? | |
| Format | Does the file adhere to the correct format? Is the correct delimiter used (|)? | |
| Source | Does the file contain a column that identifies each record as pertaining to a particular source? | |
| Null value | Are there any extra characters or fields that we did not agree to in the format? | |
| Extra characters | Are there any extra characters or fields that do not match the specified file format? |