Column Remover

The Column Remover processor removes columns and passes the remaining columns downstream.

When you configure the Column Remover processor, you specify whether to keep or exclude the selected columns. You define the columns to act on by listing or selecting individual columns, or by specifying the strings, prefixes, or suffixes to match with column names. For more advanced column name patterns, you can use regular expressions to define groups of columns.

Example

Say you want to remove a set of columns that end with _prelim. To do this, you can configure the processor as follows:

Action: Remove Listed Columns
Column Match Criteria: Column Name Suffix
Suffix: _prelim

If some of the columns that you want to remove end with _preliminary, you might add _preliminary to the Suffix property.

Or, you could alternatively configure the processor to remove all columns that include _prelim anywhere in the column name, as follows:

Action: Remove Listed Columns
Column Match Criteria: Column Name Contains Text
String: _prelim

Configuring a Column Remover

Configure a Column Remover processor to remove columns from your data.

On the General tab, configure the following properties:

General Property	Description
Name	Stage name.
Description	Optional description.
Cache Data	Caches processed data.

Configure the following properties on the Remove tab:

Remove Property	Description
Action	Method for removing columns: Remove Listed Columns Keep Listed Columns
Column Match Criteria	Criteria to determine the columns to act on: Column Name - Acts on one or more specified columns. Column Name Contains Text - Acts on columns with names that include one of the specified strings. Column Name Prefix - Acts on columns with names that start with one of the specified prefixes. Column Name Suffix - Acts on columns with names that end with one of the specified suffixes. Column Name Matches Regex - Acts on columns with names that match the results of one of the specified regular expressions. Important: To prevent removing inappropriate columns, when using any option besides Column Name, ensure that the specified details match only the columns that you want to act on.
Columns	Names of the columns to use. Available when matching based on column name.
Strings	Strings that column names must include. The processor acts on columns that match at least one of the specified strings. Available when matching based on text.
Prefixes	Prefixes, or set of characters, that column names must start with. The processor acts upon columns that include one of the specified prefixes. Available when matching based on prefixes.
Suffixes	Suffixes, or set of characters, that column names must end with. The processor acts upon columns that include one of the specified suffixes. Available when matching based on suffixes.
Regular Expressions	Regular expressions that column names must match. The processor acts upon columns that match the results of at least one of the specified expressions. Available when matching based on regular expressions.