Crawler field mappings

After you crawl a data source, data source fields are automatically mapped to dataset fields, if possible. This tab shows the results of this mapping. You can modify the mapping of individual data source fields, as well as adding or removing data source fields.

After you create a crawler and the crawler makes an initial run, the discovered native fields are listed in the Field mappings tab. The data source field to dataset field mappings are automatically created. You can change the dataset field name and type here, and also delete data sourced fields from the mapping. You can also add new mappings.

Changing a dataset field name, type, or mapped dataset field
To modify an existing mapping, click Edit... You can now modify the field name and type. To change the mapped dataset field, select a new dataset field from the drop-down list.
Deleting a mapping
To delete an existing mapping, click 3 vertical dots and then click Delete data source field.
Adding a mapping
To add a new mapping, click 3 vertical dots and then click Add data source field. Specify the crawl space, field name, and type, and then click Save. Select the mapped dataset field from the drop-down list.

After you have updated the field mappings, click Save and run the crawler to reflect the changes.

Dataset Field Name conventions

Field names must follow restrictions based on the Apache Lucerne field name conventions. These restrictions are described here.

When the native field name is id
When the native field name is id, the dataset field name id_ is generated. This rule is case-insensitive.
When native field names are non-alphanumeric
Non-alphanumeric native field names are not allowed and automatically converted by the following rules applied in sequence..
  1. URL encode native field names.
  2. Replace % by _.
When native field names are longer than 64 characters
When native field names are longer than 64 characters, field names are truncated to 64 characters. This applies to converted non-alphanumeric names.
Native fields generated by crawlers
Some crawler types generate native fields with the prefix __$ and the suffix $__. In these cases, the dataset field names are generated by removing the prefix and suffix of the native fields. For example, __$Title$__ is changed to Title.
Document date field
All crawlers specially handle the document date values. These values are stored as a data set field named Date.
Document body field
All crawlers specially handle the document body values. These values are stored as a data set field named Body.