Converting information
You use the CONVERT action to convert data according to a lookup table or a literal you supply.
You can perform the following actions:
- CONVERT
- Changes tokens.
- CONVERT_S
- Concatenates a suffix onto specified tokens.
- CONVERT_P
- Concatenates the first prefix in a token that matches a value in a lookup table onto specified tokens.
- CONVERT_PL
- Concatenates the longest prefix in a token that matches a value in a lookup table onto specified tokens.
- CONVERT_R
- Runs tokens through the tokenization process again when you implement changes to the tokens.
You can use the following actions to convert
characters from languages that are not Latin-based:
- TRANS_KH
- Converts Katakana characters to Hiragana characters.
- TRANS_HK
- Converts Hiragana characters to Katakana characters.
- TRANS_WN
- Converts full-width characters to half-width characters.
- TRANS_NW
- Converts half-width characters to full-width characters.