Creating custom NLS maps in DataStage

If the maps supplied with DataStage® do not meet your needs, you can create new ones to use in your DataStage flows.

You are most likely to want to produce a variant of an existing map rather than add an entirely new one. Existing maps cannot be overwritten, so any maps you create must have a unique name. Keep in mind that map names are case-insensitive, and underscores, dashes, and spaces are ignored, so the map name "cso_iso_latin_1" is taken to be identical to "CSOISOLATIN1".

Source files for all the ASCL_ maps are provided with DataStage. You can copy these files and base new ones on them. Do not edit the original ASCL_ files because doing so might affect the original mappings.

To create a custom map, complete the following steps:

Open a shell on the PX runtime container on the DataStage cluster.
1. Pull up a list of all pods in the current namespace:
```
kubectl get pods -n ds|grep px-runtime
```
  Example output:
```
ds-px-default-ibm-datastage-px-runtime-7d76896768-tcvkw
```
2. Open a shell in the appropriate container. For example:
```
kubectl exec -it ds-px-default-ibm-datastage-px-runtime-7d76896768-tcvkw -n ds bash
```
Produce a new map source file.
Map source files end with .ucm. They are located in the directory $APT_ORCHHOME/nls/charmaps. Map source files must be built from this location.
The following example shows how to create a new map called MY_ASCII. The map is based on the ASCL_ASCII map, except the input character 0x23 is mapped to the UK pound sign (£) instead of the number symbol (#).
1. In the $APT_ORCHHOME/nls/charmaps directory, copy the file ASCL_ASCII.ucm to MY_ASCII.ucm.
2. Edit the MY_ASCII.ucm file by running nano MY_ASCII.ucm. The format is fairly self-explanatory. The header information identifies the character set. The map itself is described between "CHARMAP" and "END CHARMAP". The string <UNNNN> gives the Unicode character in hexadecimal. The string \xNN gives the map character in hexadecimal.
3. Change copied map by replacing the <U0023> \x23 |0 line to <U0023> \xA3 |0.
4. Write the file.
Build a new map.
The following example shows a map that is built in the $APT_ORCHHOME/nls/charmaps directory.
1. Change directory to $APT_ORCHHOME/etc:
```
cd $APT_ORCHHOME/etc
```
2. Build the new map:
```
./addCustomMaps.sh MY_ASCII.ucm
```
Make the map visible to DataStage flows.
1. Check that the compiled charmap is copied to the expected directory. For example, check for the charmap MY_ASCII.cnv in the directory icudt501.
  First, change the directory to $APT_ORCHHOME/nls/charmaps:
```
cd $APT_ORCHHOME/nls/charmaps
```
  Then, list the contents:
```
ls -l icudt50l/MY_ASCII.cnv
```
2. Copy the icudt501 directory to your persistent volume (PV), for example, /ds-storage/charmaps:
```
cp -r icudt50l /ds-storage/charmaps/
```
  You need to set ICU_DATA directory to ICU_DATA=/ds-storage/charmaps in any job you run, if you want to use the new character map.

The map is visible and ready to use.