On a recent Information Insights
engagement we encountered a requirement to mask Canadian Social Insurance Numbers (SIN). For those unfamiliar with it, Canadian SIN numbers act as Canada’s National ID. They can be thought of as similar to the United States’ Social Security Number (SSN), and Brazil’s Cadastro de Pessoas Físicas (CDF). United States customers are lucky; they have a Social Security Number transformation function that comes with Optim. These customers can generate random but valid social security numbers by executing TRANS SSN inside their column maps.
For Canadian SIN numbers, until recently, we gave the standard answer we give for any data element involving an unimplemented data privacy function. That is, we suggested doing one of the following depending on your requirements, the sensitivity of the data, and the customer environment:
1) Understand your application’s check sums, and try and incorporate RAND or SEQ functions instead. This may be enough if your application is relatively lenient in its checks
2) Generate a custom lookup table with valid data and use a lookup function instead of a transformation function
3) Depending on the Nature of the Data, using a shuffle instead might be a viable alternative. I would be hesitant about using this in the context of a Canadian SIN
4) Build a custom column map procedure to build your own data privacy function in code. This can be done either in Optim BASIC using Optim Column Map Procedures or in C/C++ using Optim Exits
I said that this was our answer for Canadian SINs until recently because we found an undocumented data privacy function inside Optim that generates social insurance numbers. You can access it by placing EXIT PX0SIN in your column map:
This will generate valid but fictional Canadian SIN Numbers to replace the real ones. It behaves very similarly to TRANS SSN. You should take care to note though that this is undocumented. It was probably left out of the documentation for a reason. I would not expect it to have been put through the same kind of rigorous testing that the other documented data privacy functions have been put through. As such you should fill in this gap by being more rigorous in the testing of the function in your specific customer environments.
What about if you are not this lucky? What if you need a transformation function, but none is provided by Optim? I will spend the rest of this post explaining what the EXIT call shown above means and how you can build your own exits.
Optim Exits written in C or C++ effectively provide the same functionality as Optim Column Maps which are written in BASIC and coded within the Optim environment. Optim Exits have a couple of main advantages which make them my personal preference. The first is that you can code them in any development environment instead of being limited to Optim’s column map procedure IDE. The second is that they are more extensible and flexible than column map procedures because it is very easy to link to the numerous external C/C++ libraries and frameworks. It’s possible but more awkward to do this in Optim Column Map Procedures written in BASIC within the Optim Environement.
Let’s take a look at how Exits work. The first step is to set up your environment. You will need a C/C++ compiler. I recommend downloading something like Visual C++ and using the Microsoft compiler that comes with it. The reason is because the Optim team seems to use the Microsoft Compilers to build Optim. GNU based compilers like MinGW also work, but I find I get strange conflicts when calling certain external libraries inside my Opim Exit. Once you have your compiler, you will need the header files located in <OPTIM_HOME>\RT\SAMPLES\CMExit. Now, let’s examine one of the simplest exits you can write below:
// Replaces a value in Optim. Simple convenience function for the pPSTPutColValue callback
void replaceValueInOptim(PST_STRUCT_CM_EXIT_PARM * pInputParms, void * value, int type, short precision, short scale, long size)
int rc = 0;
rc = (*pInputParms->pPSTPutColValue)
// The function that Optim calls: Where the magic happens__declspec
PSTColMapExit ( PST_STRUCT_CM_EXIT_PARM * pInputParms, PST_STRUCT_CM_EXIT_COL_LIST * pSrcColList, PST_STRUCT_CM_EXIT_COL_LIST * pDstColList )
char hello = "Hello World!";
replaceValueInOptim(pInputParms, &hello, PST_C_CHAR_SZ, 0, 0, sizeof(hello));
return( PST_CM_EXIT_SUCCESS );
This exit writes “Hello World!” repeatedly in the column it is referenced in. The #include lines import the header files you need to interact with Optim. The PSTColMapExit function is the function that the Optim Convert process looks for to call each time it needs to mask a value. In order for Optim to call the function, it needs to be visible to Optim. You can do this by including the __declspec(dllexport) indicator so that it becomes a DLL export. The replaceValueInOptim function is a convenience function to make it easier to call pInputParms->pPSTPutColValue. pPSTPutColValue is a callback used to replace values inside the column. Note that the function is incomplete as you will likely want to take different actions based on what the return code (rc) is after the callback. For instance you could do some error handling if Optim was unable to replace the value in the column or if the value was truncated.
That’s all there is to it. You can use this code as a skeleton to build your own exits. Just remember to compile as a DLL file including the headers and drop the resulting DLL file into <optim_home>\RT\BIN. Once you have the DLL in that directory you can call it very easily by reference EXIT <FILENAME> in your column maps where <FILENAME> is the name of the DLL without the DLL extension. For example if you create TEST.DLL you would enter EXIT TEST in your column map.
The header files are a great reference for extended/advanced functionality as well as the sample exit provided by the Optim team inside the Optim installation location and/or installation media. A lot of people are hesitant to implement C/C++ based Optim Exits. I see no reason why as long as you are careful about what you are doing and keep track of memory allocation. Happy Coding!