Implementing SPL Native Functions with complex tuple types
GabrielaJS 270004FR6S Visits (1443)
Native functions are a convenient way to expose a function implemented in a native language to code written directly in SPL. Such code can be used in a Custom operator, SPL functions, or primitive operators that allow custom logic. In InfoSphere Streams 3.0.0, native functions must be implemented or wrapped in C++ (e.g., using a library with C/C++ bindings). The C++ implementation is then compiled into a shared library, which is linked to an SPL program at compile time.
When writing native functions, one might want to use a complex SPL tuple type as a return value. In its current version, the SPL compiler does not allow a native function to return a tuple type. This is because native functions are compiled into a shared library, separately from the SPL program. When compiling a shared library, developers do not have access to specific SPL tuple types, which are only generated when compiling an SPL program. Even if one forces the compiler to generate an SPL tuple type before compiling the shared library, the SPL compiler does not guarantee that the generated tuple types will not change or move. As a result, shared libraries can only access tuples via their base class SPL::Tuple, which is not a concrete type. This means that one cannot use an SPL::Tuple when concrete types are needed in C++, such as in the return of a method or in STL containers.
A trick to overcome this limitation is to pass a mutable SPL::Tuple reference as a native function parameter and then use the reflective API to access tuple attributes. A mutable parameter in SPL is equivalent to a non-const reference parameter in the C++ API.
The following code shows an example of a Custom operator that uses a native function named populateTuple (line 19) to fill up a tuple of type tuple<int32 anInt, list<int32> anIntList> (line 14).
The following segment shows the function prototype definition in the function model for the shared library.
Note that the function signature indicates a mutable tuple of a specific type. This indicates to the SPL compiler that this function can only be invoked with tuples of the specified type.
On the C++ side, the method implementation must receive the generic SPL::Tuple type. A sample C++ implementation of the populateTuple method is shown below.
The SPL reflective APIs can also help when one needs to use native functions to populate an SPL collection that contains tuples. The example below shows an example SPL invocation of a native function that receives a map as a parameter (line 21).
The segment below shows the function prototype declaration in the function model.
In C++, the method declaration has the SPL::Map generic type, which can be inspected via the reflection API.
Always keep in mind that using the reflective API to access SPL types has a greater runtime cost than using generated types. However, unlike generated types, the reflective API is always available.
[Update: On 02/15/2013, we improved the example on passing collections to a native function. Thanks to Bugra Gedik!]