Overview
This article describes how to execute MATLAB functions from within InfoSphere Streams applications. MATLAB is a scientific computing language and platform. The strong support for matrix manipulation and large collection of mathematical modeling libraries make MATLAB a popular implementation choice for various analytic assets. Reimplementing these analytic assets in other languages is often cumbersome due to the need for finding external libraries that will provide the functionality already built into MATLAB. This creates a strong motivation to integrate functions written in MATLAB into InfoSphere Streams applications without manually rewriting the function logic in C++ or Java™, which are the first class languages supported by Streams.
MATLAB supports compiling functions written in MATLAB programming language (in .m files) into C++ shared libraries. It also provides APIs to write C++ code that interacts with the generated library, such as utility classes that provide matrix data types. In addition, MATLAB also provides a runtime library to link against, for applications that make use of MATLAB routines compiled into C++. The overall approach of integrating MATLAB functions into Streams is as follows:
 Develop your MATLAB function(s).
 Use MATLAB compiler to create a C++ shared library.
 Create a native function in SPL that wraps the C++ shared library.
 Write an SPL application that uses the native functions.
These steps are illustrated with the following sample.
Develop your MATLAB function
You will write a simple MATLAB function that performs matrix inversion.
Create a directory called Matlab_inv
and put a
file named ml_inv.m
in it, with the following
content:
function out = ml_inv(in) % ML_INV Matrix inverse % Invert a given matrix out = inv(in);
Your goal is to be able to easily use this matrix inverse function in SPL
code. The following is a sample SPL function that utilizes the matrix
inverse function ml::inv
that wraps the
ml_inv
function from MATLAB. In the rest of
this article, you will look at the steps needed to create the
ml::inv
function.
void foo() { // Sample SPL code that uses the matrix inverse list<list<float64>> inM = [[1.0,3.0],[2.0,4.0]]; mutable list<list<float64>> outM = []; ml::inv(outM, inM); // ml::inv is the wrapper for ml_inv }
Use MATLAB compiler to create a C++ shared library
You will use the MATLAB compiler to create a C++ shared library that will contain the matrix inversion function.
 In the MATLAB command line, type
deploytool
.  Follow the graphical interface to create a deployment project called
libMatlab_inv
under theMatlab_inv
directory.  Click the Add File button to add the
ml_inv.m
file into the project.  Click the Build button to create the shared library.
The following result files will be generated under the
Matlab_inv/libMatlab_inv/distrib
directory.
libMatlab_inv.h
: This is the interface file that declares the C++ functions generated from the MATLAB code.libMatlab_inv.so
: This is the shared library that contains the C++ function implementations generated from the MATLAB code.libMatlab_inv.exports
: This is a text file that lists the name of the functions exported by the library. This file is not required for the operation of the library.libMatlab_inv.ctf
andlibMatlab_inv_mcr
: The former is an archive file, from which the latter directory is created by the MATLAB compiler. These contain additional support libraries. They should be located at the same place as the.so
shared library.
Try out the shared library in a standalone C++ application
Before you use the shared library in a Streams application, you will try it out on a standalone C++ application.
Create a directory named sample_c++
at the same
level as Matlab_inv
. Inside this directory,
create a file named sample.cpp
and populate it
as follows:
#include "libMatlab_inv.h" int main() { libMatlab_invInitialize(); double data[] = {1.0, 2.0, 3.0, 4.0}; mwArray in(2,2,mxDOUBLE_CLASS); mwArray out(2,2,mxDOUBLE_CLASS); in.SetData(data, 4); ml_inv(1, out, in); std::cerr << "[" << out(1,1) << ", " << out(1,2) << "; " << out(2,1) << ", " << out(2,2) << "]\n"; libMatlab_invTerminate(); return 0; }
In the above code, you call the
libMatlab_invInitialize
function to initialize
the MATLAB runtime. This is one of the functions generated for you by the
MATLAB compiler. You then create the following two matrices:
in
and out
. The
mwArray
is a C++ class provided by the MATLAB
C++ APIs for working with matrices. You can use the
ml_inv
function to take the inverse of the
in
matrix and assign it to the
out
matrix. This is the core function generated
for you by the MATLAB compiler, based on the MATLAB function of the same
name from the ml_inv.m
file. Finally, you call
the libMatlab_invTerminate
function to finalize
the MATLAB runtime. Again, this is a function generated for you by the
MATLAB compiler.
To compile this program, create a Makefile
as
follows:
.PHONY: all clean MATLAB_LIBRARY_LOCATION := /nfs/hny/apps01/matlab/bin/glnxa64/ MATLAB_INCLUDE_LOCATION := /nfs/hny/apps01/matlab/extern/include/ all: g++ o sample sample.cpp \ I ../Matlab_inv/libMatlab_inv/distrib \ I $(MATLAB_INCLUDE_LOCATION) \ L ../Matlab_inv/libMatlab_inv/distrib \ Wl,rpath,../Matlab_inv/libMatlab_inv/distrib \ L $(MATLAB_LIBRARY_LOCATION) \ Wl,rpath,/nfs/hny/apps01/matlab/bin/glnxa64/ \ lMatlab_inv lmwmclmcrrt clean: rm sample
The following are a few things that you should notice:
 The include path
I ../Matlab_inv/libMatlab_inv/distrib
is used to specify the location of the interface file (libMatlab_inv.h
) generated by the MATLAB compiler.  The include path
I $(MATLAB_INCLUDE_LOCATION)
is used to specify the location of the interface files for the MATLAB C++ APIs. This location will be specific to your environment and depends on the location of the MATLAB installation. The variable$(MATLAB_INCLUDE_LOCATION)
should be defined accordingly.  The library path
L ../Matlab_inv/libMatlab_inv/distrib
is used to specify the location of the library (libMatlab_inv.so
) generated by the MATLAB compiler. An RPATH is specified using the same path, for runtime location of the library.  The library path
L $(MATLAB_LIBRARY_LOCATION)
is used to specify the location of the MATLAB runtime libraries. This location will be specific to your environment and depends on the location of the MATLAB installation and the architecture of your system. The variable$(MATLAB_LIBRARY_LOCATION)
should be defined accordingly.  The library
lMatlab_inv
is used to specify the name of the library generated by the MATLAB compiler (libMatlab_inv.so
).  The library
lmwmclmcrrt
is used to specify the name of the MATLAB runtime library (libmwmclmcrrt.so
).
Simply type make
to build the library, and
./sample
to run it. It should produce the
following output:
[2, 1.5000; 1,
0.5000]
.
Create a native function in SPL that wraps the C++ shared library
You will create native functions in SPL to make use of the functions generated by the MATLAB compiler. This could be done in one of two ways. You could either create a toolkit that encapsulates all the native functions, or directly include native functions into an application. The former is more appropriate if there will be multiple applications that will make use of the functions. For brevity, you will use the second approach here.
 Create a directory named
sample_spl
at the same level asMatlab_inv
andsample_c++
. This will be your application directory.  Create a subdirectory named
ml
. This will be your namespace directory.  Under
ml
, create a subdirectory namednative.function
, which will hold the function model.  Under this directory, add the function model file named
function.xml
, with the contain the following content.
<functionModel xmlns="http://www.ibm.com/xmlns/prod/streams/spl/function" xmlns:cmn="http://www.ibm.com/xmlns/prod/streams/spl/common" xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/streams/spl/function functionModel.xsd"> <functionSet> <headerFileName>spl_ml_inv.h</headerFileName> <functions> <function> <description>Take inverse of a matrix</description> <prototype><![CDATA[ public void inv(mutable list<list<float64>> r, list<list<float64>> s) ]]></prototype> </function> <function> <description>Initialize Matlab runtime</description> <prototype><![CDATA[ public void initialize() ]]></prototype> </function> <function> <description>Finalize Matlab runtime</description> <prototype><![CDATA[ public void terminate() ]]></prototype> </function> </functions> <dependencies> <library> <cmn:description>Matrix inverse</cmn:description> <cmn:managedLibrary> <cmn:lib>Matlab_inv</cmn:lib> <cmn:lib>mwmclmcrrt</cmn:lib> <cmn:libPath>../../impl/lib</cmn:libPath> <cmn:libPath>/nfs/hny/apps01/matlab/bin/glnxa64/</cmn:libPath> <cmn:includePath>../../impl/include</cmn:includePath> <cmn:includePath>/nfs/hny/apps01/matlab/extern/include/</cmn:includePath> </cmn:managedLibrary> </library> </dependencies> </functionSet> </functionModel>
In the above function model file, there are a few things to note:
 The file
spl_ml_inv.h
is the header file that contains the C++ functions that will wrap the MATLAB generated functions.  The three SPL native functions
inv
,initialize
, andterminate
correspond to the three functionsml_inv
,libMatlab_invInitialize
, andlibMatlab_invTerminate
generated by the MATLAB compiler. Note that theinv
function uses SPL nested lists to represent matrices, as there is no matrix type in SPL. As you will see, theinv
function will also perform the conversion between SPL C++ types and MATLAB C++ types.  The libraries
Matlab_inv
andmwmclmcrrt
are specified as dependencies, including their library paths and include paths. Note that the include and library paths for theMatlab_inv
libraries are specified relative to the model file.
Now you will copy the relevant files into the
impl
subdirectory under the application
directory sample_spl
to make the application
selfcontained.
 Create a directory
impl
under the application directorysample_spl
.  Create two subdirectories under
impl
:include
andlib
. Copy thelibMatlab_inv.h
file fromMatlab_inv/libMatlab_inv/distrib
directory in to theinclude
directory.  Now copy the
libMatlab_inv.so
andlibMatlab_inv.ctf
files as well as thelibMatlab_inv_mcr
directory into thelib
directory.  Finally, create the wrapper
spl_ml_inv.h
header file under theinclude
directory, with the following content.
#include "libMatlab_inv.h" namespace ml { void initialize() { libMatlab_invInitialize(); } void terminate() { libMatlab_invTerminate(); } void inv(SPL::list<SPL::list<SPL::float64> > & lhs, SPL::list<SPL::list<SPL::float64> > const & rhs) { size_t nr = rhs.size(), nc = rhs[0].size(); mwArray in(nr, nc, mxDOUBLE_CLASS); mwArray out(nr, nc, mxDOUBLE_CLASS); for(size_t r=0; r<nr; ++r) for(size_t c=0; c<nc; ++c) in(r+1,c+1) = rhs[r][c]; ml_inv(1, out, in); lhs.resize(nr); for(size_t r=0; r<nr; ++r) { lhs[r].resize(nc); for(size_t c=0; c<nc; ++c) lhs[r][c] = out(r+1,c+1); } } }
In the previous code, the inv
function performs
transformations between the SPL list
types and
MATLAB mwArray
types, in order to wrap the
ml_inv
function generated by the MATLAB
compiler. Note that all the wrapper functions are placed into the
namespace ml
, since your
native.function
directory is under the
namespace directory ml
.
Write an SPL application that uses the native functions
You will write the SPL application that uses the native functions that wrap the functions generated by the MATLAB compiler.
Create a file named Main.spl
under the
sample_spl
directory, with the following
content:
composite Main { graph stream<int8 dummy> Beat = Beacon() { param iterations : 1u; } () as Sink = Custom(Beat) { logic onTuple Beat: { ml::initialize(); list<list<float64>> inM = [[1.0,3.0],[2.0,4.0]]; mutable list<list<float64>> outM = []; ml::inv(outM, inM); println(outM); ml::terminate(); } } }
This is a sample SPL application that initializes the MATLAB runtime,
performs a matrix inversion via the ml::inv
function, and finalizes the MATLAB runtime.
 Type
sc m M Main
in thesample_spl
directory to create a Makefile for this application.  Then type
make standalone
to build it.  You may see some warnings printed out. Ignore the superfluous warnings
(future versions of Streams may include support for suppressing these
warnings), and type
./output/bin/standalone
to run the application. You should see the following result:[[2,1.5],[1,0.5]]
.
Conclusion
In this article you have seen how to integrate MATLAB functions into Streams applications, without rewriting the function logic in a different language. The approach relies on using the MATLAB compiler to convert the MATLAB code into a C++ shared library, writing a C++ native function that wraps this shared library, and importing the wrapper function into SPL using a function model. Sample source code for the example given in our discussion is provided in this article.
Download
Description  Name  Size 

Sample code for this article  MatlabStreamsIntegration.zip  322KB 
Resources
Learn
 Find out more about "IBM InfoSphere Streams" (May 2011) is the product page for Streams.
 Read about the "MathWorks MATLAB Compiler" (May 2011) is the product page for the MATLAB Compiler.
 Learn more about Information Management at the developerWorks Information Management zone. Find technical documentation, howto articles, education, downloads, product information, and more.
 Follow developerWorks on Twitter.
 Watch developerWorks ondemand demos ranging from product installation and setup demos for beginners, to advanced functionality for experienced developers.
Get products and technologies
 Build your next development project with IBM trial software, available for download directly from developerWorks, or spend a few hours in the SOA Sandbox learning how to implement Service Oriented Architecture efficiently..
Discuss
 Participate in the discussion forum.
 Check out the Stream Processing group in the developerWorks community, featuring InfoSphere Streams.
 Check out the developerWorks blogs and get involved in the developerWorks community.
Comments
Dig deeper into Big data and analytics on developerWorks

Bluemix Developers Community
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

Big data and analytics
Crazy about Big data and analytics? Sign up for our monthly newsletter and the latest Big data and analytics news.

DevOps Services
Software development in the cloud. Register today to create a project.

IBM evaluation software
Evaluate IBM software and solutions, and transform challenges into opportunities.