IBM InfoSphere Streams Version 4.1.0

sc

The sc command compiles SPL applications.

Usage


sc

>>-+----------------------------------+------------------------->
   '-+- -a--------------------------+-'   
     '- --optimized-code-generation-'     

>--+-------------------------------------+---------------------->
   '- --allow-convenience-fusion-options-'   

>--+------------------------+--+---------------------------+---->
   '- --avoid-runtime-yield-'  '-+- -C-------------------+-'   
                                 '- --clean-and-suppress-'     

>--+-------------------------+---------------------------------->
   '- --c++std--=--+-c++98-+-'   
                   '-c++11-'     

>--+-----------------------------------+------------------------>
   '-+- -d------------------+--=--size-'   
     '- --default-pool-size-'              

>--+----------------------------+------------------------------->
   '- --data-directory--=--path-'   

>--+----------------------------------+------------------------->
   '- --default-relocatable--=--value-'   

>--+----------------------------------+------------------------->
   '- --default-restartable--=--value-'   

>--+-----------------------------------+--+--------------+------>
   '-+- -F--------------+--=--protocol-'  '-+- -g------+-'   
     '- --use-transport-'                   '- --debug-'     

>--+-------------+---------------------------------------------->
   '-+- -h-----+-'   
     '- --help-'     

>--+-------------------------------------------+---------------->
   '-+- -j-----------------+--=--thread_number-'   
     '- --num-make-threads-'                       

>--+-----------------------------------------------+------------>
   '-+- -J---------------------------------------+-'   
     '- --suppress-all-but-the-application-model-'     

>--+-----------------------------+------------------------------>
   '-+- -k---------------------+-'   
     '- --prefer-facade-tuples-'     

>--+--------------------------------------+--------------------->
   '-+- -K---------------------+--=--path-'   
     '- --checkpoint-directory-'              

>--+----------------------------+------------------------------->
   '-+- -m--------------------+-'   
     '- --makefile-generation-'     

>--+- -M---------------+--=--name------------------------------->
   '- --main-composite-'            

>--+--------------------------------+--------------------------->
   '- --no-mixed-mode-preprocessing-'   

>--+------------------------+----------------------------------->
   '- --no-toolkit-indexing-'   

>--+-------------------------------------------------------+---->
   '-+- -O-----------------------------------------------+-'   
     '- --set-relax-fusion-relocatability-restartability-'     

>--+------------------------------+----------------------------->
   '- --output-directory--=--path-'   

>--+----------------------------------------------+------------->
   '- --output-side-effects-behavior--=--behavior-'   

>--+---------------------------+-------------------------------->
   '-+- -p----------+--=--mode-'   
     '- --part-mode-'              

>--+-------------------------------------------------+---------->
   '-+- -P----------------------+--=--profile_number-'   
     '- --profile-all-operators-'                        

>--+------------------------------------------------+----------->
   '-+- -r------------------------+--=--host_number-'   
     '- --num-distcc-remote-hosts-'                     

>--+---------------------+--+--------------------+-------------->
   '- --rebuild-toolkits-'  '-+- -s------------+-'   
                              '- --static-link-'     

>--+-----------------------------------------+------------------>
   '-+- -S------------------------+--=--rate-'   
     '- --profiling-sampling-rate-'              

>--+------------------------------------+----------------------->
   '- --state-variable-scoping-behavior-'   

>--+--------------------------+--------------------------------->
   '-+- -t---------+--=--path-'   
     '- --spl-path-'              

>--+-------------------------------+---------------------------->
   '-+- -T-----------------------+-'   
     '- --standalone-application-'     

>--+--------------------------------+--------------------------->
   '- --transport-stat-dir--=--path-'   

>--+---------------------+--+-------------------+--------------->
   '-+- -v-------------+-'  '- --validate-hosts-'   
     '- --verbose-mode-'                            

>--+---------------------------+--+------------------------+---->
   '-+- -w---------+--=--flags-'  '- --window-clause-order-'   
     '- --ld-flags-'                                           

>--+----------------------------+------------------------------->
   '-+- -x----------+--=--flags-'   
     '- --cxx-flags-'               

>--+----------------------------+------------------------------->
   '-+- -z--------------------+-'   
     '- --code-beautification-'     

>--+-----------------------------------------+-----------------><
   '-+- -Z--------------------+--=--fraction-'   
     '- --max-pe-cpu-fraction-'

Authority

You do not need any authority to run this command.

Description

sc is the Stream Processing Language (SPL) compiler for IBM® InfoSphere® Streams. It is used to compile SPL applications. A successful compilation results in the generation of an application bundle file which can be submitted to the InfoSphere Streams instance with the streamtool utility.

Options and arguments

-a, --optimized-code-generation

Specifies to generate optimized code with less runtime error checking. Furthermore, the command compiles the resulting C++ code at the highest optimization level. With this option, the compilation flags used for C++ are -O3 and -NDEBUG for gcc. By default, the generated C++ code is compiled with -ggdb for gcc.

--allow-convenience-fusion-options

Specifies to allow the FNONE, FALL, and FRAND values in the -p or --part-mode options.

--avoid-runtime-yield=value

Specifies the behavior to use when the run time encounters a full queue in a ThreadedPort and must wait to add a tuple to the queue of the ThreadedPort. A performance problem exists when the pthread_yield subroutine is used on RHEL6. This option instructs the run time to use an alternative mechanism when it is waiting on a full queue. The following values are valid:

auto (default): The sc compiler prevents the call to the pthread_yield() subroutine on RHEL6, and forces the call to pthread_yield() on RHEL5.
yes: The sc compiler prevents the call to the pthread_yield() subroutine.
no: The sc compiler forces the call to the pthread_yield() subroutine.

-c, --clean

Specifies to clean the compiler-generated artifacts before compilation.

-C, --clean-and-suppress

Specifies to clean the compiler-generated artifacts and suppress the compilation.

--c++std=c++-support

Specifies the C++ standard supported. When you compile an SPL application with --c++std, all C++ artifacts (all operators and native functions) are compiled under that standard. The SPL compiler detects whether the C++ standard is supported by the underlying C++ compiler. The following values are valid for c++-support:

c++98: The default value is c++98. The SPL compiler passes -std=gnu++98 to the g++ compiler.
c++11: The SPL compiler passes -std=gnu++11 to the g++ compiler.

-d, --default-pool-size=size

Specifies the size of the default host pool. The value must be nonzero (greater than 0). By default, the default pool is unsized.

--data-directory=path

Specifies the location of the data directory.

--default-relocatable=value

Specifies whether operators are relocatable by default. Relocatable operators can be relocated to different resources. If you specify --default-relocatable=true, it implies that --default-restartable=true. For more information, see relocatable.

--default-restartable=value

Specifies whether operators are restartable by default. Restartable operators can be restarted. If you specify --default-relocatable=false, it implies that --default-relocatable=false. For more information, see restartable.

-F, --use-transport=protocol

Specifies the default transport protocol. The protocol can be any one of the following values: tcp; llm_rum_tcp; llm_rum_ib. These values are not case-sensitive. The default value is tcp.

Restriction: The LLM transport is not supported on IBM Power Systems™ that are running little endian.

-g, --debug

Specifies to turn on the stream debugger by adding wrapper: sdb as a config for the main composite. A wrapper config specified in the source file, if any, takes precedence.

-h, --help

Specifies to show the command syntax.

-j, --num-make-threads=thread_number

Specifies the number of threads for building the binaries. This option is used as the -j parameter for the make command during the building of the generated C++ code.

-J, --suppress-all-but-the-application-model

Specifies to create an XML-based model of the application (.adl file), but to suppress the creation of other application artifacts as well as the build process.

-k, --prefer-facade-tuples

Specifies to generate facade tuples when possible.

-K, --checkpoint-directory=path

Specifies the directory to be used for checkpointing. The specified path must be accessible by all application resources that are running processing elements that belong to this application.

Note: The --checkpoint-directory option is deprecated.

-m, --makefile-generation

Specifies to generate a default makefile for building the SPL application.

-M, --main-composite=name

Specifies the main composite to use as the root of the expansion. This value must be a fully qualified name.

--no-mixed-mode-preprocessing

Specifies that the command must not preprocess mixed-mode (.splmm) files.

--no-toolkit-indexing

Specifies that the command must not perform toolkit indexing in the compilation directory.

-O, --set-relax-fusion-relocatability-restartability

Specifies that ex-location constraints are not implied from restartable and relocatable configs. By default, such implications are taken into account. For instance, if an operator A has restartable=false and operator B has restartable=true, they must be partition ex-located.

--output-directory=path

Specifies the directory where the application artifacts are placed.

--output-side-effects-behavior=behavior

Specifies whether to output a message if there are side effects in the output clause. The following options are valid:

ignore: The sc compiler suppresses a warning message.
warn: The sc compiler gives a warning message. This is the default value.
error: The sc compiler gives an error message.

If the –output-side-effects-behavior option is not specified, the sc compiler gives a warning message if there are side effects in the output clause. For information on side effect detection in SPL output clauses, see Side-effects. This option is available in InfoSphere Streams Version 3.0 and later.

-p, --part-mode=mode

Specifies the fusion mode. Available options are:

FDEF: Default fusion (user-controlled)
FOPT: Optimized fusion
FNONE: No Fusion
FALL: Fuse all into one
FRAND: Fuse randomly

Note: FNONE, FALL, and FRAND are only valid if the --allow-convenience-fusion-options option is specified because these fusion modes ignore all placement directives in the SPL program.

-P, --profile-all-operators=profile_number

Specifies the profiling behavior for all operators. The following values are valid:

-1: No profiling (default)
0: Profile but do not keep samples
>0: Profile and keep samples in a reservoir of given size.

-r, --num-distcc-remote-hosts=host_number

Specifies the number of remote hosts to use for distributed compile when you use distcc. The hosts are selected by using the streams_get_available_hosts -query '$build' command. The local host is always part of the compilation.

--rebuild-toolkits

Specifies to run the spl-make-toolkit command, which rebuilds a toolkit if the toolkit is out of date.

The spl-make-toolkit command creates a toolkit.xml file, therefore the directory must be writable by your ID.

-s, --static-link

Specifies to generate statically linked processing elements. This option is required for SELinux support. Installing a new release of InfoSphere Streams requires a recompile of applications that were built with the -s option.

-S, --profiling-sampling-rate=rate

Specifies the sampling rate for profiling. The value must be a floating point value in the range [0.0,1.0]. The default is 1.

--state-variable-scoping-behavior

Specifies that state variables can shadow attributes with the same name.

-t, --spl-path=path

Specifies the toolkit lookup paths. Separate multiple paths with ':'. Each path is a toolkit directory, a directory of toolkit directories, or a toolkit list XML file. This option overrides STREAMS_SPLPATH environment variable.

-T, --standalone-application

Specifies to create a stand-alone application executable. The executable is named standalone and can be located under the bin directory within the output directory.

--transport-stat-dir=path

Specifies the location of the directory to be used for finding transport statistic files.

-v, --verbose-mode

Specifies to put the compiler into verbose mode.

--validate-hosts

Specifies to verify that host names in the placement configs are valid at compile time. The default behavior is to perform the verification when the streamtool submitjob command runs.

-w, --ld-flags=flags

Specifies to pass the flags to ld while linking.

--window-clause-order

Specifies whether to enforce a specific order for the elements in the window clause. The following options are valid:

any: The sc compiler does not enforce any particular window clause order.

strict: The sc compiler gives an error message if the window clause order does not conform to the following syntax.

opInvokeWindow ::= ID ':' ( 'tumbling' ',' evictionSpec
                          | 'sliding' ',' evictionSpec ( ',' triggerSpec )^? )
                          ( ',' 'partitioned' (',' partitionEvictionSpec)^? )^? ';'

If the --window-clause-order option is not specified, the sc compiler gives a warning message if the elements do not adhere to the window clause syntax.

-x, --cxx-flags=flags

Specifies to pass the flags to the C++ compiler while building.

-z, --code-beautification

Specifies to generate nicely formatted C++ code. The uncrustify tool needs to be available in the system for this feature to work.

-Z, --max-pe-cpu-fraction=fraction

Specifies the maximum processing element CPU fraction to be used for profile-driven automatic fusion. The value must be positive (greater than or equal to 0), and the default is 0.8.

Examples

Assume the following directory layout:

\_ sample/
      \_ WordCount.spl

Further assume that there is a main composite named sample::WordCount defined in the SPL program.

Generate a make file: sc -m -M sample::WordCount. This results in the following directory layout:
```
\_ Makefile
   sample/
     \_ WordCount.spl
```
The generated makefile can be used to compile the application by typing make.
Compile an application: sc -M sample::WordCount. This results in the following directory layout:
```
\_ data/
   output/
     \_ sample.WordCount.sab
        ...
   sample/
     \_ WordCount.spl
   toolkit.xml
```
The generated .sab file can be used to submit the application to the InfoSphere Streams instance.
Compile an application as standalone: sc -T -M sample::WordCount. This results in the following directory layout:
```
\_ data/
   output/
     \_ bin/
          \_ standalone
             ...
        ...
   sample/
     \_ WordCount.spl
   toolkit.xml
```
The generated stand-alone executable can be run by using the following command: ./output/bin/standalone.

Use of the C++11 option when compiling your toolkit

When you implement primitive operators and native functions in C++11, document this fact for the users of your toolkit. Toolkit users need to inform the SPL compiler that their application requires compiling under C++11.

In the C++ code, you can check the code is being compiled under C++11 by using the __cplusplus macro.

    #if __cplusplus < 201103L
    #error "This toolkit requires C++11, but is not currently being compiled as C++11."
    #endif

You can also use the opposite comparison to determine whether C++11 is used. This code can be used to take advantage of C++11 features when present, but still work if not.

    #if __cplusplus >= 201103L
    // C++11 specific code
    #else
    // pre-C++11 code
    #endif