Overview (IBM InfoSphere Streams Java Operator API)

Primitive Operator Packages
Package	Description
com.ibm.streams.operator	Java Operator API for InfoSphere Streams.
com.ibm.streams.operator.compile	SPL compile time support for Java primitive operators.
com.ibm.streams.operator.control	Java primitive operator interface to the Job Control Plane.
com.ibm.streams.operator.control.variable	Collection of utilities and generic MBeans that can be used to represent control variables in the Job Control Plane (JCP).
com.ibm.streams.operator.encoding	Encoding of tuples into alternate representations.
com.ibm.streams.operator.log4j	SPL trace and logging for the Java Operator API using the Apache log4j logging facilities.
com.ibm.streams.operator.logging	SPL trace and logging for the Java Operator API using the Java™ 2 platform's core logging facilities.
com.ibm.streams.operator.management	Operator integration with the JMX platform MBean server.
com.ibm.streams.operator.meta	Meta-data for SPL composite and bounded types.
com.ibm.streams.operator.metrics	Metrics for Operator and Processing Element.
com.ibm.streams.operator.model	Annotations to define an SPL Java primitive operator.
com.ibm.streams.operator.state	State management for operators.
com.ibm.streams.operator.types	SPL specific types.
com.ibm.streams.operator.version
com.ibm.streams.operator.window	Input port windowing interfaces and classes.
com.ibm.streams.operator.window.sort	Stream window listeners that implement window based sorting.

Mock Operator Framework Packages
Package	Description
com.ibm.streams.flow.declare	Declaration of an SPL directed flow graph of Java primitive operators.
com.ibm.streams.flow.execute	The execution of an SPL directed flow graph that contains Java primitive operators.
com.ibm.streams.flow.handlers	Stream handler implementations.
com.ibm.streams.flow.javaprimitives	Testing of an SPL directed flow graph that contains Java primitive operators.

Native Function Packages
Package	Description
com.ibm.streams.function.model	Annotation to define SPL Java native functions.

Java Operator API for InfoSphere Streams. The Java Operator API provides the ability to develop operators and functions for use in IBM Streams Processing Language (SPL). A Java class implementing Operator is invoked in SPL as a Java primitive operator (or the generic spl.utility.JavaOp operator).

The Java Operator API is contained in
$STREAMS_INSTALL/lib/com.ibm.streams.operator.jar.

The Java Operator API requires these jar files:
• Apache Commons Math 2.2 - $STREAMS_INSTALL/ext/lib/commons-math-2.1.jar
• Apache Log4j 1.2.17 - $STREAMS_INSTALL/ext/lib/log4j-1.2.17.jar
• JSON4J - $STREAMS_INSTALL/ext/lib/JSON4J.jar
The InfoSphere Streams SPL runtime execution environment ensures the classes required by com.ibm.streams.operator.jar are loaded for any Java operator or function. The SPL runtime also loads the samples jar com.ibm.streams.operator.samples.jar.

The Java Operator API requires Java Platform, Standard Edition 7 or higher. When compiling SPL applications (sc) the setting of the environment variable JAVA_HOME defines the virtual machine to be used when the application is submitted (streamtool submitjob). If JAVA_HOME is not set then the virtual machine at $STREAMS_INSTALL/java is used, which is IBM SDK Java Technology Edition Version 7.
The list of supported Java virtual machines is the Java requirements and options for InfoSphere Streams section of the InfoSphere Streams product documentation.

A collection of sample operators, functions and listeners are provided (javadoc), including source code. The source code is available in the jar file
$STREAMS_INSTALL/lib/com.ibm.streams.operator.samples.jar.

What's new in InfoSphere Streams 4.0

• Support for checkpointing of operator state.
• Support for consistent regions of an SPL application.
• Support for a Job Control Plane.
• Java primitive operator parameters now support attributes and custom literals as parameter types.

What's new in InfoSphere Streams 3.2

• Annotations to define an class implementing Operator as a SPL Java primitive operator. Classes annotated with PrimitiveOperator are indexed as primitive operators during toolkit indexing (spl-make-toolkit).
• Annotation to define a static method as a SPL native function. Methods annotated with Function are indexed as native functions during toolkit indexing (spl-make-toolkit).
• Methods to provide context when the operator is invoked in a parallel region (See documentation for the @parallel SPL annotation). Context includes the operator's channel number and logical name, and the maximum number of channels in the parallel region. Identical methods are added to OperatorContextMXBean.
• Utility methods to invoke actions at points in an operator's life cycle, see OperatorLifeCycle.
• Ability to add class libraries to the operator's class loader at runtime.
• Methods to create StreamSchema instances that are based upon an existing schema by extending them with additional attributes, or by removing attributes. These methods may be used within the Java mock SPL runtime framework to generate output port schemas based upon input schemas.
• Access to the reference of operator instances within a Java test graph using getOperatorInstance().

What's new in InfoSphere Streams 3.1

• A Java mock SPL runtime framework to support unit testing of Java classes that implement Operator. The API for the mock framework is contained in packages under the com.ibm.streams.flow name space. An empty operator graph is obtained using OperatorGraphFactory. Adding an operator to the graph returns an OperatorInvocation which then allows the invocation of the operator to be declared, such as its parameters and input and output ports. A declared graph is tested using JavaOperatorTester
• Simplified initialization for custom metrics using the @CustomMetric annotation, including optional automatic registration of a MetricMXBean.
• Additional methods to submit tuples: StreamingOutput.submitAsTuple(Object...), StreamingOutput.submitListAsTuple(java.util.List) and StreamingOutput.submitMapAsTuple(java.util.Map).
• Methods to avoid operators performing background work that is not visible to the SPL Java runtime from being completed: AbstractOperator.createAvoidCompletionThread(com.ibm.streams.operator.OperatorContext), AbstractOperator.createAvoidCompletionThread() and AbstractOperator.createAvoidCompletionThreadIfNoInputs()
• SPL types list<uint8>, list<uint16>, list<uint32> and list<uint64> supported by Tuple.getBuffer(int).
• Typed Tuple getter methods Tuple.getObject(int, Class) and Tuple.getBuffer(int, Class).
• Upgrade to Apache Commons Math 2.2

What's new in InfoSphere Streams 3.0

• SPL compile time operator invocation context checking by annotating public static methods in the operator's class or super-class with @ContextCheck.
• Simplified initialization by automatic setting of operator parameter values through Java bean setter methods by annotating the setter method with @Parameter.
• Support for encoding of tuples using JSON and SPL native encoding.
• Integration with Java platform logging (java.util.logging) and Apache Log4j to support SPL log and trace facilities
• Support for SPL types xml and enum in Java operators
• Simplified nested tuple handling for OutputTuple using OutputTuple.setTuple(int, com.ibm.streams.operator.Tuple) and OutputTuple.assignTuple(int, com.ibm.streams.operator.Tuple).
• Registration of JMX MXBeans for processing element, operator context and metrics in the PE's platform MBean Server to allow integration with JMX facilities, such as threshold notifications for operator metrics.
• Operator to PE port mapping: StreamingData.getPEPortNumber() and StreamingData.isConnectedToPEPort().
• Additional methods on ProcessingElement: ProcessingElement.getApplicationDirectory(), ProcessingElement.getApplicationScope(), ProcessingElement.getInstanceId(), ProcessingElement.getRelaunchCount(), ProcessingElement.isOptimized() and ProcessingElement.setLoggerAspects(String, String...).