createTypes script

The script provides the following optional parameters to control the output types produced and the parameters of the TextExtract operator:

• compositeName: The name to use for the composite operator that contains the TextExtract operator. By default, the script derives a composite name from the AQL file.
• convertSpan: If this parameter is specified, the script converts span data types to string data types.
• copyallattributes: Copy all the original attributes to the output. The new attributes are added to the end of the original attributes. This parameter can optionally be used with the makecomposite parameter.
• extDictionaries: The names (including the path names) for the external dictionaries.
• extraAttr attrName: Deprecated: This parameter is deprecated and might become obsolete. Use the copyallattributes parameter instead.

  Add an attribute with the name attrName to all the output streams (except the passThrough stream) of the composite operator. This attribute must be present on the input stream for the composite, and you must pass the type of the attribute to the composite operator in the attrName_type parameter. This parameter can be specified multiple times.

• extTables: The names (including the path names) for the external tables.
• floattype: The type of float variable to use. Options are float32 or float64.
• help: Displays usage information for the script.
• inputDoc: The default text field. See the text parameter description for the TextExtract operator.
• inttype: The type of integer to use. Options are int32 or int64.
• languageCode: The ISO language code that is used by IBM BigInsights.
• main: Create a Main composite operator to call the created composite operator.
• makecomposite: Create a composite operator as well as the types.
• moduleOutputDir: The path name where the modules are saved after they are compiled.
• modulePath: The path name where the modules are located.
• moduleSearchPath: The path to search for both uncompiled modules. Most useful when consuming exported AQL from BigInsights Text Analytics web tool.
• modules: The names of the modules to be loaded.
• namespace: The namespace of the composite operator. You must create this namespace directory and move the generated files into it.
• noconvertSpan: If this parameter is specified, do not convert spans to strings.
• nospl: Do not generate SPL; print the views and types to standard output and then exit.
• outputfile: The name of the output file. The SPL extension is automatically added if it is not specified.
• outputMode: The output mode. Options are singlePort or multiPort.
• outputview: The output view to activate. It can be specified multiple times. If this parameter is not specified, all output views are included.
• passThrough: Set the passThrough parameter to true in the composite operator.
• spantype: Set the type for Span data types and set the noconvertSpan parameter.
• stringtype: The type of string. Options are ustring or rstring.
• strictMapping: Control whether or not null values returned from the BigInsights Text Analytics runtime result in an exception or are ignored.
• suppressPunctuation: Create a suppressPunctuation parameter in the composite operator with a default value of true.
• tokenizer: The default tokenizer parameter. See the tokenizer parameter description for the TextExtract operator.
• uncompiledModules: The names of the modules that are to be compiled.

createTypes.pl --uncompiledModules <path_to_custom_module>
--modulePath <path_to_pre_built_extractors>/BigInsightsWesternNERStandard.jar
--makecomposite
--main
--compositename sample

type PersonType = rstring firstname, rstring middlename, rstring lastname, rstring person;
type myPerson2Type = rstring first;
type AllAnnotationsType = list<PersonType> Person , list<myPerson2Type> myPerson2 ;
// a composite for outputMode singlePort
public composite sample(input inputStream;
output AllAnnotationsStream ) { 
graph
  (stream<AllAnnotationsType> AllAnnotationsStream) 
  = com.ibm.streams.text.analytics::TextExtract(inputStream) {
    param
      modulePath: 
      "<path_to_pre_built_extractor>/BigInsightsWesternNERStandard.jar";
      uncompiledModules: "<path_to_custom_module>";
      outputMode: "singlePort";
      tokenizer: "STANDARD";
  }
}

createTypes.pl --moduleSearchPath exportedAQL/ --outputView PersonSearch  --makecomposite --outputfile PersonSearchExtractor.spl --outputMode multiPort --main --compositeName PersonSearchExtractor

	type PersonSearchType = rstring name, rstring city;
	
	// a composite for outputMode multiPort
	public composite PersonSearch(input InputStream;
			output PersonSearchStream) {
	
	graph
		(stream<PersonSearchType> PersonSearchStream) 
			= com.ibm.streams.text.analytics::TextExtract(InputStream) {
		param
			moduleSearchPath: "/homes/sample/exportedAQL";
			outputMode: "multiPort";
			outputViews: "PersonSearch";
			tokenizer: "STANDARD";
		}
	}

composite Main {
        param
    expression<rstring> $inputFile: getSubmissionTimeValue("inputFile");
        graph
       
        stream<rstring inputText> Instream = FileSource() {
            param
          file: $inputFile;
          format: line;
        }
	stream<PersonSearchType> PersonSearch=PersonSearch (Instream) {
        }

	() as SinkPersonSearch = FileSink(PersonSearch) {
		 param
		   file: "PersonSearch_PersonSearch.txt";
	}
}