distinctnode Properties

The Distinct node removes duplicate records, either by passing the first distinct record to the data stream or by discarding the first record and passing any duplicates to the data stream instead.

Example

node = stream.create("distinct", "My node")
node.setPropertyValue("mode", "Include")
node.setPropertyValue("fields", ["Age" "Sex"])
node.setPropertyValue("keys_pre_sorted", True)

Table 1. distinctnode properties
`distinctnode` properties	Data type	Property description
`mode`	`Include` `Discard`	You can include the first distinct record in the data stream, or discard the first distinct record and pass any duplicate records to the data stream instead.
`grouping_fields`	list	Lists fields used to determine whether records are identical. Note: This property is deprecated from IBM® SPSS® Modeler 16 onwards.
`composite_value`	Structured slot	See example below.
`composite_values`	Structured slot	See example below.
`inc_record_count`	flag	Creates an extra field that specifies how many input records were aggregated to form each aggregate record.
`count_field`	string	Specifies the name of the record count field.
`sort_keys`	Structured slot.	Note: This property is deprecated from IBM SPSS Modeler 16 onwards.
`default_ascending`	flag
`low_distinct_key_count`	flag	Specifies that you have only a small number of records and/or a small number of unique values of the key field(s).
`keys_pre_sorted`	flag	Specifies that all records with the same key values are grouped together in the input.
`disable_sql_generation`	flag

Example for composite_value property

The composite_value property has the following general form:

node.setKeyedPropertyValue("composite_value", FIELD, FILLOPTION)

FILLOPTION has the form [ FillType, Option1, Option2, ...].

Examples:

node.setKeyedPropertyValue("composite_value", "Age", ["First"])
node.setKeyedPropertyValue("composite_value", "Age", ["last"])
node.setKeyedPropertyValue("composite_value", "Age", ["Total"])
node.setKeyedPropertyValue("composite_value", "Age", ["Average"])
node.setKeyedPropertyValue("composite_value", "Age", ["Min"])
node.setKeyedPropertyValue("composite_value", "Age", ["Max"])
node.setKeyedPropertyValue("composite_value", "Date", ["Earliest"])
node.setKeyedPropertyValue("composite_value", "Date", ["Latest"])
node.setKeyedPropertyValue("composite_value", "Code", ["FirstAlpha"])
node.setKeyedPropertyValue("composite_value", "Code", ["LastAlpha"])

The custom options require more than one argument, these are added as a list, for example:

node.setKeyedPropertyValue("composite_value", "Name", ["MostFrequent", "FirstRecord"])
node.setKeyedPropertyValue("composite_value", "Date", ["LeastFrequent", "LastRecord"])
node.setKeyedPropertyValue("composite_value", "Pending", ["IncludesValue", "T", "F"]) 
node.setKeyedPropertyValue("composite_value", "Marital", ["FirstMatch", "Married", "Divorced", "Separated"])
node.setKeyedPropertyValue("composite_value", "Code", ["Concatenate"])
node.setKeyedPropertyValue("composite_value", "Code", ["Concatenate", "Space"])
node.setKeyedPropertyValue("composite_value", "Code", ["Concatenate", "Comma"])
node.setKeyedPropertyValue("composite_value", "Code", ["Concatenate", "UnderScore"])

Example for composite_values property

The composite_values property has the following general form:

node.setPropertyValue("composite_values", [
                        [FIELD1, [FILLOPTION1]],
                        [FIELD2, [FILLOPTION2]],
.
.
])

Example:

node.setPropertyValue("composite_values", [
    ["Age", ["First"]],
    ["Name", ["MostFrequent", "First"]],
    ["Pending", ["IncludesValue", "T"]],
    ["Marital", ["FirstMatch", "Married", "Divorced", "Separated"]],
    ["Code", ["Concatenate", "Comma"]]
])