SPL File DpsPutTTLWithReconnect.spl

Specialized toolkits - release 4.3.1.0-i20200220 > com.ibm.streamsx.dps 4.1.1 > com.ibm.streamsx.store.distributed > DpsPutTTLWithReconnect.spl

Content

Operators
  • DpsPutTTLWithReconnect: DpsPutTTLWithReconnect Uses the dpsPutTTL function to write key-value pairs to Redis (of any type).

Composites

composite DpsPutTTLWithReconnect(output Out; input In)

DpsPutTTLWithReconnect Uses the dpsPutTTL function to write key-value pairs to Redis (of any type). The purpose of this composite is to provide extensive error handling, a reconnection policy, and a complete set of informative metrics.

By default, keys and values are not encoded in order to provide maximum ease of access via non-Streams methods. If non-String types are provided for either the key or value, they will automatically be encoded. Use the Base64 encoding options to save space.

Note: The following metrics are only valid once we have established an initial connection to the DPS server. Before that connection, we will throw exceptions (without failure) and the isConnected metric will be valid. All other metrics will be unusable until first-connection is established.

Longterm and Current Metrics:
  • isConnected Maintains a metric of if we think we are connected to Redis or not.
  • totalInserts Maintains a metric of the number of total inserts done since startup.
  • totalFailedInserts Maintains a metric of the number of failed inserts since startup.
  • totalReconnections Maintains a metric of the number of reconnections to the DPS Server since startup.

Interval Metrics:

  • numInserts Maintains a metric of the number inserts completed in the last interval of time.
  • numFailedInserts Maintains a metric of the number of failed inserts done.
  • numUniqueKeyInserts Maintains a metric of the number of unique keys inserted in the last interval of time.
  • reconnectionAttptsSinceDisconnect Maintains a metric of the number of reconnection attempts in the last interval of time.
  • avgTTLMinutes Maintains a metric of the average TTL in minutes over the last interval of time.
  • avgKeyValueSizeBytes Maintains a metric of the average key-value size in the last interval of time. The size is the amount of space utilized after any encoding by the DPS toolkit. Failed inserts contribute a value of 0 to the avgKeyValueSize, skewing the results downwards.
  • maxKeyValueSizeBytes Maintains a metric of the max key-value size in the last interval of time. The size is the amount of space utilized after any encoding by the DPS toolkit.

This operator forwards punctuation.

Parameters

  • keyEncoded: (default: false) Whether or not to encode the key in base64. False is only valid if the key is rstring/ustring, otherwise it automatically defaults to true.
  • valueEncoded: (default: false) Whether or not to encode the value in base64. False is only valid if the value is rstring/ustring, otherwise it automatically defaults to true.
  • reconnectionAttempts: (default: 1) Number of reconnection attempts per incoming tuple. Use -1 for unlimited, i.e. don't ever drop tuple, keep trying to reconnect. If set to 0, we will never reconnect upon connection loss.
  • reconnectionInterval: (default: 5s) Number of seconds to wait between each reconnection attempt.
  • outputType: (default: rstring key, rstring value) The output stream can be used to forward any attributes on the input port to the output port.
  • metricsIntervalLength: (default: 15 * 60s) Time in seconds of the sliding window that the interval metrics are maintained over.
  • metricsUpdateRate: (default: 15.0s) Time in seconds between update of interval metrics.
  • configFile: (default: etc/no-sql-kv-store-servers.cfg) the name of the configuration file used for DPS
  • keyAttribute: the name of an input stream attribute containing the key for the put operation. The attribute can be of any type.
  • valueAttribute: the name of an input stream attribute containing the value for the put operation. The attribute can be of any type.
  • ttlAttribute: the name of an input stream attribute containing the TTL (time-to-live) value for the put operation. The attribute must be of type uint32. The value is given in seconds. A value of 0 means no expiration.

Input Ports

  • In: Stream to be put using DPS. Attributes containing the key, value, and ttl must be provided (they do not need to be called any specific name, as they are specified as parameters)

Output Ports

  • Out: Stream containing attributes that match input attributes.