Minimum and maximum range partitioned read method
When this method is specified, the connector calculates the minimum and maximum values for the column that is specified in the Partitioned reads method > Column name property and then divides the calculated range into subranges. Each subrange is then assigned to a node; the number of subranges equals the number of nodes that are configured for the stage. On each node, the connector runs a SELECT statement that returns the rows where the value in the specified column is in the subrange that was assigned to that node.
To use this method, you must specify a column name from the source table in the
property. The specified column must exist in the table that is specified in the property. The column that you specify must have a TINYINT, SMALLINT, INTEGER or BIGINT data type.The value for the Select statement property is used only if the Generate SQL property is set to No. Add [[range-part-expr]] placeholder to the SELECT statement expression in the WHERE clause where you see fit. Hive connector will process the SELECT statement from the Select statement property and replace the [[range-part-expr]] placeholder with appropriate condition, different on each node.
Example of using the minimum and maximum range partitioned read method
- The Generate SQL property is set to No.
- The Select statement property is set to SELECT * FROM TABLE1 WHERE [[range-part-expr]] AND COL1 > 10.
- The connector is configured to run in parallel mode on four nodes.
- The Partitioned reads method property is set to Minimum and maximum range.
- The TABLE1. property is set to
- The COL2, and COL2 is defined as INTEGER in TABLE1. property is set to
SELECT * FROM TABLE1 WHERE COL2 < 19 AND (COL1 > 10)
SELECT * FROM TABLE1 WHERE (COL2 >= 19 AND COL2 < 58) AND (COL1 > 10)
SELECT * FROM TABLE1 WHERE (COL2 >= 58 AND COL2 < 97) AND (COL1 > 10)
SELECT * FROM TABLE1 WHERE (COL2 >= 97) AND (COL1 > 10)