Writable
All key and value classes used in map and reduce signatures have to implement a Writable interface. This allows the framework to serialize/deserialize <key,value> pairs to and from database records.
The map/reduce API provides implementation of the following basic Writables: BooleanWritable, IntWritable, LongWritable, FloatWritable, DoubleWritable, Text, Ntext, and NullWritable.
A writable interface allows you to specify multicolumn key/value types. The getStorageTypesList()
method returns a list of classes that can be mapped by the framework to types of database columns.
The following table shows the mapping:
Java class | Data type |
---|---|
java.lang.Boolean | BOOL |
java.lang.Byte | BYTEINT |
java.lang.Short | SMALLINT |
java.lang.Integer | INTEGER |
java.lang.Long | BIGINT |
java.lang.Float | FLOAT |
java.lang.Double | DOUBLE |
java.lang.String | VARCHAR |
org.netezza.inza.mr.io.type.NString | NVARCHAR |
The following example of a custom Writable consists of three fields: int, float, and boolean. The
getStorageTypesList() method used in the example returns a list of corresponding java classes:
Integer.class, Float.class, Boolean.class. The method allows the framework to create appropriate
readers and
writers.
public class CustomWritable implements Writable {
private int a;
private float b;
private boolean c;
@Override
public void write(RecordOutput out) throws IOException {
out.writeInt(a);
out.writeFloat(b);
out.writeBoolean(c);
}
@Override
public void readFields(RecordInput in) throws IOException {
a = in.readInt();
b = in.readFloat();
c = in.readBoolean();
}
@Override
public List<Class<?>> getStorageTypesList() {
List<Class<?>> ret = new ArrayList<Class<?>>();
ret.add(Integer.class);
ret.add(Float.class);
ret.add(Boolean.class);
return ret;
}
}
Output database types are strictly determined by Writable's storage types defined in
getStorageTypesList() method. However, the framework supports the following automatic conversions
while reading data from database records to Writable types.
Boolean | Integer | Long | Float | Double | Text | Ntext | |
---|---|---|---|---|---|---|---|
BOOL | YES | YES | YES | ||||
FLOAT | YES | YES | YES | YES | |||
DOUBLE | YES | YES | YES | ||||
BYTEINT | YES | YES | YES | YES | YES | ||
SMALLINT | YES | YES | YES | YES | YES | YES | |
INTEGER | YES | YES | YES | YES | YES | YES | |
BIGINT | YES | YES | YES | YES | YES | ||
NUMERIC | YES | YES | |||||
CHAR | YES | YES | YES | YES | YES | YES | YES |
VARCHAR | YES | YES | YES | YES | YES | YES | YES |
NCHAR | YES | YES | YES | YES | YES | YES | YES |
NVARCHAR | YES | YES | YES | YES | YES | YES | YES |
DATE | YES | YES | |||||
TIME | YES | YES | |||||
TIMESTAMP | YES | YES | |||||
TIMETZ | YES | YES | |||||
INTERVAL | YES | YES |