Db2 Big SQL readers and writers

The Db2® Big SQL readers and writers comprise the Db2 Big SQL layer that serves as the interface between the relational engine and the external storage layer. To support all data formats that are commonly used in Hadoop, Db2 Big SQL has an external table interface that uses Java™ readers and writers.

By using the Java I/O interface, Db2 Big SQL can read and write data that is encoded in all formats that are handled by Hive. Formats such as ORC and user-defined formats that require custom Hive SerDes are handled through the Java interface. For more information, see File formats that are supported by Db2 Big SQL and Extending Db2 Big SQL.

To protect its memory sets and agents, the database manager underlying the Db2 Big SQL engine uses fenced-mode processes (FMPs) to establish and maintain connections with an I/O engine. Because the fenced-mode process (db2fmp) is isolated from the processes that interact with the database, if there is a problem with an FMP, the database will not be affected. The readers and writers are independent services that run in the FMPs of a database, and the Db2 Big SQL I/O libraries are installed as part of Db2 Big SQL. For more information, see The Db2 process model.

For optimal performance and functionality, the ORC file format is recommended for Db2 Big SQL.

Here are the most common errors that could be returned from the I/O engine:

SQL5104N: Indicates that the statement failed because data in the table does not match the external table definition. This message is commonly returned when an attempt is made to read a null value from a column that is defined as NOT NULL, or an attempt is made to insert a row that has a zero-length partitioning column value.
SQL5105N: Indicates that an exception was encountered while accessing external data. This message is returned for many different reasons and from many different components, including open source components of Db2 Big SQL.
SQL5197N: Indicates that the statement failed because of a communication error with the I/O engine. This message is most commonly returned when data being inserted into or read from an external table exceeds the defined size of the communication buffer.
SQL5199N: Indicates that the statement failed because a connection to the I/O engine could not be established or maintained. This message is returned when the database manager encounters an error with an FMP that is hosting a Db2 Big SQL I/O component. The reason code indicates the source of the problem.

For more information, see SQL messages. For many errors like SQL5105, you can find additional information about the error in log files, identified by a log entry identifier. For details about log entry identifiers and the locations of the log files, see Db2 Big SQL log management on CDP.