Db2 Big SQL I/O engine
The Db2 Big SQL I/O engine (readers and writers) comprises the Db2 Big SQL layer that serves as the interface between the Db2 relational engine and the external storage layer.
The I/O engine uses Java readers and writers to access and handle the data encoded in supported file formats or user-defined formats where a custom SerDe was provided.
To protect the memory sets and agents of the Db2 relational engine, fenced-mode processes (FMPs) are used to host the I/O engine. Because the fenced-mode process (db2fmp) is isolated from the processes that interact with the database, if there is a problem with an FMP, the database is not affected.
Unstable Java DFSIO FMP
If an error occurs within the I/O engine or environment affecting the corresponding Java DFSIO FMP that we cannot reliably recover from (for example, an out-of-memory condition), the FMP is marked as unstable. This prevents the unstable I/O engine from receiving additional requests, and a new Java DFSIO FMP is created to handle any new requests. The unstable FMP attempts to complete any outstanding work until a new request comes in, which causes the unstable FMP to be terminated along with any on-going work. A SQL5199N RC1 is returned, and a new FMP is started to handle the new request. The unstable FMP is terminated in this case to minimize the spike in resource usage with two FMPs running simultaneously. In the severe case that Java DFSIO FMPs continue to go unstable, an SQL5199N RC4 might be returned indicating that the allowed maximum number of FMPs already exists as new FMPs are created to replace the unstable ones. This generally might indicate that the memory allocated to the Java DFSIO FMPs is not sufficient for the current workload, there is some rogue query causing the OOM, or some other spike occurred in the workload.
Common errors
- SQL5104N
- Indicates that the statement failed because data in the table does not match the external table definition. This message is commonly returned when an attempt is made to read a null value from a column that is defined as NOT NULL, or an attempt is made to insert a row that has a zero-length partitioning column value.
- SQL5105N
- Indicates that an exception was encountered while accessing external data. This message is returned for many different reasons and from many different components, including open source components of Db2 Big SQL.
- SQL5197N
- Indicates that the statement failed because of a communication error with the I/O engine. This message is most commonly returned when data being inserted into or read from an external table exceeds the defined size of the communication buffer.
- SQL5199N
-
Indicates that the statement failed because a connection to the I/O engine could not be established or maintained. This message is returned when the database manager encounters an error with an FMP that is hosting a Db2 Big SQL I/O component. The reason code indicates the source of the problem.
For more information, see SQL messages: SQL5000 - SQL5999. For many errors like SQL5105, you can find additional information about the error in log files, identified by a log entry identifier. For details about log entry identifiers and the locations of the log files, see Db2 Big SQL log files.