webMethods Knowlegebase : Unable to get a connection to Database."IO Error: Socket read timed out" from IS to the database. (1802749)

Problem

An application is running on the Integration Server (IS) 10.1 and user is using Oracle 11g running on Oracle Database Appliance as a database.

IS is able to make the initial connection with the listener and then listener provides the details about which database it should connect back to the IS and then IS should connect to the database mentioned by the listener. However, according to user's network team, after listener providing the database details to the IS, IS is not connecting to the database for 5 seconds. As user has the 5 seconds connection timeout defined in the connection string, the timeout is kicking in and there is a "IO Error: Socket read timed out" from IS to the database.

The connection is working fine sometimes and failing with below mentioned error sometimes. The same can be also observed from the below logs within a minute we a successful connection and a failure connection. User frequently gets the below errors in server log about server unable to get a connection to the database

Successful case:

[ISP.0090.0003I] MID DB test started

[SCC.0126.0206T] Created connection EXDI.DB_Connection:MID_connection(2713) in the <Default> partition of EXDI.DB_Connection:MID_connection

[SCC.0126.0204T] Connection EXDI.DB_Connection:MID_connection(2713) in the <Default> partition of EXDI.DB_Connection:MID_connection is now busy

[ISP.0090.0003I] MID DB test completed

Failure case:

[ISP.0090.0003I] MID DB test started

[ART.0114.1007E] Adapter Runtime: Error Logged. See Error log for details. Error: [ADA.1.204] Cannot connect to the database with DataSource class "oracle.jdbc.pool.OracleConnectionPoolDataSource".

IO Error: Socket read timed out

[SCC.0126.0119E] An error occurred while attempting to create connections for EXDI.DB_Connection:MID_connection

[SCC.0126.0106E] An error occurred while attempting to retrieve a connection from EXDI.DB_Connection:MID_connection

[SCC.0126.0202D] SCC ConnectionManager pool EXDI.DB_Connection:MID_connection reset

[ART.0114.1007E] Adapter Runtime: Error Logged. See Error log for details. Error: [ART.118.5053] Adapter Runtime (Connection): Unable to get a connection to resource EXDI.DB_Connection:MID_connection.

[ADA.1.204] Cannot connect to the database with DataSource class "oracle.jdbc.pool.OracleConnectionPoolDataSource".

IO Error: Socket read timed out

The network team analyzed the network packet captures while establishing the connection from IS to database and mentioned below findings.

- The Integration Server initiating the database connection.

- The TNS listener on the ODA replying back by telling the IS to which database it should connect out of two databases.

- In successful case the IS is connecting to the database and the process is completing successfully

- In failure case, the IS is waiting for 5 seconds without connecting to the mentioned database by the TNS listener and sending a connection reset after 5 seconds (which is the connection timeout mentioned in our connection string on IS. Provided below is our connection String)

url=jdbc:oracle:thin:@(DESCRIPTION=(CONNECT_TIMEOUT=5)(RETRY_COUNT=3)(ADDRESS_LIST=(FAILOVER=ON)(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=<Primary database>)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=<Secondary database>)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=******)));