PH07586: QUERIES AGAINST PARQUET FILE FORMAT TABLES WITH COMPLEX TYPES (ARRAY, ROW or STRUCT) FAIL WITH ERROR

APAR status

• Closed as fixed if next.

Error description

• After querying a table which includes a complex type (ARRAY,
  ROW or STRUCT), all subsequent queries against PARQUET file
  format tables will use the non-optimized reader, which will
  result in poor performance.   You might also see a trap similar
  to the following when selecting from a PARQUET file format
  table:
  db2diag.log:
  FUNCTION: DB2 UDB, BSU Java support, sqlejLogExceptionToDiag,
  probe:1519
  DATA #1 : String, 403 bytes
  COM.ibm.db2.app.DB2ExtTableException:
  DeprecatedParquetHiveInput : size of object differs. Value size
  :  0, Current Object size : 21
      at
  com.ibm.biginsights.bigsql.dfsrw.DfsException.mapToDb2Exception(
  DfsException.java:161)
      at
  com.ibm.biginsights.bigsql.dfsrw.DfsException.mapAndLogException
  (DfsException.java:245)
      at
  com.ibm.biginsights.bigsql.dfsrw.reader.DfsBaseReader.read(DfsBa
  seReader.java:703)
  bigsql.log:
  2018-12-27 15:48:44,079 ERROR
  com.ibm.biginsights.bigsql.dfsrw.reader.DfsBaseReader
  [Master-17-S:21.17.1000001.175.0.23282] : [BSL-17-7f16d302e]
  Exception raised by Reader at node: 17 Scan ID:
  S:21.17.1000001.175.0.23282 Table: bigsql.testtable Spark:
  false VORC: false VPQ: false VAVRO: false VTEXT: false VRCFILE:
  false VANALYZE: false
  Exception Label: UNMAPPED(java.io.IOException:
  DeprecatedParquetHiveInput : size of object differs. Value size
  :  0, Current Object size : 21)
  java.io.IOException: DeprecatedParquetHiveInput : size of
  object differs. Value size :  0, Current Object size : 21
      at
  org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWra
  pper.next(ParquetRecordReaderWrapper.java:216)
      at
  org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWra
  pper.next(ParquetRecordReaderWrapper.java:61)
      at
  com.ibm.biginsights.bigsql.dfsrw.reader.DfsHiveSplit2Batch.split
  2Batch(DfsHiveSplit2Batch.java:110)
      at
  com.ibm.biginsights.bigsql.dfsrw.jaro.DfsSplitManager$SplitRunna
  ble.run(DfsSplitManager.java:123)
      at
  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExec
  utor.java:1160)
      at
  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe
  cutor.java:635)
      at java.lang.Thread.run(Thread.java:785)
      at
  com.ibm.biginsights.bigsql.dfsrw.jaro.DfsSplit2BatchThread.run(D
  fsSplit2BatchThread.java:58)
  

Local fix

• - Recycle Big SQL to restore performance
  - Use ORC file format tables
  

Problem summary

• Please see problem description
  

Problem conclusion

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels