Reading files in HDFS subdirectories

Db2® Big SQL can be configured to read data files in subdirectories.

For example, consider an external table that is located in /tmp/testtable/:


[root@bdsup006 ~]# hadoop fs -ls -R /tmp/testtable/
drwxr-xr-x - bigsql hdfs 0 2015-12-09 22:29 /tmp/testtable/subtab
-rw-r--r-- 3 bigsql hdfs 6 2015-12-09 22:29 /tmp/testtable/subtab/text2.txt
-rw-r--r-- 3 bigsql hdfs 6 2015-12-09 22:28 /tmp/testtable/text1.txt

The scheduler.read.subdirectories parameter (in Ambari under IBM Db2 Big SQL > Configs > Advanced bigsql-conf) controls whether data in subdirectories is read by Db2 Big SQL. Setting this option to true enables Db2 Big SQL to read data in subdirectories. In this example, both text1.txt and text2.txt are read. If you set this option to false, only text1.txt is read. You must restart the Db2 Big SQL service after you set this configuration parameter.