There are several configurations across several services which Big Match
depends on.
The following table lists properties, the default settings for them, and their purpose that
Big Match
depends on.
Table 1. hbase-site.xml
| Property Name |
Default Value |
Purpose |
| hbase.coprocessor.master.classes |
com.ibm.mdm.bi.pme.hbase.coprocessor.PMEMasterObserver |
Master Observer for Big Match Hbase processes. |
| hbase.coprocessor.regionserver.classes |
com.ibm.mdm.bi.pme.hbase.coprocessor.PMERegionServerObserver |
Specifies the Big Match specific classes with this implementation. |
| hbase.region.server.rpc.scheduler.factory.class |
com.ibm.mdm.bi.pme.hbase.regionserver.PMERpcSchedulerFactory |
Specifies the Big Match specific classes with this implementation. |
| hbase.regionserver.impl |
com.ibm.mdm.bi.pme.hbase.regionserver.PMERegionServer |
Specifies the Big Match specific classes with this implementation. |
| giraph.zkList |
A comma delimited list of all nodes running oozie server |
Used by Big Match Hbase coprocessors. |
| pme.queue.log.dir |
/hadoop/bigmatch |
This path is used by the big data match queue service for logging output. It defaults to
/hadoop/pme and must exist prior to the queue service being started. If this path
doesn't exist on the nodes when the queue service starts, an exception is logged in the log file for
the HBase region server. The exception indicates that the path is not valid. |
| pme.slave.avg.num.cores |
min(((yarn_scheduler_max_cores * num_datanodes) -
2),floor(((yarn.nodemanager.resource.memory-mb/yarn.scheduler.minimum-allocation-mb)* num_datanodes)
- 2) |
This value is used to set NUMREGIONS when creating HBASE tables. |
| pme.link.per.worker.memory.mb |
2000 |
This parameter will be used if the linking application is put into automatic mode. The
default heap size for each link worker is 1000 MB. It is recommend to set this heap size to 2000 MB
or more. The basic calculation is 2000 additional MB for each additional 1 million records. |
Table 2. oozie-site.xml
| Property Name |
Default Value |
Purpose |
| oozie.credentials.credentialclasses |
hbase=com.ibm.mdm.bi.pme.hbase.util.PMEHbaseCredentials |
The code to obtain delegation tokens is pluggable so that it is easy to add support for
different services by simply subclassing org.apache.oozie.action.hadoop.Credentials
to retrieve a delegation token from the service and add it to the Configuration. |