FMP memory can be configured with the help of the registry variable DB2_FMP_COMM_HEAPSZ and the database manager configuration parameter aslheapsz.
The default DB2_FMP_COMM_HEAPSZ is 256 MB on AIX and 20 MB on other OS
memory used by each FMP process = 2 * ASLHEAPSZ
If the active fenced routines exceed this limit then we see the following message in the db2diag.log
MESSAGE : Insufficient memory available for IPC communication with the db2fmp
process. Use the DB2_FMP_COMM_HEAPSZ registry variable to adjust the
amount of memory available for fenced routines.
FMP processes that would be active on the database = 500
ASLHEAPSZ set to 128 4K pages = 512 KB
Memory taken by each FMP process = 2 * ASLHEAPSZ = 2 * 512 = 1 MB
To accommodate 500 FMP processes we need 500 MB of FMP memory
db2 set DB2_FMP_COMM_HEAPSZ = 128000 (4K pages)