Set maximum Java heap size

Edit online

Before running an analytics configuration that involves seasonality, you must set the maximum Java heap size.

About this task

The formula for the lowest recommended maximum Java heap size needed for configuring seasonality runs depends on the following data points:
 A  Total number of rows in the historical event database that match the configuration filter and the data range set
To retrieve this value, use an SQL statement to count the total number of rows from the start of the report to the end, with the filter in the where clause. For example, for Oracle, using a date range of 1st July to 31st July and a filter of Severity>=3, the SQL would be:
select count(*) from <schema>.<table_name> where FIRSTOCCURRENCE >= to_date('2017-07-01 00:00:00','yyyy-mm-dd hh24:mi:ss') AND FIRSTOCCURRENCE <=  to_date('2017-07-31 00:00:00','yyyy-mm-dd hh24:mi:ss') and Severity>=3
 B  Total number of distinct event identity values in the historical event database that match the configuration filter and the data range set
The Identifier field is typically used for the event identity. To retrieve this value you need the number of distinct Identifier values in the historical event database that match the configurations filter and the data range set.
 C  Maximum length of the event identity field in the historical event database
This is typically the maximum length of the Identifier field. For example, for a DB2 historical event database with a REPORT_STATUS table, use SQL similar to the following: 
select COLNAME, TYPENAME AS DATA_TYPE from syscat.columns where TABNAME = 'REPORTER_STATUS'
Multiply this value by 2 and add 80 as this is a character-based field.
 D  Length of the name of the event identity field
Find the length of the name of the event identity field, typically 10 as Identifier is usually used. Multiply this value by 2 and add 80 as this is a character-based field. Then for each aggregate (rollup) field, both the name of the aggregate (rollup) field, and the data for the aggregate (rollup) field must be stored.
 E  Length of the name of each aggregate (rollup) field
Find the length of the name of each aggregate (rollup) field . As there can be many aggregate (rollup) fields, we will call these E1, E2 and so on. Multiply each value by 2, and add 80 as these are character-based fields.
 F  Number of bytes for the data for all of the aggregate (rollup) fields
As there can be many aggregate (rollup) fields, we will call these F1, F2 and so on. For numeric aggregate (rollup) fields the number of bytes is typically 8. For character-based aggregate (rollup) fields the number of bytes is the maximum length of the rollup field, in the historical event database , multiplied by 2 plus 80. To find the length for each column, use SQL as follows for an Oracle Database: using a where clause to restrict the time frame and filter by report setup.
select max(vsize(column_name)) from <schema>.<table_name> where FIRSTOCCURRENCE >= to_date('2017-07-01 00:00:00','yyyy-mm-dd hh24:mi:ss') AND FIRSTOCCURRENCE <= to_date('2017-07-31 00:00:00','yyyy-mm-dd hh24:mi:ss') and Severity >=3

Procedure

  1. Calculate the memory required for storing the Identifier fields with each date occurrence.
    The formula is:
    B * (C + D) + A * (320)
  2. Calculate the memory required for storing the Identifier fields containing aggregate (rollup) fields.
    The formula is :
    B * (C + D + E1 + F1 +E2 + F2 + E3 + F3 + ...En + Fn)
    Where n is the number of aggregate (rollup) fields.
  3. Add an extra 500MB for other processing.
  4. Log into Netcool®/Impact and set the maximum heap size based on the calculated value, as described in Setting the memory for the Java Virtual Machine. For example, based on the values in the following example, you would set the maximum heap size as 35 GB.

Example

This section provides samples values from a production environment.
 A  Total number of rows in the historical event database that match the configuration filter and the data range set: 19,000,000
19 million total rows qualify for the configuration filter and date range.
 B  Total number of distinct event identity values in the historical event database that match the configuration filter and the data range set: 18,000,000
18 million distinct Identifier values. In the historical event database Identifier values reoccur. This is how the algorithm works out if the events are seasonal..
 C  Maximum length of the event identity field in the historical event database: 121
Multiply this value by 2 and add 80 as this is a character-based field.
C = 121 * 2 + 80 = 322
 D  Length of the name of the event identity field: 10
Multiply this value by 2 and add 80 as this is a character-based field.
C = 10 * 2 + 80 = 100
 E and F  Aggregate (rollup) field calculations
The aggregate (rollup) fields are as follows:
Number Name Type Space for name (E) Space for data (E)
1 Severity Numeric 
8 * 2 + 80 = 96
 
8
2 Severity Numeric 
8 * 2 + 80 = 96
 
8
3 FirstOccurrence Numeric 
14 * 2 + 80 = 108
 
8
4 AlertGroup Character 
10 * 2 + 80 = 100
 
99 * 2 + 80 = 278
5 Node Character 
4 * 2 + 80 = 88
 
21 * 2 + 80 = 122
 Step 1  Calculate the memory required for storing the Identifier fields with each date occurrence.
Memory required = B * (C + D) + A * (320) = 18000000 * (322 + 80) + 19000000 * 320 = 7,236,000,000 + 6,080,000,000 = 13,316,000,000
 Step 2 : Calculate the memory required for storing the aggregate (rollup) fields.
Memory required = B * (C + D + E1 + F1 +E2 + F2 + E3 + F3 + ...En + Fn)
                = 18000000(322+80 +96+8 + 96+8 + 108+8 +100+278 + 88+122)
                = 18000000(402    + 104  + 104   + 116     +378     +210)
                = 18000000(1210)
                = 21,780,000,000
In addition, an extra 500MB is required for other processing.
Step 1 + Step 2 + 500MB = 13,316,000,000 + 21,780,000,000 + 500,000 = 35,096,500,000
This is approximately 35 GB.