Log files

Log files generally record Product Master runtime events, including exception traces and error messages and can help you resolve issues.

The Product Master log files are in the <install dir>/logs folder and can provide helpful debugging information.

For troubleshooting and analysis of your log files, you can use the Log and Trace Analyzer (LTA). For message troubleshooting and analysis, you can use the symptom catalog. All Product Master log file messages are collected and listed in the symptom catalog to create a centralized knowledge base of issues and resolutions. The symptom catalog uses the XML file format and contains specific message information to help you identify issues by providing the following information about the message:

Message ID numbers
Descriptions
Time stamps
Thread IDs
Severity levels
Possible resolutions
Source components

The symptom catalog file is named xxxx, where xxxx is the name of your catalog, and is located in the $TOP/locales/en_US/system_resource_bundle folder. For more information, see Using LTA tool.

Runtime log files

Each product service includes several different runtime-generated log files that record certain events within each of the services. The svc.out log file is a runtime log file that each service creates, and should be one of the first log files that you analyze when troubleshooting a problem. After starting a service, view the svc.out file for errors or exceptions to ensure that the service started correctly.

The svc.out log file is located in the following folders:

$TOP/logs/service/service_Name

Where,

service - Can be admin, appsvr, default, eventprocessor, queuemanager, rmi, scheduler, and workflowengine.
service_Name is the unique service name of a service.

Types of log files

The following table defines each type of log file that is under each service folder and refers Product Master as the 'Application'. To view the Logs folder structure, see the Fetching the Log folder structure section.

Table 1. Admin UI log files
Log file name	Contains
ehcache.log	Messages for EHCache third-party component.
gdsmsg.log	Errors, events, and messages for message processing from the gdsmsg component.
mdm_cache.log	Errors, events, and diagnostic information for the system cache.
svc.err	Runtime errors for each JVM. It is the standard error (STDERR) output.
svc.out	Runtime errors for each service created, and it should be one of the first log files that you analyze when troubleshooting a problem. After starting a service, view the svc.out file for errors or exceptions to ensure that the service started correctly. For more information, see Log files.

Table 2. Persona-based UI log files
Log file name	Contains
`derby_installation_logs`	Contains the Apache Derby database installation logs.
`engine.log`	Runtime errors for Dashboards OED engine.
`indexer.log`	Runtime errors for the Free text search services.
`installation.log`	Contains the Persona-based UIconfiguration and installation logs.
`pim_collector.log`	Runtime errors for the Free text search services.
attributes.log	Runtime errors for the machine learning attributes service APIs.
categorization.log	Runtime errors for the machine learning categorization service APIs.
controller.log	Runtime errors for the machine learning controller APIs.
standardization.log	Runtime errors for the machine learning standardization service APIs.
`restapi.log`	Runtime errors for the Product Master REST APIs.

Log file format

You can set the log file format of all of your logs files to either the log4j format or the Common Base Event (CBE) format.

All the messages by default, are logged by using the log4j PatternLayout format. The log4j format provides the Product Master logging infrastructure that outputs an easy to read format with one message per line in a .log file. The optional CBE log file format that is used with the Log and Trace Analyzer (LTA) produces messages in the XML file format and is more appropriate for machine reading.

You specify the log format on the log_format parameter in the common.properties file. Following are the valid values of the log_format parameter.

PatternLayout: The default log4j log file format.
CBELayout: The CBE log file format.
CBELayout_PatternLayout: Logs messages in both the log4j and the CBE log file formats.

Note: The CBE log file format creates a performance overhead and is recommended for use only when instructed by IBM Software Support, for example, when performing log analysis and troubleshooting with the LTA.

Customizing log4j log files

You can customize your log4j formatted log files including file location, size, and number of backup files.

Use the log formatting options to modify and manage your log4j log files. Edit the log4j2.xml file located in the $TOP/etc/default folder.

Generated file location

To change the location of a generated log file, change the File and filePattern parameters of the specified log configuration file.

Example

<RollingFile name="EXCEP" fileName="$TOP/logs/webserver_db.log" append="true" 
filePattern="/opt/MDM/logs/${svc_name}/exception-%d{MM-dd-yyyy}-%i.log">

Log file size

To control when the log file truncates,

To control when the file begins to truncate, change the Size parameter value in the SizeBasedTriggeringPolicy tag.

In this example, the maximum file size is 10MB.

<SizeBasedTriggeringPolicy Size="10MB"/>

Maximum backup files

To control the number of files you store, change the max parameter value.

Example

<DefaultRolloverStrategy max="2" />

Ignoring a particular exception through the log4j2.xml file

If you want to ignore a particular type of exception that is logged in the log files, then you need to add a filter in the log4j2.xml located in the $TOP/etc/default/ folder. If you add the filter, you can avoid warnings, and less severe exceptions from getting logged.

In the log4j2.xml file add following to filter out a particular exception:

<RegexFilter regex="(?s).*ExceptionName.*" onMatch="DENY" onMismatch="ACCEPT"/>

ExceptionName: Specify the name of exception that you want to avoid getting logged.
onMatch: Set the value DENY, so whenever the parameter finds an exception having particular exception name. The exception is not logged.
onMismatch: Set to the value ACCEPT, so whenever the parameter finds an exception not having particular exception name then only exception is logged.

<RollingFile name="EXCEP" fileName="<log_location>/logs/${svc_name}/ipm.log" append="true"
filePattern="/home/mdmpim/opt/IBM/MDM/logs/${svc_name}/ipm-%d{MM-dd-yyyy}-%i.log">
<RegexFilter regex="(?s).*ExceptionName.*" onMatch="DENY" onMismatch="ACCEPT"/>
<PatternLayout>
<ScriptPatternSelector defaultPattern="%d [%t] %-5p %c %x- %m%n">
<ScriptRef ref="decideLoggingPattern"/>
<PatternMatch key="showIpAddressAndUserName" pattern="%d ip_address=[%X{loggedInUserIp}] user=[%X{UserName}] [%t] %-5p %c %x- %m%n"/>
</ScriptPatternSelector>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="10 MB" />
</Policies>
<DefaultRolloverStrategy max="2" />
</RollingFile>

Fetching the log folder structure

Proceed as follows:

Stop the application.
Upgrade the application to the latest build.
Run the configureEnv.sh file, and overwrite the following files:
- log4j2.xml.bak
- log_cbe_xml.bak
- log_cbe_pattern_xml.bak

Take a backup of the existing Logs folder by using the following command:

cp $TOP/logs    <destination_directory>
OR 
cp $TOP/opt     <destination_directory>

Example

cp $TOP/logs   /bkp_folder

Delete the existing Logs folder by using the following command:
```
rm -rf $TOP/logs
OR
rm -rf $TOP/opt
```
Run the ./start_local.sh file. After the services are started, the folder structure resembles the list of the log files that are mentioned in the List of the log files section.