Topic
  • 4 replies
  • Latest Post - ‏2015-09-08T08:51:25Z by sree19
SystemAdmin
SystemAdmin
6056 Posts

Pinned topic Tivoli Log File Monitoring - Regex to exclude errors

‏2012-12-28T13:05:26Z |
Hi All,

We use Tivoli to monitor our logs files. The log4j log level is set to ERROR and Tivoli would raise tickets for these statements. But there are some known issues for which we would not want Tivoli to raise tickets. Is there a way to specify that some statements need to be ignored ?

Current regex : [/var/tmp/abc.log;ERROR(.*);error found: RegExp1]

This is very generic. We need to exclude certain framework errors (Hibernate / Mule) for a known issue. Is there a way to specify using a regex ?

Thanks,
Midhun
Updated on 2013-03-14T09:10:03Z at 2013-03-14T09:10:03Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    6056 Posts

    Re: Tivoli Log File Monitoring - Regex to exclude errors

    ‏2013-01-15T09:48:25Z  
    You can use a DISCARD statement within a Tivoli Log File Agent format file to explicitly prevent certain log entries from being matched. The DISCARD statement needs to be positioned after your generic match in the format file so that it is processed first. Page 30 in the Logfile Agent 6.2.3 FP2 User Guide mentions it briefly.
  • SystemAdmin
    SystemAdmin
    6056 Posts

    Re: Tivoli Log File Monitoring - Regex to exclude errors

    ‏2013-02-27T07:45:42Z  
    You can use a DISCARD statement within a Tivoli Log File Agent format file to explicitly prevent certain log entries from being matched. The DISCARD statement needs to be positioned after your generic match in the format file so that it is processed first. Page 30 in the Logfile Agent 6.2.3 FP2 User Guide mentions it briefly.
    Thanks a lot for suggesting DISCARD. Never knew that such a thing existed. :) I did go through the User Guide to get ready to implement it. But the guide had very little information in it. Can you please suggest the syntax to specify the DISCARD statement ?

    Midhun
  • SystemAdmin
    SystemAdmin
    6056 Posts

    Re: Tivoli Log File Monitoring - Regex to exclude errors

    ‏2013-03-14T09:10:03Z  
    Thanks a lot for suggesting DISCARD. Never knew that such a thing existed. :) I did go through the User Guide to get ready to implement it. But the guide had very little information in it. Can you please suggest the syntax to specify the DISCARD statement ?

    Midhun
    Here is an example for monitoring a HTTP server access log. As the DISCARD statement is at the bottom of the format file it is processed first. Successful requests (status 200) match the DISCARD regex and no further processing is performed. Without this, an attempt would be made to match each status 200 entry against the 502, 500 and 404 error regexs, significantly increasing the amount of work the agent had to do.

    
    REGEX HTTP_404 ^((?:[0-9]
    {1,3
    }\.)
    {3
    }[0-9]
    {1,3
    })\s(.+?)\s(.+?)\s(\[.+?\])\s(
    ".+?")\s404\s(.+?)\s(.+?) client $1 CustomSlot1 identd $2 user $3 time $4 CustomSlot2 request $5 CustomSlot3 status 404 CustomSlot4 size $6 unknown $7 msg PRINTF(
    "%s %s %s %s %s %s %s %s", client, identd, user, time, request, status, size, unknown) END   REGEX HTTP_500 ^((?:[0-9]
    {1,3
    }\.)
    {3
    }[0-9]
    {1,3
    })\s(.+?)\s(.+?)\s(\[.+?\])\s(
    ".+?")\s500\s(.+?)\s(.+?) client $1 CustomSlot1 identd $2 user $3 time $4 CustomSlot2 request $5 CustomSlot3 status 500 CustomSlot4 size $6 unknown $7 msg PRINTF(
    "%s %s %s %s %s %s %s %s", client, identd, user, time, request, status, size, unknown) END   REGEX HTTP_502 ^((?:[0-9]
    {1,3
    }\.)
    {3
    }[0-9]
    {1,3
    })\s(.+?)\s(.+?)\s(\[.+?\])\s(
    ".+?")\s502\s(.+?)\s(.+?) client $1 CustomSlot1 identd $2 user $3 time $4 CustomSlot2 request $5 CustomSlot3 status 502 CustomSlot4 size $6 unknown $7 msg PRINTF(
    "%s %s %s %s %s %s %s %s", client, identd, user, time, request, status, size, unknown) END   REGEX *DISCARD* ^((?:[0-9]
    {1,3
    }\.)
    {3
    }[0-9]
    {1,3
    })\s(.+?)\s(.+?)\s(\[.+?\])\s(
    ".+?")\s200\s(.+?)\s(.+?) END
    


    Hope that helps.

    Ant
  • sree19
    sree19
    1 Post

    Re: Tivoli Log File Monitoring - Regex to exclude errors

    ‏2015-09-08T08:51:25Z  

    Hi

     

    I am using tivoli log file agent to monitor the below log file . can anyone help me how i can create   regular expression fmt file for below log file

     

     Sep  7 14:27:24 BST 2015 itmtest1 itmtest  rg_move start itmtest 2 RELEASE db2inst1_ResourceGroup

     Sep  7 14:27:24 BST 2015 itmtest1 itmtest2  rg_move start itmtes2  2 RELEASE db2itmp1_ResourceGroup

     

    i have tried with below code but it is not working

    #..............
    REGEX ErrorLogs
    ^([A-Z][a-z]{2}[0-9]{1,2}:[0-9]{2}[0-9]{4}) ([0-9]) (\S+)(\S+) (\S+) ([0-9]+) (.*)
    day $1 CustomSlot1
    Month $2 CustomSlot2
    Date $3 Customslot3
    time $4 CustomSlot4
    time2 $5 CustomSlot5
    year $6 CustomSlot6
    clustername $7 CustomSlot7
    hostname $8 CustomSlot8
    description $9 CustomSlot9


    END
    #.............

     

     

    Regards,

    sree