Topic
  • 5 replies
  • Latest Post - ‏2014-06-12T18:32:30Z by JonathanPechtaIBM
DJey
DJey
14 Posts

Pinned topic Some events are not received by qradar from HP-ELO

‏2014-06-12T10:03:11Z |

Dear All;


I have integrated HP-ELO with Qradar. I am receiving the logs from the HP-ELO.
I am receiving all logs from HP-ELO except below two logs.When i saw the HP-ELO  system log i can  see this information. But Qradar UI these 2 events are not there.
 
jun 8 14:11:41 OA: Management processor on Blade 3 appears unresponsive

jun 3 14:15:02 OA: Server Blade in bay 5 has been powered down.

Please advice me why i didnt get these 2 events in Qradar UI

Thanks

John

  • JonathanPechtaIBM
    JonathanPechtaIBM
    197 Posts

    Re: Some events are not received by qradar from HP-ELO

    ‏2014-06-12T12:00:02Z  

    John,

     

    The events are going to be collected by QRadar, however, since HP-ELO does not have an official DSM to support its events, they are likely not being parsed properly by the system.

     

    Events parsed by QRadar are done so by what we call Device Support Modules (DSMs). You can think of DSMs as software plug-ins that are responsible for understanding and parsing events provided by an event source. An event source can be a security appliance, server, operating system, firewall, database..really any type of system that generates an event when an action occurs. The DSM Configuration Guide contains a list of product manufacturers and the DSMs we have officially tested and validated against specific products. It does not appears that we support HP-ELO (iLO) as an official DSM.

     

    Not having an official DSM does not mean that we do not collect events, just that the event received by QRadar could be identified an "Uknown" on the Log Activity tab of QRadar. Unknown simple means that we saw and collected the event, however, we are unable to understand the event format to parse and categorize the event. Depending on your appliance type, for example, a server running Linux, your HP-ELO events might be discovered under our Linux OS DSM or might be discovered under our HP-UX DSM. However, there are likely some unique events in HP-ELO that cannot be parsed or identified if they do not follow an event format that we expect to see. When an event cannot be understood by the system, such as the two you listed, they are categorized as "Unknown".

     

    What is the difference between an unknown event and a stored event?

    Events can be thought of in three different categorizes:

    1. Parsed events - QRadar collects, parses, and categorizes the event to the proper log source.
    2. Unknown events - The event is collected and parsed, but cannot be mapped/categorized to a specific log source. When this occurs, we set the the Event Name and the Low Level Category as Unknown. Log sources that do not automatically discover are typically identified as Unknown Event Log until a log source is manually created in the system. When an event cannot be associated to a log source, then we assign to event to a generic log source. So you can identify them be doing a search for events associated with the "SIM Generic" log source or using the Event is Unparsed filter.
    3. Stored events - The event cannot be understood or parsed by QRadar. When we cannot parse an event, we write the event to disk and categorize the event as Stored.

     

    How do I find these in the Log Activity tab?

    • To find events specific to your HP-ELO device, you can do a search in QRadar for the "Source IP" address of the HP-ELO device or attempt to select a unique value from the event payload and do a search for "Payload Contains". One of these searches should locate your event and it is likely either categorized as unknown or stored.
       
    • The easiest way to locate unknown events or stored events is to add a search filter for "Event in Unparsed". This search filter locates all events that either cannot be parsed (stored) or events that could not be associated with a log source or auto discovered (Unknown Log Event).

     

    For more information on DSMs that we officially support, see the DSM Configuration Guide (PDF, new window)

     

    What do I do if my product version or device is not listed in the DSM Configuration Guide?

     

    • My version is not listed - The index at the back of the document should list the supported versions. If the DSM is for a product that is officially supported by QRadar, but the version is out-of-date, you might just need a DSM update to resolve any parsing issues. Your device version might be newer than listed in the DSM Configuration Guide, but that does not mean we do not support that version. We list product versions in the back of the DSM guide that we have tested in house. Software updates by vendors might add or change the event format for a specific DSM. In these cases, a support ticket can be opened for us to review the log source.
       
    • My device is not listed - When a device is not officially supported, customers have two options:
      1. Open a request for enhancement (RFE) to have your device officially supported.
      • Link to the QRadar SIEM RFE page: https://ibm.biz/BdRPx5 (new window, shortened URL)
      • Log in to the support portal page.
      • Click the Submit tab and fill in the required information.
        Note: If you have event logs from a device, it is helpful to attached the event information and include the product version of the device that generated the event log.
         

    2. Write a log source extension to parse events for your device.

     

    Hope this helps..

     

     

    -----

    Our first support webcast is announced for June 18th @ 11am EST (Webcast details: http://bit.ly/1wUIT32).
    Vote on topics you are interested in for future webcasts. See our anonymous survey https://www.surveymonkey.com/s/QRadarOpenmic.

     

     

    *Edit: I included two example screen captures from my Console. One image is what you might see when you apply the "Event is Unparsed" filter. The second screen capture is filtering by the SIM Generic log source to find unknown events.

     

  • DJey
    DJey
    14 Posts

    Re: Some events are not received by qradar from HP-ELO

    ‏2014-06-12T12:34:24Z  

    John,

     

    The events are going to be collected by QRadar, however, since HP-ELO does not have an official DSM to support its events, they are likely not being parsed properly by the system.

     

    Events parsed by QRadar are done so by what we call Device Support Modules (DSMs). You can think of DSMs as software plug-ins that are responsible for understanding and parsing events provided by an event source. An event source can be a security appliance, server, operating system, firewall, database..really any type of system that generates an event when an action occurs. The DSM Configuration Guide contains a list of product manufacturers and the DSMs we have officially tested and validated against specific products. It does not appears that we support HP-ELO (iLO) as an official DSM.

     

    Not having an official DSM does not mean that we do not collect events, just that the event received by QRadar could be identified an "Uknown" on the Log Activity tab of QRadar. Unknown simple means that we saw and collected the event, however, we are unable to understand the event format to parse and categorize the event. Depending on your appliance type, for example, a server running Linux, your HP-ELO events might be discovered under our Linux OS DSM or might be discovered under our HP-UX DSM. However, there are likely some unique events in HP-ELO that cannot be parsed or identified if they do not follow an event format that we expect to see. When an event cannot be understood by the system, such as the two you listed, they are categorized as "Unknown".

     

    What is the difference between an unknown event and a stored event?

    Events can be thought of in three different categorizes:

    1. Parsed events - QRadar collects, parses, and categorizes the event to the proper log source.
    2. Unknown events - The event is collected and parsed, but cannot be mapped/categorized to a specific log source. When this occurs, we set the the Event Name and the Low Level Category as Unknown. Log sources that do not automatically discover are typically identified as Unknown Event Log until a log source is manually created in the system. When an event cannot be associated to a log source, then we assign to event to a generic log source. So you can identify them be doing a search for events associated with the "SIM Generic" log source or using the Event is Unparsed filter.
    3. Stored events - The event cannot be understood or parsed by QRadar. When we cannot parse an event, we write the event to disk and categorize the event as Stored.

     

    How do I find these in the Log Activity tab?

    • To find events specific to your HP-ELO device, you can do a search in QRadar for the "Source IP" address of the HP-ELO device or attempt to select a unique value from the event payload and do a search for "Payload Contains". One of these searches should locate your event and it is likely either categorized as unknown or stored.
       
    • The easiest way to locate unknown events or stored events is to add a search filter for "Event in Unparsed". This search filter locates all events that either cannot be parsed (stored) or events that could not be associated with a log source or auto discovered (Unknown Log Event).

     

    For more information on DSMs that we officially support, see the DSM Configuration Guide (PDF, new window)

     

    What do I do if my product version or device is not listed in the DSM Configuration Guide?

     

    • My version is not listed - The index at the back of the document should list the supported versions. If the DSM is for a product that is officially supported by QRadar, but the version is out-of-date, you might just need a DSM update to resolve any parsing issues. Your device version might be newer than listed in the DSM Configuration Guide, but that does not mean we do not support that version. We list product versions in the back of the DSM guide that we have tested in house. Software updates by vendors might add or change the event format for a specific DSM. In these cases, a support ticket can be opened for us to review the log source.
       
    • My device is not listed - When a device is not officially supported, customers have two options:
      1. Open a request for enhancement (RFE) to have your device officially supported.
      • Link to the QRadar SIEM RFE page: https://ibm.biz/BdRPx5 (new window, shortened URL)
      • Log in to the support portal page.
      • Click the Submit tab and fill in the required information.
        Note: If you have event logs from a device, it is helpful to attached the event information and include the product version of the device that generated the event log.
         

    2. Write a log source extension to parse events for your device.

     

    Hope this helps..

     

     

    -----

    Our first support webcast is announced for June 18th @ 11am EST (Webcast details: http://bit.ly/1wUIT32).
    Vote on topics you are interested in for future webcasts. See our anonymous survey https://www.surveymonkey.com/s/QRadarOpenmic.

     

     

    *Edit: I included two example screen captures from my Console. One image is what you might see when you apply the "Event is Unparsed" filter. The second screen capture is filtering by the SIM Generic log source to find unknown events.

     

    Dear Jonathan,

    I have created universal DSM for HP-ELO 10 days back. Events are parsing correctly. I have received all events from HP-ELO system log except these two events

    jun 8 14:11:41 OA: Management processor on Blade 3 appears unresponsive

    jun 3 14:15:02 OA: Server Blade in bay 5 has been powered down.

    I have seen this two events in the HP-ELO system log. But i dint see this information in Qrdar UI.

    Apart from these two events it is working fine.

    Please advice.

    Thanks

     

  • JonathanPechtaIBM
    JonathanPechtaIBM
    197 Posts

    Re: Some events are not received by qradar from HP-ELO

    ‏2014-06-12T14:30:52Z  
    • DJey
    • ‏2014-06-12T12:34:24Z

    Dear Jonathan,

    I have created universal DSM for HP-ELO 10 days back. Events are parsing correctly. I have received all events from HP-ELO system log except these two events

    jun 8 14:11:41 OA: Management processor on Blade 3 appears unresponsive

    jun 3 14:15:02 OA: Server Blade in bay 5 has been powered down.

    I have seen this two events in the HP-ELO system log. But i dint see this information in Qrdar UI.

    Apart from these two events it is working fine.

    Please advice.

    Thanks

     

    John,

     

    So, since you mentioned that this is a Universal DSM, you might turn off Event Coalescing for your log source. We tend to coalesce fairly heavily on uDSM log sources. So, the event might be there, but hidden under another event. Coalesced events are listed as Multiple (x) with x representing the number of events that are coalesced under the core event. By turning off coalescing, each event will be listed individually and it might help you spot the missing event.

     

    If you believe that QRadar is not receiving the event, then you should take a look at this article to review events coming in on the wire to QRadar:

    Using the command-line of QRadar to troubleshoot and event source -  http://www-01.ibm.com/support/docview.wss?uid=swg21674902

     

     

    Also, as I mentioned in my previous post you might attempt to do a search against a value specific to those two events with the Event Payload Contains filter.

     

    Hope this helps....

     

     

    -----

    Our first support webcast is announced for June 18th @ 11am EST (Webcast details: http://bit.ly/1wUIT32).
    Vote on topics you are interested in for future webcasts. See our anonymous survey https://www.surveymonkey.com/s/QRadarOpenmic.

     

  • DJey
    DJey
    14 Posts

    Re: Some events are not received by qradar from HP-ELO

    ‏2014-06-12T15:53:33Z  

    John,

     

    So, since you mentioned that this is a Universal DSM, you might turn off Event Coalescing for your log source. We tend to coalesce fairly heavily on uDSM log sources. So, the event might be there, but hidden under another event. Coalesced events are listed as Multiple (x) with x representing the number of events that are coalesced under the core event. By turning off coalescing, each event will be listed individually and it might help you spot the missing event.

     

    If you believe that QRadar is not receiving the event, then you should take a look at this article to review events coming in on the wire to QRadar:

    Using the command-line of QRadar to troubleshoot and event source -  http://www-01.ibm.com/support/docview.wss?uid=swg21674902

     

     

    Also, as I mentioned in my previous post you might attempt to do a search against a value specific to those two events with the Event Payload Contains filter.

     

    Hope this helps....

     

     

    -----

    Our first support webcast is announced for June 18th @ 11am EST (Webcast details: http://bit.ly/1wUIT32).
    Vote on topics you are interested in for future webcasts. See our anonymous survey https://www.surveymonkey.com/s/QRadarOpenmic.

     

    Dear Jonathan,

    I am aware of tcpdump. That is not my problem.

    But i accepted that event coalescing .Is there is a chance diffeerent events are coalesed. How qradar identify it is a same event or different event?

    Thanks

  • JonathanPechtaIBM
    JonathanPechtaIBM
    197 Posts

    Re: Some events are not received by qradar from HP-ELO

    ‏2014-06-12T18:32:30Z  
    • DJey
    • ‏2014-06-12T15:53:33Z

    Dear Jonathan,

    I am aware of tcpdump. That is not my problem.

    But i accepted that event coalescing .Is there is a chance diffeerent events are coalesed. How qradar identify it is a same event or different event?

    Thanks

    John,

     

    Yes, it is possible that two different events can be coalesced together. If an event is coalesced together is going to depend on how similar the event structure is and how much information is actually in the event payload to correlate from.

     

    When an Event Processor/Flow Processor coalesces data, the system keys off of specific fields, such as source IP address, destination IP address, destination port, username, or event ID. It looks at these key fields and analyzes the event to determine if coalescing should occur with prior events that have already been processed. If the events belong to the same log source and have enough matching criteria, then it is possible they could be coalesced together.  Coalescing takes place over 10 second windows, so depending on when the event occurs can also determine when coalescing occurs. As the counters that group events are reset after a 10 second interval.

     

    If you are not sure if the event is being coalesced, you can always turn it off temporarily and evaluate your incoming events to determine if coalescing is why you are not seeing the issue.

     

     

    Hope this helps....

     

     

    -----

    Our first support webcast is announced for June 18th @ 11am EST (Webcast details: http://bit.ly/1wUIT32).
    Vote on topics you are interested in for future webcasts. See our anonymous survey https://www.surveymonkey.com/s/QRadarOpenmic.