IBM Support

IV91288: OFFENSES CAN SOMETIMES STOP GENERATING WHEN OFFENSES ARE INDEXED ON CUSTOM PROPERTIES

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • It has been observed that QRadar Offense generation can
    sometimes stop when Offenses are indexed on Custom Properties
    that can cause some parsed values to be larger than 255
    characters.
    
    Messages similar to the following might be visible in
    /var/log/qradar.log when this issue is occurring:
    
    [ecs-ep] [MPC/PersisterThread@0000000030]
    com.q1labs.sem.magi.contrib.ModelPersister: [WARN]
    [NOT:0180002100][9.180.234.76/- -] [-/- -]Exception encounted
    when executing transaction 30.
    [ecs-ep] [MPC/PersisterThread@0000000030]
    com.q1labs.sem.magi.contrib.PersistenceException: Failed to
    persist sem model
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCurrentTransac
    tion(ModelPersister.java:642)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCommands(Model
    Persister.java:419)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.process(ModelPersiste
    r.java:261)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.TxStateManager.playCurrent(TxStateMa
    nager.java:259)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister$Persister.playCurrent
    (ModelPersister.java:2899)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister$Persister.run(ModelPe
    rsister.java:2855)
    [ecs-ep] [MPC/PersisterThread@0000000030] Caused by:
    [ecs-ep] [MPC/PersisterThread@0000000030]
    org.postgresql.util.PSQLException: ERROR: value too long for
    type character varying(255)
      Where: SQL statement "INSERT INTO offense_regex_property(id,
    parsed_value, offense_count, event_count, offense_type_id)
            values (nextId, value_p, 0, 0, offense_type_p)"
    PL/pgSQL function create_offense_regex_property(character
    varying,integer) line 9 at SQL statement
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(Qu
    eryExecutorImpl.java:2157)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExe
    cutorImpl.java:1886)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorIm
    pl.java:255)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc
    2Statement.java:555)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(Abs
    tractJdbc2Statement.java:417)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc
    2Statement.java:410)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.core.dao.sem.light.OffenseRegexProperty.create(Offens
    eRegexProperty.java:381)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.PersistenceContext.createRegexProper
    ty(PersistenceContext.java:5908)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.commands.offense.OffenseRegexPropert
    yCreateCommand.execute(OffenseRegexPropertyCreateCommand.java:58
    )
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.executeOffenseKeyComm
    ands(ModelPersister.java:2399)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.process(ModelPersiste
    r.java:1032)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCurrentTransac
    tion(ModelPersister.java:545)
    

Local fix

  • Change the Offense Index property to a non custom property and
    then perform a SIM Clean from the Admin tab -> Advanced drop
    down -> 'Clean Sim Model'
    Note :
    Soft Clean - Closes all offenses in the database. If you select
    the Soft Clean option, you can also select the Deactivate all
    offenses check box.
    Hard Clean - Purges all current and historical SIM data, which
    includes offenses, source IP addresses, and destination IP
    addresses.
    

Problem summary

  • It has been observed that QRadar Offense generation can
    sometimes stop when Offenses are indexed on Custom Properties
    that can cause some parsed values to be larger than 255
    characters.
    
    Messages similar to the following might be visible in
    /var/log/qradar.log when this issue is occurring:
    
    [ecs-ep] [MPC/PersisterThread@0000000030]
    com.q1labs.sem.magi.contrib.ModelPersister: [WARN]
    [NOT:0180002100][9.180.234.76/- -] [-/- -]Exception encounted
    when executing transaction 30.
    [ecs-ep] [MPC/PersisterThread@0000000030]
    com.q1labs.sem.magi.contrib.PersistenceException: Failed to
    persist sem model
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCurrentTransac
    tion(ModelPersister.java:642)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCommands(Model
    Persister.java:419)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.process(ModelPersiste
    r.java:261)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.TxStateManager.playCurrent(TxStateMa
    nager.java:259)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister$Persister.playCurrent
    (ModelPersister.java:2899)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister$Persister.run(ModelPe
    rsister.java:2855)
    [ecs-ep] [MPC/PersisterThread@0000000030] Caused by:
    [ecs-ep] [MPC/PersisterThread@0000000030]
    org.postgresql.util.PSQLException: ERROR: value too long for
    type character varying(255)
      Where: SQL statement "INSERT INTO offense_regex_property(id,
    parsed_value, offense_count, event_count, offense_type_id)
            values (nextId, value_p, 0, 0, offense_type_p)"
    PL/pgSQL function create_offense_regex_property(character
    varying,integer) line 9 at SQL statement
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(Qu
    eryExecutorImpl.java:2157)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExe
    cutorImpl.java:1886)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorIm
    pl.java:255)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc
    2Statement.java:555)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(Abs
    tractJdbc2Statement.java:417)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc
    2Statement.java:410)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.core.dao.sem.light.OffenseRegexProperty.create(Offens
    eRegexProperty.java:381)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.PersistenceContext.createRegexProper
    ty(PersistenceContext.java:5908)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.commands.offense.OffenseRegexPropert
    yCreateCommand.execute(OffenseRegexPropertyCreateCommand.java:58
    )
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.executeOffenseKeyComm
    ands(ModelPersister.java:2399)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.process(ModelPersiste
    r.java:1032)
    [ecs-ep] [MPC/PersisterThread@0000000030]    at
    com.q1labs.sem.magi.contrib.ModelPersister.processCurrentTransac
    tion(ModelPersister.java:545)
    

Problem conclusion

  • This issue was resolved with QRadar/QRM/QVM/QRIF 7.3.0
    

Temporary fix

Comments

APAR Information

  • APAR number

    IV91288

  • Reported component name

    QRADAR SOFTWARE

  • Reported component ID

    5725QRDSW

  • Reported release

    727

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2016-12-06

  • Closed date

    2017-04-11

  • Last modified date

    2017-04-11

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    QRADAR SOFTWARE

  • Fixed component ID

    5725QRDSW

Applicable component levels

  • R730 PSY

       UP

[{"Business Unit":{"code":"BU048","label":"IBM Software"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"727","Edition":"","Line of Business":{"code":"LOB24","label":"Security Software"}}]

Document Information

Modified date:
11 April 2017