Topic
  • 8 replies
  • Latest Post - ‏2014-01-27T14:21:04Z by NaipulO
dth0m
dth0m
5 Posts

Pinned topic Reference Set Creation

‏2013-10-08T17:33:53Z |

Is there any way that I can create a reference set from the command line in Qradar?  I have a txt file that is updated on a daily and is uploaded into a reference set.  I would like download the file (very easy) and then use that file to update an existing reference set from the linux CLI.  Deleting and recreating the reference set would also be an option.  I have been told that it is possible to create a reference set from the CLI.  Any input would be appreciated.

Thanks,

Derek Thomas

  • Nikodim
    Nikodim
    20 Posts

    Re: Reference Set Creation

    ‏2013-10-09T07:22:20Z  

    Derek,

    You can do this from a shell script:

    #!/bin/sh
    export CLASSPATH='/opt/qradar/jars/q1labs_qradarapi_common.jar:/opt/qradar/jars/q1labs_qradarapi_v100.jar:/opt/qradar/jars/q1labs_qradarapi_v101.jar:/opt/qradar/jars/args4j-2.0.8.jar:/opt/qradar/jars/q1labs_frameworks.jar:/opt/qradar/jars/log4j-1.2.8.jar'

    COMMAND="java com.q1labs.qradarapi.common.APIClient -d -c com.q1labs.qradarapi.v101.CommandLineClient"

    clear
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Calling HELP screen +++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -help
    # -? (-help, --help) : Print this help screen
    # -a (--args, -args) ARGS : The optional arguments to pass to the method.
    # -c (--class, -class) VAL : The API command line client class to use
    # -d (--debug, -debug) : Display debugging information
    # -h (--host, -host) VAL : The optional host to use to connect (default: loca
    # lhost)
    # -l (-list, --list-methods) : List the methods supported by this API client
    # -m (--method, -method) VAL : The method to execute
    # -t (--token, -token) VAL : The optional host token to use to authenticate (de
    # fault: /opt/qradar/conf/host.token contents)
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Creating New Reference Set ++++++++++++++++++++++++++
    echo + Options: AlphaNumeric, Numeric, IP, PORT +++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "createReferenceSet" -a Test1 10000 AlphaNumeric
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Populating new RefSet with new entries ++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "populateReferenceSet" -a Test1 {Administrator,Pupkin,root}
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Remove all entries from RefSet ++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "purgeReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Destroy RefSet ++++++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "destroyReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo +++++++++++++++++++ Have fun! ++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  • Jonathan.Pechta (IBM)
    80 Posts

    Re: Reference Set Creation

    ‏2013-10-09T10:22:14Z  
    • Nikodim
    • ‏2013-10-09T07:22:20Z

    Derek,

    You can do this from a shell script:

    #!/bin/sh
    export CLASSPATH='/opt/qradar/jars/q1labs_qradarapi_common.jar:/opt/qradar/jars/q1labs_qradarapi_v100.jar:/opt/qradar/jars/q1labs_qradarapi_v101.jar:/opt/qradar/jars/args4j-2.0.8.jar:/opt/qradar/jars/q1labs_frameworks.jar:/opt/qradar/jars/log4j-1.2.8.jar'

    COMMAND="java com.q1labs.qradarapi.common.APIClient -d -c com.q1labs.qradarapi.v101.CommandLineClient"

    clear
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Calling HELP screen +++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -help
    # -? (-help, --help) : Print this help screen
    # -a (--args, -args) ARGS : The optional arguments to pass to the method.
    # -c (--class, -class) VAL : The API command line client class to use
    # -d (--debug, -debug) : Display debugging information
    # -h (--host, -host) VAL : The optional host to use to connect (default: loca
    # lhost)
    # -l (-list, --list-methods) : List the methods supported by this API client
    # -m (--method, -method) VAL : The method to execute
    # -t (--token, -token) VAL : The optional host token to use to authenticate (de
    # fault: /opt/qradar/conf/host.token contents)
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Creating New Reference Set ++++++++++++++++++++++++++
    echo + Options: AlphaNumeric, Numeric, IP, PORT +++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "createReferenceSet" -a Test1 10000 AlphaNumeric
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Populating new RefSet with new entries ++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "populateReferenceSet" -a Test1 {Administrator,Pupkin,root}
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Remove all entries from RefSet ++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "purgeReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Destroy RefSet ++++++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "destroyReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo +++++++++++++++++++ Have fun! ++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    The supported method is to use ReferenceDataUtil.sh script in the comand-line.

    There is a technical note on how to use the script to create reference sets. Here is the information: Creating_Reference_Maps.pdf

  • Jonathan.Pechta (IBM)
    80 Posts

    Re: Reference Set Creation

    ‏2013-10-09T10:23:15Z  

    The supported method is to use ReferenceDataUtil.sh script in the comand-line.

    There is a technical note on how to use the script to create reference sets. Here is the information: Creating_Reference_Maps.pdf

  • dth0m
    dth0m
    5 Posts

    Re: Reference Set Creation

    ‏2013-10-10T14:16:32Z  
    • Nikodim
    • ‏2013-10-09T07:22:20Z

    Derek,

    You can do this from a shell script:

    #!/bin/sh
    export CLASSPATH='/opt/qradar/jars/q1labs_qradarapi_common.jar:/opt/qradar/jars/q1labs_qradarapi_v100.jar:/opt/qradar/jars/q1labs_qradarapi_v101.jar:/opt/qradar/jars/args4j-2.0.8.jar:/opt/qradar/jars/q1labs_frameworks.jar:/opt/qradar/jars/log4j-1.2.8.jar'

    COMMAND="java com.q1labs.qradarapi.common.APIClient -d -c com.q1labs.qradarapi.v101.CommandLineClient"

    clear
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Calling HELP screen +++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -help
    # -? (-help, --help) : Print this help screen
    # -a (--args, -args) ARGS : The optional arguments to pass to the method.
    # -c (--class, -class) VAL : The API command line client class to use
    # -d (--debug, -debug) : Display debugging information
    # -h (--host, -host) VAL : The optional host to use to connect (default: loca
    # lhost)
    # -l (-list, --list-methods) : List the methods supported by this API client
    # -m (--method, -method) VAL : The method to execute
    # -t (--token, -token) VAL : The optional host token to use to authenticate (de
    # fault: /opt/qradar/conf/host.token contents)
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Creating New Reference Set ++++++++++++++++++++++++++
    echo + Options: AlphaNumeric, Numeric, IP, PORT +++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "createReferenceSet" -a Test1 10000 AlphaNumeric
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Populating new RefSet with new entries ++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "populateReferenceSet" -a Test1 {Administrator,Pupkin,root}
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Remove all entries from RefSet ++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "purgeReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo ++++++++++ Destroy RefSet ++++++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    $COMMAND -m "destroyReferenceSet" -a Test1
    sleep 3
    clear

    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    echo +++++++++++++++++++ Have fun! ++++++++++++++++++++++++++++++++++
    echo ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Whoa, is there an API that I don't know about?  I have always wished we had access to an API.

  • dth0m
    dth0m
    5 Posts

    Re: Reference Set Creation

    ‏2013-10-10T14:19:49Z  

    The supported method is to use ReferenceDataUtil.sh script in the comand-line.

    There is a technical note on how to use the script to create reference sets. Here is the information: Creating_Reference_Maps.pdf

    Can you give me an example of creating a reference set using this?  I see different examples in the command line syntax and the Tech note:

    CLI Syntax

    create <name> <count> [ALN | NUM | IP | PORT | ALNic] [timeout_type] [timeToLive]

    Tech Note

    create <name> [MAP | MAPofSETS | MAPofMAPS] [timeout_type] [timeToLive] 

    What is the format of time to live, I do not see that anywhere, the interface lists DD:HH:MM:SS.  So far none of my attempts have successfully created the reference set.

     
  • dth0m
    dth0m
    5 Posts

    Re: Reference Set Creation

    ‏2013-10-10T15:08:13Z  
    • dth0m
    • ‏2013-10-10T14:19:49Z

    Can you give me an example of creating a reference set using this?  I see different examples in the command line syntax and the Tech note:

    CLI Syntax

    create <name> <count> [ALN | NUM | IP | PORT | ALNic] [timeout_type] [timeToLive]

    Tech Note

    create <name> [MAP | MAPofSETS | MAPofMAPS] [timeout_type] [timeToLive] 

    What is the format of time to live, I do not see that anywhere, the interface lists DD:HH:MM:SS.  So far none of my attempts have successfully created the reference set.

     

    I found the issue I think.  I can't get the option for Alphanumeric Ignore Case to work.  ALNic does not work as an option.

    ./ReferenceSetUtil.sh create Tessting6 6 ALN 0 works

    ./ReferenceSetUtil.sh create Tessting8 6 ALNic 0 does not

    ./ReferenceSetUtil.sh create Tessting8 6 ALNIC 0 does not

    ./ReferenceSetUtil.sh create Tessting8 6 alnic 0 does not

     

     

  • Nikodim
    Nikodim
    20 Posts

    Re: Reference Set Creation

    ‏2013-10-11T07:39:57Z  
    • dth0m
    • ‏2013-10-10T15:08:13Z

    I found the issue I think.  I can't get the option for Alphanumeric Ignore Case to work.  ALNic does not work as an option.

    ./ReferenceSetUtil.sh create Tessting6 6 ALN 0 works

    ./ReferenceSetUtil.sh create Tessting8 6 ALNic 0 does not

    ./ReferenceSetUtil.sh create Tessting8 6 ALNIC 0 does not

    ./ReferenceSetUtil.sh create Tessting8 6 alnic 0 does not

     

     

    ReferenceSetUtil

    QRadar 7.1: /opt/qradar/bin/ReferenceSetUtil.sh create UserSet 10000 ALNic 1 24:00:00

    QRadar 7.2: /opt/qradar/bin/ReferenceSetUtil.sh create UserSet 10000 ALNic 1 '5 hours'

    ReferenceDataUtil

    QRadar 7.1: /opt/qradar/bin/ReferenceDataUtil.sh create UserToIP_Map 10000 MAPofSETS ALN 1 24:00:00

    QRadar 7.2: /opt/qradar/bin/ReferenceDataUtil.sh create UserToIP_Map MAPofSETS 1 '24 hours'

    It seems size (count) RefData option was removed from QR 7.2. There's also a note: after script execution: "only alphanumeric maps are supported at the moment"

     

    Updated on 2013-10-11T07:40:50Z at 2013-10-11T07:40:50Z by Nikodim
  • NaipulO
    NaipulO
    2 Posts

    Re: Reference Set Creation

    ‏2014-01-27T14:21:04Z  

    The supported method is to use ReferenceDataUtil.sh script in the comand-line.

    There is a technical note on how to use the script to create reference sets. Here is the information: Creating_Reference_Maps.pdf

    Is there a method of finding the number of entries in a Reference Set when accessing the RS within a QRadar Rule ?

    Our customer has a requirement that their "Portal" requires a maximum number of active users per hour, else raise an offence.

    if the existing user is the RSet, then do not add.

    If the user is not in RSet and the count is < x, then add user to RSet