Integrating with Datadog (outbound)

The outbound integration allows you to pull queries per second (QPS) and usage statistics from IBM® NS1 Connect® to report and visualize the data in Datadog.

Note: NS1 Connect supports ​two​ integrations with Datadog:
  • The inbound integration pulls data from Datadog as a third-party data source used for monitoring purposes.
  • The outbound integration pushes data from NS1 Connect to Datadog for reporting purposes.
The following instructions describe configuring the outbound integration to Datadog for reporting purposes. Refer to Configuring Datadog as a third-party data source for instructions to configure the inbound integration.

Located in the Datadogintegrations-extras repository​​, this integration allows you to view QPS and usage statistics from NS1 Connect in Datadog dashboards​.

Before you begin

  • You must have a Datadog account with the ​Datadog agent installed.​​
    Note: The minimum agent version supported is v7.21.0. However, it is recommended that you use the latest version of the Datadog agent and only have one agent running when using this integration.
  • You must ​have a valid NS1 Connect API key​​ with the Manage invoices and Manage zones, View zones, and Allow by default permissions enabled.

Step 1: Install the NS1 Connect integration on the Datadog platform

  1. Log in to Datadog and select ​Integrations > Integrations.
  2. Search and select the NS1 integration tile.
  3. Click Configuration.
  4. Click Install Integration.
  5. Once complete, verify the dashboard connection by clicking Dashboards > Dashboard List. The preset dashboard NS1 Screenboard opens in the list.
    Note: Validate the integration by going to the Infrastructure page. Your machine should appear in the host list if the agent is installed and configured.
  6. Click ​Integrations > Integrations.
  7. Click the NS1 integration tile.
  8. Follow the instructions from Datadog Community integration installation documentation to install the integrations with NS1 Connect on the Datadog agent. When running the installation, use the following variables which are specific to this integration.
    • <INTEGRATION_NAME>:NS1
    • <INTEGRATION_VERSION>:0.0.4

Step 2: Edit the YAML configuration file

In the configuration file, edit the parameters to specify what data Datadog should collect from NS1 Connect. After implementation, you can view this data in the Datadog UI.

Note: The YAML configuration file is saved automatically upon downloading the integration.
  1. Navigate to the YAML file (/etc/datadog-agent/conf.d/ns1.d).
  2. Rename the default conf.yaml.example file to conf.yaml and edit the file to indicate the desired metrics to gather.
  3. Save the configuration file.
  4. Restart the agent.
Attention: The YAML file has strict formatting rules. Use of tabs to indent is not allowed. Each indent must be exactly two spaces. Elements arranged in a list must be preceded with -, which must be indented exactly two spaces.

The following example file and parameters can be used for reference when editing the configuration file. Refer to the API documentation to learn more about the supported endpoints, requests, and parameters.

## All options defined here are available to all instances.
#
init_config:

    ## @param service - string - optional
    ## Attach the tag `service: <SERVICE>` to every metric, event, and service check emitted by this integration.
    ##
    ## Additionally, this sets the default `service` for every log source.
    #
    # service: <SERVICE>

## Every instance is scheduled independent of the others.
#
instances:

    ## @param api_endpoint - string - required
    ## The URL to check.
    #
  - api_endpoint: https://example.org

    ## @param api_key - string - required
    ## The api authentication key.
    #
    api_key: test_key_123

    ## @param networks - list - optional
    ## If present, usage stats are queried and reported by the network.
    ## Each network’s stats are tagged with network name.
    ## If omitted, usage stats are reported only account wide.
    ## If a list of network IDs is supplied, only those networks are queried.
    ## If the parameter is present without a list of network IDs,
    ## all available networks are queried.
    #
    # networks:
    #   - 0
    #   - 5
    #   - 80

    ## @param metrics - dictionary - required
    ## List of metrics to report to Datadog
    ## Each individual entry within the metric dictionary is optional,
    ## however at least one has to be present
    ## i.e. you must specify at least one of the following:
    ## qps, usage, account, ddi, pulsar, pulsar_by_app, pulsar_by_record
    #
    metrics:
    ## @param pulsar - boolean - optional
    ## if parameter is present, the following is reported account-wide:
    ## number of decisions, number of undetermined decisions, route map hits and misses
    #
    #   pulsar:

    ## @param pulsar_by_record - list - optional
    ## if parameter is present, the following is reported on per record basis:
    ## number of decisions, route map hits and misses
    ## enter list of dns records in form:
    ## - <record name>: <record type>
    #
    #   pulsar_by_record:        
    #     - www.test.com: A
    #     - www.test1.com: A

    ## @param pulsar_by_app - list - optional
    ## if parameter is present, aggregate performance data
    ## and pulsar availability data is reported.
    ## Enter a list of pulsar application IDs and job IDs in form:
    ## - <app_id>: <job_id>
    #
    #   pulsar_by_app:        
    #     - 1xy4sn3: 1b1o94j
    #     - 1xy4sn3: 1uy3acz

    ## @param ddi - list - optional
    ## if parameter is present, lease statistics will be reported account-wide
    ## you can specify array of scope groups ID's to additionally 
    ## report leases for each scope
    #
    #   ddi:
    #     - 1
    #     - 2
      
    ## @param qps - list - optional
    ## (Queries per second)
    ## if parameter is present, QPS statistics will be reported account-wide
    ## you can specify an array of domain names to additionally 
    ## report qps for each domain separately
    ## if sub-array of records within domain is specified, 
    ## qps will be reported for each record separately
    #
    #   qps:
    #     - test.com:
    #       - www.test.com: A

    ## @param usage - list - optional
    ## (Number of queries)
    ## if parameter is present, usage statistics will be reported account-wide
    ## you can specify array of domain names to additionally 
    ## report usage for each domain separately
    ## if sub-array of records within domain is specified, 
    ## usage will be reported for each record separately
    #
    #   usage:
    #     - test.com:
    #       - www.test.com: A

    ## @param account - list - optional
    ## (billing and ttl)
    ## if 'billing' element is present, account query limit and current number of queries
    ## will be reported.
    ## If ttl element is present, TTL will be reported for each of the domains
    ## specified in sub-array
    #
    #   account:
    #     - billing:
    #     - ttl:
    #       - test.com
    #       - test1.com

    ## @param query_parameters - dictionary - optional
    ## Pulsar API endpoints can have query string parameters 
    ## For pulsar, period should be at least 1 hour
    #
    # query_params:
    #   pulsar_period: 1h
    #   pulsar_geo: "*"
    #   pulsar_asn: "*"
    #   pulsar_agg: avg

    ## @param min_collection_interval - integer - optional - default: 15
    ## This changes the collection interval of the check. For more information, see:
    ## https://docs.datadoghq.com/developers/write_agent_check/#collection-interval
    #
    # min_collection_interval: 60

    ## @param empty_default_hostname - boolean - optional - default: false
    ## This forces the check to send metrics with no hostname.
    ##
    ## This is useful for cluster-level checks.
    #
    # empty_default_hostname: false

Instances parameters

Configure the following parameters to determine which metrics will be collected for the agent instance.

Table 1. Instances parameters
Parameter Type Description
api_endpoint string The host URL for the NS1 Connect API: https://api.nsone.net
api_key string A valid NS1 Connect API key
metrics dictionary A list of metrics to report to Datadog. At least one metric must be present: qps, usage, account, pulsar, pulsar_by_app, pulsar_by_record.
Table 2. Metrics parameters
Parameter Type Description
account list If set, specify billing, ttl, or both. If billing is set, the account query limit and current number of queries will be supported. If ttl is set, the time-to-live value will be reported for each of the domains specified in a sub-array. For example:
    #   account:
    #     - billing:
    #     - ttl:
    #       - test.com
    #       - test1.com
pulsar boolean If true, this parameter reports the total number of decisions, number of undetermined decisions, route map fits, and route map misses.
pulsar_by_app list If set, aggregate performance data and RUM-based availability data will be reported. Specify a list of application IDs and job IDs in the following format: ​<app_id>: <job_id>​​ If set, this parameter reports the number of decisions, route map hits/misses per application.
pulsar_by_record list Lisa of DNS records in the format <recordName>:<recordType>. If set, this parameter reports the number of decisions, route map hits/misses per record.
qps dictionary If set, returns queries per second (QPS) account-wide. Optionally, you can specify a list of domain names to report QPS for each zone separately. If a sub-array of records within a domain is specified, QPS will be reported for each record separately. For example, the following configuration will collect QPS for the entire account, the test.com​ zone, and the www.test.com A record.
  qps:
      - test.com:
         - www: A
query_parameters dictionary List of key-value pairs (strings) applicable to usage and RUM-based API endpoints.
usage list If set, returns usage statistics account wide. Similar to QPS, you can specify an array of domain names to report usage for each, individually. You can also specify a sub array of records in that domain to view record-level statistics.
  usage:
      - test.com:
         - www: A
Table 3. Query parameters
Parameter Type Description
pulsar_asn string (enum) Specifies an Autonomous System Number (ASN) by which to filter
pulsar_geo string (enum) Specifies a location such as US, FR, GR, JP, US_NY, and so on
pulsar_period string (enum) Specifies a period for querying Pulsar endpoints (default 1 hour), applicable only to pulsar_by_app
usage_networks string (enum) Supports networks parameter and allows you to filter results for a specific network

Step 3: Validate the integration

Run the agent's status subcommand and look for NS1 under the Checks header.

Step 4: View data on the Datadog dashboard

You can view the preconfigured Datadog dashboard, called NS1 Screenboard. The various charts display the data specified in the configuration file. Note the metrics are prefixed with ns1. in the UI, whereas this prefix does not appear in the configuration file.

The following table aligns the metrics shown in the dashboard with the relevant NS1 Connect API endpoint.

Table 4. Queries per second (QPS)
Datadog metric name Dashboard charts API endpoint
ns1.qps
  • Average QPS over the last month with trend lines indicating hour, day, current week, previous week
  • Average QPS in the past day
/v1/stats/qps
ns1.qps.zone
  • Average QPS - Top 10 zones
  • Average QPS - Top zone values and changes
/v1/stats/qps/<domain>
ns1.qps.record
  • Average QPS - Top 10 records
  • QPS - Top records values and changes
/v1/stats/qps/<domain>/<record>/<recordtype>
Table 5. Usage
Datadog metric name Dashboard charts API endpoint
ns1.usage
  • DNS - Queries past 30 days
  • DNS - Queries past 24h
  • DNS - Queries past hour
  • DNS - Usage weekly past year
  • Queries per day over past month (with trend lines)
  • Queries over past day
  • Usage - Top networks
/v1/stats/usage
ns1.usage.zone
  • Queries per zone - Top 10 zones
  • Usage - Top 10 zones
  • Usage - Top zone values and changes
/v1/stats/usage/<domain>
ns1.usage.record
  • Queries per record - Top 10 records
  • Usage - Top 10 records
  • Usage - Top records values and changes
/v1/stats/usage/<domain>/<recordType>
Table 6.
Datadog metric name Dashboard charts API endpoint
ns1.ttl
  • TTL - Top records values and changes
/v1/zones/<domain>
ns1.billing
  • DNS - Usage weekly past year
  • Billing - Queries monthly limit
  • Billing - Queries current billing period
  • Billing - Queries limit vs. Actual past quarter
/v1/account/billataglance
Table 7.
Datadog metric name Dashboard charts API endpoint
ns1.pulsar.decisions
  • Pulsar - Total decisions past month
  • Pulsar - Total decisions past hour
  • Pulsar - Total decisions
  • Pulsar - Top 10 decisions by resources account-wide
/v1/pulsar/query/decisions
ns1.pulsar.decisions.insufficient /v1/pulsar/query/decision/​customer/undetermineds
ns1.pulsar.routemap.hit /v1/pulsar/query/routemap/​hit/<customerID>
ns1.pulsar.routemap.miss /v1/pulsar/query/routemap/​miss/<customerID>
Table 8. RUM (Pulsar) application
Datadog metric name Dashboard charts API endpoint
ns1.pulsar.performance
  • Pulsar - Top 10 performances by app and resource
  • Pulsar - Top 10 availability by app and resource
/v1/pulsar/apps/<app_id>/jobs/<job_id>/data
ns1.pulsar.availability
  • Pulsar - Top 10 availability by app and resources
/v1/pulsar/apps/<app_id>/jobs/<job_id>/availability
Table 9. RUM (Pulsar) data by record
Datadog metric name Dashboard charts API endpoint
ns1.pulsar.decisions.record
  • Pulsar - Top 10 decisions by record
/v1/pulsar/query/decision/​record/<recordName>/<recordType>
ns1.pulsar.routemap.hit.record
  • Pulsar - Route map top 10 hits and misses by record
/v1/pulsar/query/routemap/hit/​record/<recordName>/<recordType>
ns1.pulsar.routemap.miss.record
  • Pulsar - Route map top 10 hits and misses by record
/v1/pulsar/query/routemap/miss/​record/<recordName>/<recordType>