Enabling partial query logs
DNS query logs detail all requests handled by the DNS nameservers, providing insight into DNS traffic, dead records, and growth analysis. On the IBM® NS1 Connect® platform, query logs are aggregated within time buckets (30 seconds) and emitted to a customer-defined S3 location (bucket and prefix). S3 objects are gzip-encoded JSONL (line-delimited JSON) where each line represents a single aggregation. The object keys are formatted with process times, whereas the logs are timestamped with the event time.
Granting access to the IBM NS1 Connect® platform
Follow the instructions provided in the AWS documentation to grant third-party access to your S3 location. Then, contact IBM support with the following information:
- S3 bucket name, region, and prefix. Note: The prefix should terminate with a forward slash (‘/’).
- Amazon Resource Name (ARN) of an AWS Identity and Access Management (IAM) role with the
following policies:
- At least
s3:PutObject
permissions for the above S3 location. - A trust policy granting NS1
sts:AssumeRole
permissions toarn:aws:iam::025043166333:role/ service-role/pipeline-querylogs-role-ukj3oed7
.
- At least
- An
sts:ExternalId
on which the above trust policy is conditioned (typically a UUID).
Refer to the example Terraform configuration for creating the proper IAM roles and policies.
Example data
Object prefixes are partitioned by year, month, day, and hour (in GMT).
s3://<customer_bucket>/<customer_prefix>dns.query.logs/2019/10/16/20/2019-10-16-20-46- 33.115951011.gz
{
“count”: 10,
“customer”: 12345,
“domain”: “foo.bar.com”,
“metric_name”: “dns.query.logs”,
“network”: “0”,
“rectype”: “A”,
“timestamp”: 1571250180,
“zone”: “bar.com”
}
Parameter | Description |
count |
The number of times this record was queried within the aggregation window (30 seconds). |
customer |
The IBM NS1 Connect® account ID. |
domain |
The queried record. |
metric_name |
dns.queries.logs is the only included data set at this time. |
network |
The unique network identifier for customers with dedicated networks (default is 0). |
rectype |
The type of DNS record queried. |
timestamp |
The query event time. |
zone |
The encompassing DNS zone for the queried record. |
If you are using Terraform to manage your AWS resources, copy and paste the code below to apply the configuration via Terraform.
variable "ns1_querylogs_s3_bucket" {
type = string
description = "The name of the destination bucket for NS1 query log objects."
}
variable "ns1_querylogs_s3_prefix" {
type = string
description = "The s3 prefix to prepend to all NS1 query log objects. Omit leading slash. Include trailing slash."
}
variable "ns1_querylogs_external_id" {
type = string
description = "An agreed-upon value for assuming external IAM roles (typically a UUID): https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html"
}
resource "aws_iam_role" "ns1_querylogs" {
name = "ns1-querylogs-role"
description = "The role that NS1 assumes to send query logs logs to this AWS account."
path = "/"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::025043166333:role/service-role/pipeline-querylogs-role-ukj3oed7"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "${var.ns1_querylogs_external_id}"
}
}
}
]
}
EOF
}
resource "aws_iam_policy" "ns1_querylogs" {
name = "ns1-querylogs-policy"
description = "Allows s3 objects to be put to a specific bucket and prefix."
path = "/"
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::${var.ns1_querylogs_s3_bucket}/${var.ns1_querylogs_s3_prefix}*"
}
]
}
EOF
}
resource "aws_iam_role_policy_attachment" "ns1_querylogs" {
role = aws_iam_role.ns1_querylogs.name
policy_arn = aws_iam_policy.ns1_querylogs.arn
}