Beaconing RUM data via gRPC or HTTP

Note: Private RUM data collection is available only to IBM® NS1 Connect® accounts with the RUM-based (Pulsar) traffic steering solution. Contact IBM support to learn more.

If you are already collecting RUM data from your endpoints, you can beacon the telemetry to the IBM NS1 Connect® platform via API. This method allows you to "bring your own" (BYO) data from your CDN footprint or any combination of servers, data centers, or cloud providers. Beaconed data must contain a performance-based measurement, such as latency, to reference when routing.

Before you begin

Consult with IBM support or your dedicated support engineer for guidance on the best data collection method(s) for your organization and to overview the configuration process.

Step 1 - Create the RUM (Pulsar) application and jobs

A RUM (Pulsar) application represents a group of resources or assets from which the IBM NS1 Connect® platform ingests RUM data. Each application is identifiable by its unique application ID (app ID). The IBM NS1 Connect® platform references the app ID to determine which set of jobs should take the measurement. After configuring the application, you create a RUM (Pulsar) job for each monitored resource.

Note: If using the API to configure RUM applications and jobs, you must associate each job with the respective application using the unique app ID.

Step 2 - Configure beacon data via gRPC or HTTP

If your application is already gathering performance data, you can push your telemetry to IBM NS1 Connect® via beacon requests to inform routing decisions. Data can be beaconed in bulk via gRPC or HTTP. One payload can support multiple RUM applications and jobs (up to a 4 MB limit if using gRPC). Refer to https://github.com/ns1/pulsar-rum for details.

In general, for each endpoint from which you want to push data to NS1, you must provide the eyeball IP or geographic data (refer to this topic for details), the job ID corresponding to that endpoint, and some unit of measurement for the performance metric. Note that the metric you provide must be measured similarly for each endpoint.

Note: Some fields in the beacon message format are for future compatibility only and are not actively used in routing. Specifically, the DeviceType and DetailedPerformance fields are not used at this time. While it is possible to set DetailedPerformance instead of SimpleLatency, the ingestion endpoint currently uses TTFB or TTLB (whichever is present) and treats it as a SimpleLatency value. If no SimpleLatency, DetailedPeformance.TTFB or DetailedPerformance.TTLB value is returned, the data point is discarded. Contact IBM support to learn more about alternatives for generating leads for multi-metric data ingestion.

Method 1 - Send bulk data to the IBM NS1 Connect® platform via gRPC

You can send bulk data via gRPC to g.ns1p.net:443. Bulk ingestion via gRPC requires authentication using an API key generated from your NS1 account. Refer to https://github.com/ns1/pulsar-rum#bulk-beacons for implementation details. Details on how to implement a gRPC client are found at https://grpc.io.

Method 2 - Send bulk data to the IBM NS1 Connect® platform via HTTP

Alternatively, you can use HTTP by issuing a PUT request to b.ns1p.net:443. Instead of serializing a protobuf message, the JSON of the same data should be included in the POST body. Specifically, the POST body JSON should be able to be unmarshalled by protobuf into a Beacons object as defined in bulkbeacon.proto. Therefore, the same message sent in the gRPC example client linked above would be sent via HTTP as follows:

curl -X POST -H "X-NSONE-Key: $NSONE_API_KEY" -d '
{
  "beacons": [
    {
      "appid": "appid",
      "measurements": [
        {
          "attribution": {
            "jobid": "jobid1",
            "location": {
              "ipAddress": "72.89.27.210",
              "geoCountry": "FR"
            },
            "deviceType": "DESKTOP"
          },
          "payloads": [
            {
              "statusCode": 200,
              "simple": {
                "valueMs": 50
              }
            },
            {
              "statusCode": 200,
              "simple": {
                "valueMs": 65
              }
            },
            {
              "statusCode": 404,
              "detailed": {
                "timeToFirstByteMs": 60
              }
            },
            {
              "statusCode": 200,
              "attribution": {
                "jobid": "jobid2",
                "location": {
                  "ipAddress": "72.89.27.210",
                  "geoCountry": "US",
                  "geoSubdiv": "NY",
                  "asn": 701
                },
                "deviceType": "DESKTOP"
              },
              "detailed": {
                "timeToFirstByteMs": 60,
                "timeToLastByteMs": 100
              }
            },
            {
              "statusCode": 200,
              "attribution": {
                "jobid": "jobid3",
                "location": {
                  "ipAddress": "24.188.193.7"
                },
                "deviceType": "DESKTOP"
              },
              "detailed": {
                "timeToLastByteMs": 100
              }
            }
          ]
        }
      ]
    }
  ]
}' 'b.ns1p.net/v1/beacon/bulk'

Next steps

After creating a RUM application and the corresponding jobs, you can connect the job directly to a DNS answer like a monitor or third-party data feed. For instructions, refer to Connecting a monitor or data source to a DNS answer.