Data sources and feeds
Data sources and feeds are essential mechanisms to intelligent traffic steering. A data source can be an IBM® NS1 Connect® monitoring job, an external monitoring service, or a Pulsar real user monitoring (RUM) service. A data feed connects an individual monitoring job to the corresponding DNS answer(s) to enable automatic updates based on the status or conditions of the monitored endpoint.
After configuring your NS1 Connect monitoring jobs, third-party data sources and feeds, or Pulsar (RUM) jobs, you can use them to automatically update critical metadata for answers in your DNS records. Answer metadata provides information about the application endpoint or service that the answer represents. Metadata is automatically updated when connected to a monitoring job.
If a monitoring job detects a change to the status of an endpoint or as it collects new metrics, an update is pushed through the data feed, updating the connected answer metadata field in the DNS record. When a record is queried, the record's Filter Chain configuration determines which answer(s) to return to the requesting client. Each traffic steering filter uses a unique processing method to eliminate or rearrange answers based on some rule or behavior. Many traffic steering filters leverage a specific metadata field for each answer. For example, the Up traffic steering filter references the Up/Down metadata field to avoid routing clients to unavailable endpoints.
Data sources
You can configure multiple types of data sources in NS1 Connect to inform traffic steering decisions. All data sources use similar methods to ingest, process, and distribute the data to the edge.
Slight differences exist in the configuration and steps involved for data sources associated with monitoring jobs compared to data sources related to integrations (third-party or native NS1 Connect API).
To locate the source ID for an NS1 Connect monitor, select the Show Generated sources check box. The NS1 Connect monitoring service appears in the list.
This information is critical when configuring third-party integrations or using the NS1 Connect API to manage DNS resources and monitors.
NS1 Connect monitoring jobs support basic health checks for your application endpoints and services through DNS, HTTP/S, PING, and TCP. The number of NS1 Connect monitoring jobs that you can create depends on your account plan type. You can view your plan type and usage limits on the Usage page.
When you connect a monitoring job to the metadata of an answer, NS1 Connect automatically generates a data source object, nsone_monitoring, and a notifier list containing a data feed notifier. The data feed notifier is critical for configuring automatic updates from the monitor to any DNS answers that you connect it to. You can add more notifiers to the notifier list (_NS1_.notify_list.#) attached to a monitoring job to send monitoring alerts through email, Slack, or another notification channel.
By default, auto-generated data sources associated with monitors don't appear in the list of data sources () in NS1 Connect. To view auto-generated data sources, select the Show Generated sources checkbox. If using the NS1 Connect API, all data sources appear in response to a GET command to https://api.nsone.net/v1/data/sources.
Third-party monitoring services, including AppDynamics, Datadog, Pingdom, Catchpoint, AWS CloudWatch, and more, integrate with NS1 Connect. Once configured, data from your third-party monitoring service is sent to NS1 Connect, where it is used to inform DNS traffic steering decisions. You can configure an unlimited number of third-party data sources.
Supported monitoring integrations include the native NS1 Connect API and several third-party tools that you can use to monitor various aspects of your applications, databases, and infrastructure — such as active connections, load average, and more.
If your preferred monitoring service is not included in the supported integrations, you can generate an API webhook to push data from third-party services to NS1 Connect.
NS1 Connect customers with the Pulsar RUM-based traffic steering solution enabled on their account can leverage real user monitoring (RUM) data collected from shared or private resources to inform traffic steering decisions based on real-time conditions.
Refer to Getting started with RUM-based traffic steering (Pulsar) for details.
Data feeds
A data feed connects a data source (API, monitoring, or third-party integration) with a DNS record or to answer metadata. Record or answer metadata can include various metrics associated with the endpoint represented by the record or answer. You can input metadata manually or connect a data feed to a specific metadata field to update the metadata automatically as conditions change.
The Feeds tab on the Integrations page lists all active data sources and feeds and displays the associated feed URL. You must select the Show Generated sources checkbox to view auto-generated objects if you created a monitoring job in the NS1 Connect portal.
Each data feed has a unique feed URL in the following format: https://api.nsone.net/v1/feed/<sourceID>. Many data sources connect directly to this URL to feed data to NS1 Connect. The request sent to this URL depends on the type of data source. Typically, each data source has a unique request protocol.