Endpoints

Endpoints define the API of a service and provide a fine-grained view into what operations are exposed by the services. Every call to a service happens on a single endpoint. Every endpoint has a single, automatically discovered type: BATCH, SHELL, DATABASE, HTTP, MESSAGING, RPC. Like services, endpoints can be viewed through the lens of an application perspective.

The endpoint can be statically declared or can be based on call tags (in contrast to the service, which is typically determined using infrastructure tags, e.g. SpringBoot name). For example a combination of {call.http.method} {call.http.path} would be a typical endpoint of an HTTP service.

Another benefit for defining separate endpoints for a service is that, mainly with 'monoliths', services can include many different frameworks and technologies; HTTP/REST APIs, database access, message bus integration, etc. By creating at least endpoints per type of API, and possibly further breaking apart such as protocol details, metrics and statistics are captured per API type.

Instana automatically maps endpoints based on default rules. For more information, see Endpoint mapping.

Predefined rules

Instana automatically maps endpoints based on a set of predefined rules. The following rules are considered in the descending order. When a rule matches, the respective endpoint is created.

Rule Tags
Custom endpoint rule Tags defined by the user (see custom endpoint mapping)
User defined endpoint name via HTTP Header {call.http.header.x-instana-endpoint}
User defined endpoint name in sdk span data {call.tag.endpoint}
OpenTelemetry defined operation name {otel.operation}
Jaeger defined operation name {call.tag.jaeger.operation}
Zipkin defined operation name {call.tag.zipkin.operation}
SOAP Action HTTP Header {call.http.header.soapaction}
RPC with WSDL namespaces {call.rpc.method-1}
RPC {call.rpc.method}
GraphQL {call.graphql.operationType?} {call.graphql.operationName?}
Batch Quartz Type Instead of Hex Pattern {call.batch.jobType}
Batch Quartz Type Instead of Deadline {call.batch.jobType}
Batch Hex With Prefix {call.batch.job-1}
Batch Quartz Numeric Suffix {call.batch.job-1}
Batch {call.batch.job}
Prisma model {call.database.statement-1}
Prisma model fallback Prisma
JDBC parsed statement {call.database.statement-1}
JDBC stored procedure statement result {call.database.statement-1}
Neo4j parsed statement {call.database.statement-1}
Cassandra parsed statement {call.database.statement-1}
Cosmos non-sql command {call.database.schema-1}
ElasticSearch Multiple Indices {call.database.schema-1}
ElasticSearch Single Index {call.database.schema}
ElasticSearch Action {call.database.commandType}
ElasticSearch Miscellaneous API endpoints Miscellaneous API endpoints
MongoDB collection parsed from query {call.database.schema-1}.{call.database.statement-2}
MongoDB collection not parseable from query {call.database.schema-1}
MongoDB collection parsed from namespace {call.database.schema}
DynamoDB Table {call.database.schema}
Redis command {call.database.statement}
AWS S3 bucket {call.database.schema}
AWS S3 operation Miscellaneous API endpoints
GCS bucket {call.database.schema}
GCS operation Miscellaneous API endpoints
Azure Storage Container {call.database.schema}
Azure Storage Operation Miscellaneous API endpoints
Memcache / Memcached / EhCache operation {call.database.commandType}
Couchbase bucket {call.database.schema}
Data Grid Cache {call.database.schema}
Couchbase parsed statement {call.database.statement-1}
Aerospike statement {call.database.statement}
IMS DB with statement {call.database.statement}
IMS DB without statement IMS DB
Temporary messaging queues temporary queue
IBM MQ span using address {call.messaging.address-1}
IBM ACE span using address {call.messaging.address-1}
Messaging default exchange for AzureQueue {call.messaging.exchangeType}
Apache RocketMQ span using address {call.messaging.address-1}
Messaging generic {call.messaging.address}
Messaging default exchange for RabbitMQ default exchange
Temporary messaging queues temporary queue
z/OS Connect API {call.zcee.apiName}
HTTP with path template {call.http.method?} /{call.http.pathTemplate-1}
HTTP with route ID {call.http.routeId}
First HTTP Path Segment after the API version, if available {call.http.method?} {call.http.path-1}
HTTP Method with Root Path {call.http.method} /
HTTP Root Path {call.http.method?} /
Event Generic {call.event.source}
Get Shell Binary Name Without Arguments and the Script Name Without Arguments {call.shell.command-1} {call.shell.command-2}
Get Shell Binary Name Without Arguments {call.shell.command}
Function as a service with name and version {faas.functionname}:{faas.functionversion}
Function as a service with name only {faas.functionname}
Function as a service with identifier {cloud.id}
SDK.name {call.sdk.name}

Endpoint mapping

Endpoints are automatically mapped based on the endpoint type.

For example, Instana automatically groups HTTP endpoints based on the path template if accessible.

/hospital/1948/patient/291148
/hospital/728/patient/924892
/hospital/47/patient/25978
/hospital/108429/patient/1847

In the above example, the preceding endpoints, which are being served by the same application code and hence have a shared performance profile, will be grouped and reported together like so:

/hospital/{hid}/patient/{pid}

This is done automatically and no user steps are required for known frameworks.

When the path template is not available, the endpoints are mapped to the first path segment or can be configured as desired

Here are few examples when the path template is not available:

  • No path templates are used in the monitored service.
  • Tracer does not support path template.
  • Tracer does not have access to the path template information in some edge cases like authentication failure.
  • The destination service is not monitored and the exit span from the client doesn’t know the path templates.

If you are writing your own tracing code, see our Custom tracing best practices on how you can achieve the same results.

Customizing endpoint mapping

For each service, Instana allows to have one configuration which controls how the services endpoints are getting extracted. To access the configuration, navigate to a services dashboard, go to the endpoints tab and hit "Configure Endpoints".

Application Dependency Map Overview

Note that custom endpoint configurations are available for HTTP-based services only.

Custom endpoint rules

Instana comes with three default HTTP rules which are always part of the extraction chain:

  • Detected Framework: Extracts endpoints as specified in detected framework (if accessible).
  • /*: Extracts endpoints based on the first path parameter.
  • Unspecified: Calls that do not match a preceding rule are assigned to this endpoint.

To specify additional configuration, you can define multiple custom rules. To do so, access the custom endpoint configuration dialog from any HTTP service and hit "Add Custom HTTP Rule". The upcoming dialog offers you the ability to define a specific path (the actual rule) and multiple test cases to check if the defined path is working as expected. For the path you can use static segments like /api or /myShopEndpoint, path parameters like /{productId}, or match any segment with /*.

In the rule tester part of the dialog, you can define multiple test cases to validate rules. For example given a query /api/*/{version}, the following test case /api/anyName/123 will match, while /otherApi/anyName/123 will not.

Application Dependency Map Overview

Sequential evaluation

Rules are applied from top to bottom, and calls are assigned to the first matching rule. The change sequence, simply reorder rules but just drag & drop. Instana default rules can be disabled, but not reordered.

Application Dependency Map Overview

Synthetic endpoints

Endpoints that receive only Synthetic traffic, such as calls to health-check endpoints, can skew the KPIs of your applications and services. Instana auto-detects these endpoints and marks them as synthetic to prevent them from affecting application and service KPIs.

How Synthetic endpoints are used in Instana

Every call filed under a Synthetic endpoint is marked as Synthetic in Unbounded Analytics and does not count against the KPIs of the Application Perspective, Service and Endpoint views.

Synthetic endpoints

Despite their calls not counting against KPIs of application perspectives and Services, the individual Synthetic endpoints have their own dashboards like their non-Synthetic counterparts.

Synthetic endpoint

Default Synthetic endpoints

By default, Synthetic endpoints are identified by the following URL patterns:

The built-in rule listed above can be disabled, and custom rules can be specified to flag additional endpoints as Synthetic, see the custom rules for Synthetic endpoints documentation. The configuration of built-in and custom endpoint rules is accessed through the Configure Services button on top of the Services list.

Custom rules for Synthetic endpoints

Custom rules apply globally, across all services.

To aid in configuration, the affected endpoints and services are displayed.

Synthetic endpoints configuration

Synthetic calls

Besides being able to mark entire endpoints as Synthetic, it is also possible to mark individual calls as Synthetic. This is particularly useful when your Synthetic checks use "production" APIs, which is in general an excellent idea.

Synthetic call rules

A call is marked in Instana as Synthetic if one or more of the following conditions are met:

Synthetic calls in Unbounded Analytics

Synthetic calls are, by default, hidden in Unbounded Analytics, as they tend to be interesting for performance analysis only when they go very wrong. To have Unbounded Analytics query also Synthetic calls, you must opt-in via the Hidden calls -> Synthetic calls switch.

Opting in to Synthetic calls in Unbounded Analytics

Unspecified endpoints

When there is insufficient information to automatically map a call to an endpoint (eg. manually instrumented endpoints without providing sufficient data), those calls are mapped to the special "Unspecified" endpoint.

Other endpoints

When too many endpoints are detected on a given service, calls are grouped under the special "Others" endpoint. This safeguard is meant to keep the set of endpoints to a reasonable size.

It usually happens when one of the Instana predefined rules or a custom rule is grouping calls into a large number of endpoints which received a single call each.

For example, one of the Instana predefined rules is mapping HTTP calls to endpoints whose names come from the first path segment found in the URL. If those segments are built from dynamic values (e.g. "GET /blue-item", "GET /red-item"...), applying this rule would lead to a large number of endpoints, which is not useful.

When dealing with HTTP endpoints, the situation may be improved by changing the configuration of endpoint extraction rules.

Internal trigger endpoints

Traces automatically captured by Instana usually start with an incoming call to a service. However, when the Instana SDK is used, it is possible to start a trace with an outgoing call from a service. In that case Instana will create a Synthetic parent call (mapped to the "Internal trigger" endpoint) to that service to improve readability.

The internal trigger endpoint

Endpoint flow map

The endpoint flow map displays the upstream and downstream endpoints of an endpoint.

Endpoint flow map limitation

The endpoint flow map shows accurate data currently only for short running synchronous traces that finish within 20 seconds. In every other case, the endpoint flow map might show inaccurate data.