Metrics of runtimes

Monitor key performance metrics such as status, performance, throughput, response time, latency, and error rate to assess the health and efficiency of runtimes. It helps you to make informed business decisions by analyzing the performance metrics of the runtimes.

The widgets on the Monitor runtimes page are explained in detail as follows.

Status, performance, and used capacity

The widget provides a comprehensive view of all runtimes by displaying their status, performance, and used capacity.

Status

The status of the runtime indicates the health of the runtime. It is determined based on the availability of the runtime. The following are the health status of the runtime.

Status	Description
	Indicates that the runtime’s availability is 100%.
	Indicates that the runtime’s availability is greater than 99% and lesser than 100%.
	Indicates that runtime’s availability is less than 99%.
	Indicates that there is no adequate data to compute the availability of runtimes. For example, if you register a runtime an hour ago and try to view the dashboard by applying the default Last 1-hour time range, the status appears gray due to the lack of available data during that time period.

Performance

Performance shows how good each runtime performs based on two metrics - error rate and latency. The threshold for the latency and error rate is set in the settings page. Set your preferences by customizing the display settings of the widgets and threshold values for the parameters that control the monitoring capability of the assets.

The following are the performance status of the runtime.

Performance status	Description
	Green indicates that both latency and error rate threshold values are within the acceptable limit.
	Amber indicates that either the latency or error rate threshold value is within the warning limit or greater than the unacceptable limit.
	Red indicates that both latency and error rate threshold values are within the warning limit or greater than the unacceptable limit.
	Gray indicates that there is no adequate data to compute the performance of the runtimes in the data plane. For example, if you register a runtime an hour ago and you try to view the dashboard by applying the default Last 1 hour time range, the status appears as gray due to the lack of available data during that time period.

Used capacity

Capacity displays the used capacity rate of the runtime. Capacity is the number of transactions that a runtime can handle.

The following are the used capacity status of the runtime.

Used capacity status	Description
	Green indicates that the capacity rate is lesser than or equal to 30%
	Amber indicates that the capacity rate is greater than 30% and lesser than or equal to 70%
	Red indicates that the capacity rate is greater than 70%
	Gray indicates one of the following, There is no adequate data to compute the capacity rate. For example, if you register a runtime an hour ago and try to view the dashboard by applying the Last 24 hours filter, the status appears gray due to the lack of available data during that time period. The capacity is not defined for all runtimes in the data plane.

How to determine the used capacity of the runtime?

$Used capacity of the
runtime = (\frac{Actual TPS of the runtime}{Expected TPS the runtime}) X 100$

Example,

To determine the used capacity of a runtime, assume that you define the capacity as 10000 transactions per hour for runtime A. For more details about how to define the capacity of a runtime, see Managing runtimes in federated API management.

The expected TPS of the runtime A = 10000 / (1 hr * 60 min * 60 secs) = 2.77 transactions per second.

Assume that runtime A performed 39000 transactions in the last 6 hours.

The actual TPS of the runtime A = 39000 / (6 hrs * 60 mins * 60 secs) = 1.81 transactions per second.

Used Capacity of the runtime A =((1.81+2.77)*100 = 65%

The capacity of the runtime A is displayed as 65% in amber.

Problematic runtimes by

The widget helps identify issues across runtimes by analyzing total transaction volumes, error rate, response time, and latency of all associated runtimes. Choose a business metric to view its corresponding least and most problematic runtime. The runtime split and line graph display the runtime in a sequence from the most problematic to the least. Click the Show as table icon to view the line graph in table format. Click ellipsis icon to download the table in CSV, PNG, or JPG formats.

Transactions

The widget provides a comprehensive overview of total transaction volumes across all runtimes with a trend percentage, along with detailed insights into the most and least performing runtimes to help you make informed business decisions. It presents both pictorial and graphical representations of the runtimes based on performance, highlighting maximum and minimum transaction counts. Click Top runtimes or All runtimes tab to view its respective transactions in the runtime split and line graph.

The Upward Trending tab displays the list of runtimes for which the transaction trend percentage is greater than the set transaction trend threshold. The Downward trending tab displays the list of runtimes for which the transaction trend percentage is lesser than the set transaction trend threshold.

How to determine the total transactions and trend analysis for all runtimes?

$Total transactions of the runtime = \sum_{i=1}^{n} (Total transactions of the runtime in the time stamp i)$

Example,

To determine the total transactions of the runtime A for 4 hours based on the following metrics, assume,

Time stamp	Total transactions
1st one hour	100
2nd one hour	200
3rd one hour	200
4th one hour	50

Total transaction of the runtime A for 4 hours=100+200+250+100=550

Assume the total transactions of the runtime B for 4 hours are 450.

$Total transactions of all runtimes = \sum_{i=A}^{n} (Total transactions of runtime i)$

Runtime	Total transactions
A	550
B	450

Total transaction of all runtimes for 4 hours=550+450=1000

With the transactions trend analysis, you can compare current and past transactions based on the time range you choose to spot changes.

\begin{array}{l} Transactions trend (%) for all runtimes for 24 hours = \\ (\frac{Total transactions of runtime in last 24h - Total transactions of runtime in previous 24h}{Total transactions of runtime in previous 24h}) \times 100 \end{array}

You can replace 24 hours with the time range you select in the filter. If the determined trend percentage of transactions is,

A positive value, it is displayed in green with an upward arrow that indicates that the transaction volume is increased.
A negative value, it is displayed in red with a downward arrow that indicates that the transaction volume is decreased.

Example,

To determine the transactions trend percentage for all runtimes for 24 hours, assume,

Total transactions of runtime in the last 24 hours	Total transactions of runtime in the previous 24 hours (that is the 24 hours prior to the last 24 hours)
1000	500

Transactions Trend percentage for 24 hours for all data planes =[(1000-500)/500]*100 = 100%

The determined transactions trend percentage, 100% (positive), appears in green with an upward arrow that indicates increased transactions compared to the previous 24 hours.

Error rate

The error rate indicates the percentage of errors that occurred during API transactions. This widget provides a comprehensive overview of the error rate of all runtimes along with the details of the runtime. It provides the details of the runtime with maximum and minimum error rate to make informed business decisions. Click Top runtimes or All runtimes tab to view its respective error rate in the runtime split and line graph.

The Upward Trending tab displays the list of runtimes for which the error rate trend percentage is lesser than the set error rate trend threshold. The Downward trending tab displays the list of runtimes for which the error rate trend percentage is greater than the set error rate trend threshold.

How to determine the error rate and trend analysis for all runtimes?

$Error rate of the runtime = (\frac{\sum_{i=1}^{n} (Error count of the runtime in the time stamp i)}{\sum_{i=1}^{n} (Transactions count of the runtime in the time stamp i)}) \times 100$

Example,

To determine the error rate of the runtime A for 4 hours based on the following metrics, assume,

Time Stamps	Error count	Total transactions
1st one hour	150	300
2nd one hour	115	200
3rd one hour	30	500
4th one hour	100	300

Error rate of the runtime A for 4 hours=[(300+200+500+300)/(150+115+30+100)∗100]=35.90%

Assume the error rate of the runtime B for 4 hours is 50%

$Error rate of all runtimes = (\frac{\sum_{i=A}^{n} (Error count of the runtime i)}{\sum_{i=A}^{n} (Transactions count of the runtime i)}) \times 100$

To determine the error rate of all runtimes for 4 hours, assume,

Runtime	Error count	Total transactions
A	395	1100
B	250	500

Error rate of all runtimes for 4 hours=[(395+250)/(1100+500)∗100]=40.31%

With the error rate trend analysis, you can compare current and past error rates that are based on the time range you choose to spot changes.

\begin{array}{l} Error rate trend (%) for all runtimes for 24 hours = \\ (\frac{Error rate of all runtimes in last 24h - Error rate of all runtimes in previous 24h}{Error rate of all runtime in previous 24h}) \times 100 \end{array}

You can replace 24 hours with the time range that you select in the filter. If the determined error rate trend percentage is,

A positive value, it is displayed in red with an upward arrow, which indicates that the error rate is increased.
A negative value, it is displayed in green with a downward arrow, which indicates that the error rate is decreased.

Example,

To determine the error rate trend percentage for all runtimes for 24 hours, assume,

Error rate of all runtimes in the last 24 hours	Error rate of all runtimes in the previous 24 hours (that is 24 hours prior to the last 24 hours)
35.90%	50%

Error rate trend percentage for 24 hours filter for all runtimes = [(0.35-0.50)/0.50] * 100 = -30%

The determined error rate trend percentage, -30% (negative), appears in green with a downward arrow that indicates the error rate is decreased compared to the previous 24 hours.

Availability

The widget shows the availability of all runtimes.

The availability of a runtime is determined based on the sum of the heartbeat status value of the associated runtimes.

If federated API management receives the heartbeat status from the runtime, the value of the heartbeat status is represented as 1.
If federated API management does not receive the heartbeat status from the runtime, the value of the heartbeat status is represented as 0.

The frequency at which the runtime must send the heartbeat status to federated API management is defined in the federated API management Agent configuration.

For example, if the heartbeat interval is defined as 60 seconds in the federated API management agent configuration. Then for an hour the expected heartbeat status value must be 3600.

$Availability of a runtime = (\frac{\sum_{i=1}^{n} (Actual heart beat value of the runtime in the time stamp i)}{\sum_{i=1}^{n} (Expected heart beat value of the runtime in the time stamp i)}) \times 100$

Example,

To determine the availability of the runtime A for 4 hours based on the following metrics, assume,

Time stamps	Sum of the actual heartbeat value received	Sum of the expected heartbeat value
1st one hour	1800	3600
2nd one hour	3600	3600
3rd one hour	3000	3600
4th one hour	3600	3600

Availability of a runtime= [(1800+3600+3000+3600)/ (3600+3600+3600+3600)]*100 = 83.33%

Assume the availability of the runtime B for 4 hours is 100%

$Availability of all runtimes = \frac{(\sum_{i = A}^{n} Availability of runtime i)}{Total Number of runtimes}$

Assume,

Runtime	Sum of the heart beats received	Sum of the heart beats expected
A	55	60
B	35	60

Availability of all runtimes=(283.33+100)∗100=91.66%

With the availability trend analysis, you can compare the current and past availability status based on the time range you choose to spot changes.

\begin{array}{l} Availabilty trend (%) for all runtimes for 24 hours = \\ (\frac{Availabilty of all runtimes in last 24h - Availabilty of all runtimes in previous 24h}{Availabilty of all runtimes in previous 24h}) \times 100 \end{array}

You can replace 24 hours with the time range that you select. If the determined trend percentage of availability is

A positive value, it is displayed in green with an upward arrow that indicates that the availability is increased.
A negative value, it is displayed in red with a downward arrow that indicates that the availability is decreased.

Example,

To determine the availability trend percentage for all runtimes for 24 hours, assume,

Availability of all runtimes in the last 24 hours	Availability of all runtimes in the previous 24 hours (that is 24 hours prior to the last 24 hours)
98%	80%

Availability trend percentage for runtimes for 24 hours filter =](0.98-0.80)/0.80]*100 = 22%

The determined availability trend percentage, 22% (positive), appears in green with an upward arrow that indicates the availability is increased compared to the previous 24 hours.

Response time

Response time indicates the total time that is taken to process an incoming API request and send the response to the client during API transactions. It includes the time that is taken by the runtime and the backend service to process the request. This widget provides a comprehensive overview of response time of all runtime. It provides details of the runtime with maximum and minimum latency to make informed business decisions. Click Top runtimes or All runtimes tab to view its respective response time in the runtime split and line graph.

The Upward Trending tab displays the list of runtimes for which the response time trend percentage is lesser than the set response time trend threshold. The Downward trending tab displays the list of runtimes for which the response time trend percentage is greater than the set response time trend threshold.

How to determine the response time and trend analysis for all runtimes?

$Response time of a runtime = \frac{(\sum_{i = 1}^{n} Response time of the runtime in time stamp i \times Total transaction count of the runtime in time stamp i)}{\sum_{i = 1}^{n} Transaction count of runtime i}$

Example,

To determine the response time of the runtime A for 4 hours based on the following metrics, assume,

Time Stamps	Response time	Total transactions
1st one hour	1.05 ms	10
2nd one hour	2.07 ms	15
3rd one hour	0.08 ms	20
4th one hour	1.07 ms	30

Response time of the runtime A for 4 hours=[((1.05∗10)+(2.07∗15)+(0.08∗20)+(1.07∗30))/75]=1.00 ms

Assume the response time of the runtime B for 4 hours is 0.05 ms

$Response time of all runtimes = \frac{(\sum_{i = A}^{n} Response time of runtime i \times Total transaction count of runtime i)}{\sum_{i = A}^{n} Transaction count of runtime i}$

To determine the response time of all runtimes for 4 hours, assume,

Runtime	Response time	Total transactions
A	1.00 ms	75
B	0.05 ms	60

Response time of all runtimes=[((1.00∗75)+(0.05∗60))/135]=0.57 ms

With the Response time trend analysis, you can compare the current and past response time based on the time range you choose to spot changes.

\begin{array}{l} Response time trend (%) for all runtimes for 24 hours = \\ (\frac{Response time of all runtimes in last 24h - Response time of all runtimes in previous 24h}{Response time of all runtimes in previous 24h}) \times 100 \end{array}

You can replace 24 hours with the time range that you select. If the determined trend percentage of response time is

A positive value, it is displayed in red with an upward arrow that indicates the response time is increased.
A negative value, it is displayed in green with a downward arrow that indicates the response time is decreased.

Example,

To determine the response time trend percentage for all runtimes for 24 hours, assume,

Response time of all runtimes in the last 24 hours	Response time of all runtimes in the previous 24 hours (that is 24 hours prior to the last 24 hours)
0.57 ms	3.41 ms

Response time trend percentage for runtimes for 24 hours time range= [(0.57-3.41)/3.41] * 100 = -83.28%

The determined response time trend percentage, -83.28% (negative), appears in green with a downward arrow that indicates the error rate is decreased compared to the previous 24 hours.

Latency

Latency indicates the time that is taken by the runtime to process an API request during API transactions. It does not include the time that is taken by the backend service to process the request. This widget provides a comprehensive overview of latency of all runtimes along with the details of the runtime with maximum and minimum latency to make informed business decisions. Click Top runtimes or All runtimes tab to view its respective latency in the runtime split and line graph.

The Upward Trending tab displays the list of runtimes for which the latency trend percentage is lesser than the set latency trend threshold. The Downward trending tab displays the list of runtimes for which the latency trend percentage is greater than the set latency trend threshold.

How to determine the latency and trend analysis for all runtimes?

$Latency of a runtime = \frac{(\sum_{i = 1}^{n} Latency of the runtime in time stamp i \times Total transaction count of the runtime in time stamp i)}{\sum_{i = A}^{n} Transaction count of runtime i}$

Example,

To determine the latency of the runtime A for 4 hours based on the following metrics, assume,

Time Stamps	Latency	Total transactions
1st one hour	0.05 ms	10
2nd one hour	0.07 ms	15
3rd one hour	0.08 ms	20
4th one hour	0.07 ms	30

Response time of the runtime A for 4 hours=[((0.05∗10)+(0.07∗15)+(0.08∗20)+(0.07∗30))/75]=0.04 ms

Assume the latency of the runtime B for 4 hours is 0.05 ms.

$Latency of all runtimes = \frac{(\sum_{i = A}^{n} Latency of runtime i \times Total transaction count of runtime i)}{\sum_{i = A}^{n} Transaction count of runtime i}$

To determine the response time of all runtimes for 4 hours, assume,

Runtime	Reponse time	Total transactions
A	0.04 ms	75
B	0.05 ms	60

Response time of all runtimes=[((0.04∗75)+(0.05∗60))/135]=0.04 ms

With the Latency trend analysis, you can compare the current and past latency based on the time range you choose to spot changes.

\begin{array}{l} Latency Trend (%) for all runtimes for 24 hours = \\ (\frac{Latency of all runtimes in last 24h - Latency of all runtimes in previous 24h}{Latency of all runtimes in previous 24h}) \times 100 \end{array}

You can replace 24 hours with the time range that you select. If the determined trend percentage of latency is,

A positive value, it is displayed in red with an upward arrow that indicates the latency is increased.
A negative value, it is displayed in green with a downward arrow that indicates that the latency is decreased.

Example,

To determine the latency trend percentage for all runtimes for 24 hours, assume,

Latency of all runtimes in the last 24 hours	Latency of all runtimes in the previous 24 hours (that is 24 hours prior to the last 24 hours)
0.04 ms	3.41 ms

Response time trend percentage for runtimes for 24 hours time range= [(0.04-3.41)/3.41] * 100 = -98.82%

The determined response time trend percentage, -98.82% (negative), appears in green with a downward arrow that indicates the error rate is decreased compared to the previous 24 hours.