Using a Circuit Breaker with Services

About Circuit Breaker

Circuit breaker is an established design pattern that applications implement to prevent a failure in one part of the system from cascading to the rest of the system. In an architecture with distributed applications, such as microservices, many services call other services running on remote servers. If the remote service is unavailable or the network is slow, the calling service may wait until a timeout occurs. During this time, the calling service continues to consume critical resources such as server threads and memory. If multiple services call the unresponsive or failing remote service, the impact of the remote service cascades throughout all the calling services, causing even more resources to be consumed and affected by a single failing service. Implementing a circuit breaker on the call to the remote service can prevent the impact of the failing or unresponsive service or network latency from cascading throughout the system.

The circuit breaker design pattern works much like an electrical circuit breaker which is intended to “trip” or open the circuit when failure is detected. This prevents the flow of electrical current through the circuit. After a time delay, the electrical circuit breaker resets and closes the circuit, which causes the flow of electricity to resume.

In a software application, a circuit breaker functions as a proxy that executes the remote services and monitors the remote service for failures. A failure can be an exception and/or a timeout. When the number of failures meets a predetermined threshold within a specified time period, the circuit breaker “trips” or opens the circuit. Subsequent requests for the service end with an error or result in execution of an alternative service. After a reset period elapses, the circuit breaker sets the circuit state to half-open and executes the next request for the service. By allowing a single request to execute and causing other requests to wait, the circuit breaker gauges the health of the service. Upon success of the service, the circuit breaker closes circuit and waiting requests proceed. However, if the service ends with another failure, the circuit breaker re-opens the circuit.

Circuit breakers can be especially useful in systems with a microservices architecture as these systems often feature a large number of distributed components. By configuring a circuit breaker on the invocation of the remote service, you can limit the impact the abnormal behavior of a remote service on other micorservices and critical resources in your system.

Note: The circuit breaker feature is available by default for a service that resides in a Microservices Runtime. To use the circuit breaker feature with Integration Server, your Integration Server must have additional licensing.

Circuit States

The state of a circuit for a service determines how circuit breaker responds to a request to invoke the service.

The following table identifies how the circuit breaker responds to an invocation request based on the circuit state.

If the circuit state is Then
Closed The circuit breaker executes the service.
Open The circuit breaker does not execute the service. Instead, the circuit breaker returns an exception, which allows the request to fail immediately, or circuit breaker invokes an alternative service.
Half-open The circuit breaker executes the service the next time it is requested. If the service executes successfully, then the circuit breaker changes the circuit state to closed. If the service ends because of a failure event (exception and/or timeout), the circuit breaker changes the circuit state to open.

While the circuit is in a half-open state and circuit breaker is already executing a request for the service, any other requests for the service wait until the circuit exits the half-open state. If service execution is successful, circuit breaker closes the circuit and executes the waiting requests for the service.

How Does a Circuit Breaker for a Service Work?

A circuit breaker works by monitoring a service for failures. The circuit breaker allows service execution to proceed when failure events do not meet an established threshold. However, when the number of failure events meets the established failure threshold within the failure time period, the circuit breaker opens the circuit, preventing further execution of the service.

The following table provides an overview of how a circuit breaker works.

Step Description
1 The server receives a request to invoke a service. Upon receiving a request to invoke a service, the server first determines whether or not a circuit breaker is configured for the service.
  • If a circuit breaker is configured for the service, the server passes the request to the circuit breaker which checks the state of the circuit as described in step 2, below.
  • If a circuit breaker is not used with the service, the server invokes the service.
2 The circuit breaker examines the state of the circuit for the service.
  • If the circuit is closed, the circuit breaker invokes the service using a new thread from the circuit breaker thread pool, which is separate from the server thread pool. The thread that called the service originally waits for the results of the service execution.
  • If the circuit is open, the circuit breaker does not invoke the service. See step 5, below, for more information about how the circuit breaker responds to a request for service with an open circuit.
  • If the circuit is half-open and this is the first request for the service since the circuit state became half-open, the circuit breaker invokes the service.

    When executing a service in a half-open state, any other requests for the service wait until the circuit exits the half-open state. For more information, see step 8, below.

3 When the circuit state is closed, upon execution of the service by the circuit breaker, one of the following occurs:
  • The service executes successfully.
    Note: If the Failure event property is set to Timeout only or Exception or timeout, a successful execution indicates that the service executed successfully within the specified timeout period.
  • The service ends with an exception.

    If the Failure event property is set to Exception only or Exception or Timeout, the circuit breaker considers the exception to be a failure event. The circuit breaker increments the failure count for the service by 1.

  • The service does not execute to completion before the timeout period elapses. If the Failure event property is set to Timeout only or Exception or Timeout, the circuit breaker considers the timeout to be a failure event. The circuit breaker increments the failure count for the service by 1.

    If the Cancel thread on timeout property is set to true, the circuit breaker attempts to cancel the thread executing the service.

If the service ends with a failure event and it is the first failure event, the circuit breaker starts the failure timeout period. If the number of failure events specified in Failure threshold property occurs before the failure timeout period ends, the circuit opens. Circuit breaker determines the time that the failure period ends by adding the number of seconds specified in the Failure period property to the time the first failure event occurred. If the service executes successfully and completes after the failure timeout period ends, circuit breaker resets the failure count for the service to 0.

4 Subsequent invocations of the service result in the circuit breaker opening or “tripping” the circuit for the service if:
  • The number of failure events for the service equals the Failure threshold property value and
  • The failure events occur within the failure timeout period which circuit breaker determines by adding the value of Failure period property to the time of the first failure event.

The circuit breaker starts the reset period which determines the length of time to keep the circuit in an open state. The Circuit reset period property determines the length of the reset period.

For example, suppose that circuit breaker treats exceptions as a failure event, the Failure threshold property is set to 5, and the Failure period is set to 60 seconds. Further suppose that the first failure event occurs at 10:07:10. Circuit breaker starts the failure timeout period. The failure timeout period ends at 10:08:10 which circuit breaker determines by adding the Failure period value to the time of the first failure event. If 5 invocations of the service end with an exception in less than 60 seconds, that is, before 10:08:10, the circuit breaker opens the circuit for the service.

5 When the circuit breaker receives a request to execute the service, the circuit breaker does one of the following:
  • If the Circuit open action property is set to Throw exception, the circuit breaker responds to the invoke request by throwing the exception that caused the circuit to open.
  • If the Circuit open action property is set to Invoke service, the circuit breaker executes the alternate service specified in the Circuit open service property and returns the results to the calling service. The circuit breaker places the $circuitBreakerService and $circuitBreakerEvent parameters in the input pipeline for the alternate service. For more information about the circuit open service, see the section Building a Service for Use with an Open Circuit .
Note: When the circuit is open, circuit breaker does not use the circuit breaker thread pool to throw the exception or execute the alternate, open circuit service. Instead, circuit breaker uses the same thread that executes the calling service to return the exception or execute the alternate, open circuit service.
6 The circuit breaker repeats the previous step in response to a request for the service until the time specified in the Circuit reset period property elapses.
7 When the circuit reset period elapses, the circuit breaker sets the circuit to a half-open state.
8 The circuit breaker receives a request for the service.
  • If this is the first request for the service since the circuit state became half-open, the circuit breaker invokes the requested service. One of the following occurs:
    • If the service executes successfully, the circuit breaker closes the circuit. The circuit breaker will invoke the service upon subsequent service executions. The circuit breaker resets the failure count to 0 (zero).
    • If service execution results in a failure event, the circuit breaker re-opens the circuit and restarts the reset period. The circuit breaker proceeds as described in step 5.
  • While the circuit is in a half-open state and circuit breaker is already executing a request for the service, any other requests for the service wait until the circuit exits the half-open state. If service execution is successful, circuit breaker closes the circuit and executes the waiting requests for the service. If service execution is not successful, circuit breaker re-opens the circuit and responds to the waiting services as described in step 5.

Configuring a Circuit Breaker for a Service

About this task

You can configure a circuit breaker for any user-defined service. Use the Service Development perspective in Software AG Designer to enable and configure a circuit breaker for a service. For configuration instructions, considerations, and guidelines, see the webMethods Service Development Help.

Building a Service for Use with an Open Circuit

When a circuit for a service is open, the service does not execute. Instead, circuit breaker responds to requests for the service by throwing the same exception that caused the last failure event or by executing an alternate service. The alternate service, known as the open circuit service, is a user-defined service that performs an action that makes sense for your application. In some applications, you might want the open circuit service to return a cached value or an alternate result to the calling service. You can also code the open circuit service to perform a different action based on whether it is the first invocation of the open circuit service since the circuit opened or if it is a subsequent invocation.

Whether a circuit breaker throws an exception or executes an alternate service depends on the value of the Circuit open action property. If you specify Invoke service as the circuit open action, you must identify the service to invoke in the Circuit open service property.When circuit breaker invokes the open circuit service, circuit breaker places parameters and values in the service pipeline for use by the open circuit service.

The following identifies and describes the parameters that circuit breaker places in the pipeline for an open circuit service.

Parameter Name Description
$circuitBreakerService String. Fully qualified name of the service with the open circuit.
$circuitBreakerEvent String. Indicates whether this is the first time circuit breaker called the open circuit service since the circuit opened or if this is a subsequent invocation. The $circuitBreakerEvent parameter has one of the following values:
  • CIRCUIT_OPENED if this is the first execution of this open circuit service since the circuit opened.
  • CIRCUIT_OPEN if this is not the first execution of the open circuit service since the circuit opened.
Note: The open circuit service has access to the parameters in the above table as well as any parameters that existed in the pipeline at the time of the execution request for the service with a configured circuit breaker.

Keep the following information in mind when building an open circuit service for use with a circuit breaker:

  • If the open circuit service interacts with a remote resource, such as a database or web server, make the interaction asynchronous to prevent a service execution from blocking other threads or delaying the execution of the original calling service.
  • An open circuit service can be used with more than one service with a configured circuit breaker.
  • Circuit breaker does not use a thread from the circuit breaker thread pool to execute the open circuit service. Circuit breaker uses the same thread that executed the calling service to execute the open circuit service.
  • If an exception occurs while executing this service, it does not impact the circuit breaker failure count for the originally requested service.
  • Software AG recommends that the open circuit service is not configured to use a circuit breaker.
  • Software AG recommends that the open circuit service is not the same as the service with the configured circuit breaker. That is, do not create a circular situation where a service with an open circuit calls itself.

Configuring the Circuit Breaker Thread Pool

Circuit breaker uses a dedicated thread pool, separate from the server thread pool, to execute services for which a circuit breaker is configured. This thread pool is referred to as the circuit breaker thread pool. You can specify the minimum and maximum number of threads in the circuit breaker thread pool.

At run time, circuit breaker uses a thread from the circuit breaker thread pool to execute the requested service, passing the service invocation pipeline to the new thread. This thread is separate from the thread executing the calling service. The calling service waits for a response from the requested service. Circuit breaker returns the service results and then returns the thread to the circuit breaker thread pool. The calling service then proceeds with execution. If the requested service is configured to treat a time out as a failure event and the service does not execute to completion before the timeout period elapses, circuit breaker returns an exception to the calling service. If the Cancel thread on timeout property is set to false, circuit breaker orphans the thread. If the Cancel thread on timeout property is set to true, circuit breaker attempts to cancels the thread. If the thread cannot be canceled, circuit breaker abandons the thread.

Circuit breaker uses a separate thread pool because it decouples the thread that executes the calling service from the thread that executes the requested service. This decoupling allows the calling service to proceed with execution if the requested service does not complete before the timeout period elapses. As a result, failures can return quickly. For example, suppose that a circuit breaker is configured for a service that reads information from a database. If the database goes off line, an attempt to connect to the unavailable database and execute a query may wait a while before returning because network input/output operations typically cannot be interrupted. By using a separate thread for the requested service, the circuit breaker can abandon the thread and return an exception to the client without needing to wait for the input/output operation to complete.

Note: Circuit breaker uses the circuit breaker thread pool to execute only those services for which a circuit breaker is configured. Circuit breaker does not use a thread from the circuit breaker thread pool to execute the circuit open service.

You can configure the size of the circuit breaker thread pool by specifying the minimum and maximum number of threads for the pool. When the server starts, the circuit breaker thread pool initially contains the minimum number of threads. The server adds threads to the pool, as needed, until the pool contains the maximum number of allowed threads. If the pool reaches the maximum number of threads, before executing the next requested service with a configured circuit breaker, circuit breaker must wait for a thread to be returned to the pool.

The server provides server configuration parameters for specifying the minimum and maximum number of threads in the circuit breaker thread pool.

  • watt.server.circuitBreaker.threadPoolMin specifies the minimum number of threads that the server maintains in the circuit breaker thread pool. The circuit breaker thread pool is used to execute services with a configured circuit breaker. When the server starts, the circuit breaker thread pool initially contains this minimum number of threads. The server adds threads to the pool as needed until it reaches the maximum allowed, which is specified by the watt.server.circuitBreaker.threadPoolMax. You must specify a value greater than or equal to 0 (zero) and less than or equal to the value of watt.server.circuitBreaker.threadPoolMax. The default is 10.
    Note: You must restart Microservices Runtime for changes to take effect.
  • watt.server.circuitBreaker.threadPoolMax specifies the maximum number of threads that the server maintains in the circuit breaker thread pool. The circuit breaker thread pool is used to execute services with a configured circuit breaker. If this maximum number is reached, the server waits until services complete and return threads to the circuit breaker thread pool before running more services with a configured circuit breaker. You must specify a value greater than 0 and greater than or equal to the value of watt.server.circuitBreaker.threadPoolMin. The default is 75.
    Note: You must restart Microservices Runtime for changes to take effect.

Use the server configuration properties to size the circuit breaker thread pool appropriately for your environment. Keep in mind that all services for which a circuit breaker is configured share the circuit breaker thread pool.

Circuit Breaker Statistics

Microservices Runtime gathers circuit breaker statistics for each service with a configured circuit breaker. Microservices Runtime Administrator displays statistics in the Circuit Breaker Information table on the Server > Service usage page.

The following table identifies the circuit breaker information that Microservices Runtime maintains for each service with a configured circuit breaker.

Field Description
Name Name of the service for which a circuit breaker is configured.
State The state of the circuit. The circuit state can be one of the following:
  • Closed. The service executes.
  • Open. The service does not execute. Instead, depending on the circuit breaker configuration, the service returns an exception or executes an alternative service.
  • Half-Open. The first request for this service since the circuit state changes to half-open results in service execution. All other requests wait. If the service executes successfully, the circuit is closed and waiting requests execute. If the service ends with a failure exception, the circuit is re-opened.

For a detailed explanation of possible circuit states, see Circuit States.

Open Time Time at which circuit breaker last set the circuit state to open.
Half-Open Time Time at which circuit breaker last set the circuit state to half-open.
Closed Time Time at which the circuit breaker last set the circuit state to closed.
Open Count Number of times that circuit breaker set the circuit state to open since Microservices Runtime started.
Request Count Number of incoming requests for the service since the circuit breaker changed the circuit state to open. For information about how circuit breaker handles requests for a service with an open circuit, see How Does a Circuit Breaker for a Service Work? .
Note: If a circuit breaker is not configured for any service, the Circuit Breaker Information table displays the message “No Services with a Circuit Breaker Enabled”.