Services Framework graceful shutdown

The Services Framework API provides a task with the ability to detect that the Services Framework Liberty server is shutting down. And then to take appropriate action allowing the task to resume execution from the point of shutdown on the next available Services Framework server.

The SampleExtractTask Services Framework sample task that is included with FTM provides an example of using this feature of the API. The specifics of using this feature are provided in the following sections.

The codes that need to be added to the SampleExtractTask task to enable the shutdown of Services Framework are described in the following sections.

Implementing a new class

Implement a new class that extends the abstract TaskShutdownHandler class and overrides the abstract processShutdown method.
public class MyTaskShutdownHandler extends TaskShutdownHandler
{
    @Override
    public boolean processShutdown()
    {
        // Custom shutdown handling code
        // Return true to allow the task to be shutdown
        return true;
    }
}

Overriding createTaskShutdownHandler

The Services Framework runtime invokes the createTaskShutdownHandler method before the task’s runTask method. This step registers the custom TaskShutdownHandler with the framework and enables the task shutdown or resume feature for the task.

The steps to override the createTaskShutdownHandler method are shown in the following list.
  1. Declare MyTaskShutdownHandler by using a class variable.
  2. Override the createTaskShutdownHandler method of the BaseIPDTask class in your custom task class. This method needs to create an instance of the custom Task Shutdown Handler and return it.
    @Override
    protected TaskShutdownHandler createTaskShutdownHandler()
    {
        this.myShutdownHandler = new MyTaskShutdownHandler();
        return this.myShutdownHandler;
    }

Invoking registerSqlStatementForShutdown

This step enables the shutdown thread to cancel the database query/update/insert operation at the time of shutdown.

Invoke the registerSqlStatementForShutdown method of the TaskShutdownHandler to register a Java PreparedStatement with the DatabaseStatementService before issuing a call to the database.
public void registerSqlStatementForShutdown(Statement dbStatement,
String sql, String moduleName)

When using database accessor classes to access the database, the prepared statements are automatically registered and unregistered. Therefore, it is not needed to use the registerSqlStatementForShutdown and deRegisterSqlStatementForShutdown methods.

Note: Database accessor classes are those classes that extend the com.ibm.ftm.base.util.database.accessors.DatabaseAccessor class.

Invoking deRegisterSqlStatementForShutdown

Invoke the deRegisterSqlStatementForShutdown method when PreparedStatement is not used to minimize the memory footprint.
public void registerSqlStatementForShutdown(Statement dbStatement)

Invoking TaskShutdownHandler.checkAndProcessShutdown

Invoke the TaskShutdownHandler.checkAndProcessShutdown method from your task execution code by using the following steps.
  1. Call checkAndProcessShutdown when long-running loops occur and for each iteration of the loop.
  2. Call checkAndProcessShutdown when SQLException or the finally clause that is associated with that exception is caught. It is because the cancel operation on the PreparedStatements database that is registered with TaskShutdownHandler throws SQLException after cancellation.
  3. Call checkAndProcessShutdown to determine whether the shutdown occurred and if so, to process the shutdown.
  4. Call checkAndProcessShutdown when catching a DatabaseAccessorException or in the finally clause that is associated with that exception.

When shutdown is complete, the status of the task instance displays Shutdown.

This method determines whether the server shutdown is signaled and returns the following values.
  • false if shutdown is not signaled.
  • If shutdown is signaled, it invokes the processShutdown method that was implemented in the custom task shutdown handler.
  • If it returns true, checkAndProcessShutdown does not return. Instead, it throws the TaskShutdownException exception to signal the Services Framework runtime that the task completed shutdown processing and is ready to be shutdown.
    Note: Your task code must not catch this exception. It must be allowed to propagate to the upstream runtime code.

Resuming the task after shutdown

The steps that occur when a task is being shutdown and the TaskShutdownHandler is registered with the framework are shown in the following list.
  1. An IBM® MQ message is posted to the Services Framework input queue (FXH.SERVICESFRAMEWORK.INPUT.QUEUE) when shutdown processing is complete. This IBM MQ message resumes the task that is being shutdown.
  2. The next available Services Framework engine processes the message and resumes the task execution based on the parameters that the task’s shutdown process creates. For example, the resume-points created at the time of shutdown.

Task shutdown process

When a Services Framework task runs, the steps that occur on the task execution thread are shown in the following list.
  1. The task’s createTaskShutdownHandler is called.
  2. When a custom TaskShutdownHandler is implemented for this task, the createTaskShutdownHandler method creates an instance of this class and returns it to the Services Framework runtime. This registers the task instance with the Services Framework runtime. It also effectively enables the runtime to execute graceful shutdown or to resume the task.
  3. The custom task code invokes the TaskShutdownHandler.checkAndProcessShutdown method during some iterative processes. Also, database prepared statements are registered with TaskShutdownHandler.
  4. The operator manually deletes the services-engine pod or the Red Hat® OpenShift® operator deletes it. The Kubernetes preStop lifecycle hook is triggered.
  5. The FTM operator issues an HTTP post request to the Services Framework engine.
  6. The Services Framework shutdown process executes on a distinct thread within the Services Framework engine.
  7. For each task instance that is registered with the runtime, the Thread.interrupt method is called. Also, any database-prepared statements that were registered with the TaskShutdownHandler are canceled.
  8. The shutdown process waits for each registered task instance status to be set to a value of shutdown or until the time the Shutdown Timeout core property elapses.
  9. The HTTP post call returns and the Liberty server shutdown quiesce process runs, and then the server shuts down.
  10. Back on the task thread, the
    1. TaskShutdownHandler.checkAndProcessShutdown method is called and it detects that the thread is interrupted. It runs the processShutdown method of the custom TaskShutdownHandler. This step runs the required shutdown processing for the task.
    2. The TaskShutdownException exception is thrown. The Services Framework runtime catches this exception.
  11. A resume-task message is posted. This message enables the next available server to automatically resume the task. The task instance status is set to shutdown.
  12. The task execution completes. When Services Framework engine receives new task messages on its input queue after shutdown, the messages are reposted to the Services Framework input queue so that the next available server can process them.