Threads, connections, and transactions overview

It is important to understand how threads, connections, and transactions interact.

You must separate threads for adapter connections and map logic. This allows connections to remain open even after maps complete. The Resource Manager maintains a pool of connection threads and associates one or more with a map thread during the execution of a map.

There is always a one-to-one relationship between a connection and a thread; one connection is associated with one thread. Multiple calls associated with one connection all occur on a single thread, that is, if the Connect method is called from thread n, all subsequent Listen, Get, Put, BeginTransaction, EndTransaction,and Disconnect calls for that connection will also be called from thread n.

Adapters fit into one of two categories:
  • Those that have a one-to-one relationship between a connection and a transaction.
  • Those that allow for multiple, concurrent transactions on the same connection.
The determination as to the category into which an adapter falls is normally made by the implementation of the transaction mechanism in the resource to which the adapter interfaces.

The adapter defines the mode in which it is operating by setting the nTransMode value in the vtable library to one of the following values:

  • MPI_TRANSACTIONS_NONE
  • MPI_TRANSACTIONS_SINGLE
  • MPI_TRANSACTIONS_MULTIPLE
This parameter takes affects when the BeginTransaction and EndTransaction methods are called as follows:
  • For adapters allowing for a single transaction per connection, there is only one call to BeginTransaction and EndTransaction.
  • For adapters allowing for multiple transactions per connection, there will be multiple calls.
  • For adapters with a transaction mode of NONE, there will be no calls to BeginTransaction or EndTransaction.
The following example shows the sequence of calls for a map that has two output cards, both with Scope setting set to Map, which can operate within the same connection:
Single transaction per connection Multiple transactions per connection
Connect Connect
BeginTransaction (1) BeginTransaction (1)
Put (1) Put (1)
CompareConnection --> MPI_CMP_EQUAL CompareConnection --> MPI_CMP_EQUAL
Put (2) BeginTransaction (2)
EndTransaction (1) Put (2)
Disconnect EndTransaction (1)

EndTransaction (2)

Disconnect
There is a requirement of some resources that all activity for a single resource occurs on a single thread. The capability to execute Listen, Get and Put with the same connection thread is possible.

Obviously, if a connection thread is being used during the execution of a map, it cannot be listening for the arrival of new events. Therefore, multiple listener threads must be started to enable concurrent execution and responsive handling of events. This adds a new constraint for the Resource Manager—the number of maps that can run concurrently to process a specific watch depends on the number of listener threads that have been created.

The config.yaml configuration file has a parameter (MaxWatchThreads), which is the maximum number of threads that can be created for connections for a given resource. This is used to determine the number of threads that will be started to listen for a given event if the adapter property MPIP_ADAPTER_LISTEN_USESAMECONN is TRUE.