IBM Business Analytics Proven Practices
IBM Cognos BI Dispatcher Routing Explained
Nature of Document: Guideline; Product(s): IBM Cognos BI; Area of Interest: Infrastructure; Version: 1.2
This content is part # of # in the series: IBM Business Analytics Proven Practices
This content is part of the series:IBM Business Analytics Proven Practices
Stay tuned for additional content in this series.
Purpose of Document
This document is meant to supplement the Security and Administration Guide and Architecture and Deployment Guide which are part of the IBM Cognos BI product documentation.
It explains the concepts of Dispatcher routing including load balancing and Advanced Routing in depth for both, interactive and batch processing scenarios to Administrators and Architects.
The concepts described herein apply to all versions of IBM Cognos 8 BI and IBM Cognos 10 BI unless stated otherwise explicitly.
Exclusions and Exceptions
This document is restricted to Dispatcher based routing and does not touch on Service routing, that is, how particular services might apply other forms of routing among multiple instances of service executables.
The reader is expected to be familiar with the concepts described in the Architecture and Deployment Guide.
Some basic understanding of Java and J2EE is helpful.
Routing in IBM Cognos BI
IBM Cognos BI is based on a Service Oriented Architecture (SOA). This implies the product consists of a set of independent services which communicate via SOAP over a network. There are many different services which all implement different features of the product. Each service can only serve a certain type of request. This is one of the routing challenges in an SOA, to route a request to a service which can serve this type of request.
Another aspect of the routing challenges in an SOA is load balancing and/or fail-over. There can be multiple instances of the same type of service in an overall system. If there are multiple instances, then either load balancing (each instances get's assigned a configurable percentage of the requests of that type) or fail-over (requests for a certain type of service get re-routed to an active instance because another one failed) can happen.
IBM Cognos BI addresses both routing challenges by a software component called the Dispatcher. The Dispatcher, technically, is a Java Servlet which implies it handles HTML input and generates HTML output. In the case of IBM Cognos BI, the input and output payload is actually using the Simple Object Access Protocol (SOAP) which again, technically, is XML payload transported over the HTTP protocol.
Each Dispatcher hosts a set of services which are determined by the system components installed in this instance of IBM Cognos BI. The services get registered to the Dispatcher and it's the Dispatcher which controls them. At the same time the Dispatcher “knows” which service instances it hosts and hence which types of requests it can serve locally.
When a Dispatcher is started up, it will register itself with the active Content Manager (CM). It will report the services it hosts and will obtain information about the system from CM. Through this process, each Dispatcher gathers information about all other Dispatchers in the system and the services they host.
While a simple single server environment may be sufficient for testing, production systems usually consist of several installed instances, sometimes called nodes, each running a Dispatcher with it's own set of services registered. With multiple nodes, load-balancing and fail-over become possible. The IBM Cognos BI architecture implements this by a logical bus which exists between the Dispatchers on each node. On this logical bus requests get passed/routed between Dispatchers in a system and eventually to a specific service instance registered with one of the Dispatchers. This process will acknowledge load balancing and service availability as will be explained during the course of this document.
Clients send requests destined for a certain service to a Dispatcher to get them served. The Dispatchers of a system will ensure the request is routed to an available instance of the requested service which will handle the request and relay back the result to the client.
Entry Point Considerations
Because the Dispatcher is a Java Servlet, it must be deployed to a Servlet Container/Java Application Server which typically is located in the Application Tier of a classical 3-tier architecture. It's considered a potential risk to expose the Application Tier to external (e.g. from an uncontrolled or less controlled environment) clients for direct access, thus as a best practice, some other component in the web tier like a web server is used to handle the communication with the external clients. Only dedicated controlled and secured communication paths between Web Tier servers and Application Tier servers are allowed.
The IBM Cognos BI architecture supports this approach by supplying the IBM Cognos BI Gateway which exists in 6 different implementations (CGI, MOD, MOD2, MOD2_2, ISAPI) and an additional Servlet implementation. Typical installations use Gateway(s) to act as the entry point to the IBM Cognos BI system. The Gateway component, though, does nothing to the request but relay it to the first available Dispatcher in it's configured list of Dispatchers, it can be perceived as a proxy. The Servlet Gateway is a special implementation of a Gateway in such that it has to be deployed to a Java Application server or Servlet Container, typically located in the Application Tier. It is used for setups where other software is used to proxy requests from the Web Tier to the Application Tier. One example are Application Server Plug-Ins deployed to web servers for this particular purpose like they exist for IBM WebSphere or JBOSS.
The Gateway does not route, it's using a static connection to the first available Dispatcher in its configuration, which only changes if the first configured Dispatcher is not reachable. Only in this case the Gateway will try to forward the request to the second Dispatcher in its configured list of Dispatchers and so forth. That being said, routing only starts whenever a given request hits an IBM Cognos BI Dispatcher for the very first time. The Dispatcher which initially receives a client request is called the FRONT Dispatcher for this request. This becomes important for several specific request flows involved with authentication (refer Appendix A).
Technically though, it doesn't make a difference what the entry point is: a Dispatcher or a Gateway. This is remarkable because it implies that one can use application server features like web server plug-ins which proxy requests received in the web tier to the application tier in the same manner as the IBM Cognos BI Gateway does.
For the rest of the document no differentiation between Dispatcher or Gateway is made, whenever required it will reference to an entry point to simplify things.
The following section will explain the main concepts of IBM Cognos BI routing.
Server Groups and System Views
The first concept to be described is the one of Server Groups.
A Server Group (SG) is a defined set of Dispatchers. Each Dispatcher is assigned to exactly one Server Group at a time.
By default there is a single default Server Group to which all Dispatchers belong automatically. This default Server Group only exists logically and cannot be changed or configured.
An Administrator can however assign a Dispatcher to another, explicitly defined SG by specifying a string value for the Server Group property of the Dispatcher object in IBM Cognos Administration. That string value implicitly defines an additional, new Server Group using the string value for its name, if it doesn't exist already. If it does exist already, the Dispatcher is added to the existing Server Group of that name. Explicit Server Groups do not replace the default Server Group though, they supplement it.
Figure 1 - Screencapture from Cognos Administration showing the properties of a Dispatcher which was assigned to SG 'Endor'
So once the first explicit Server Group is defined, one can perceive the overall system to be partitioned by Server Groups. Some Dispatchers may remain in the default Server Group, while others have been moved to an explicit Server Group.
Logically there do exist two logical "System Views".
The “simple View” by which all Dispatchers are considered to belong to the default Server Group, although their actual Server Group property may have been defined to move them to another explicit Server Group. This simple view therefore ignores Server Group assignments completely.
In contrast, the “advanced view” perceives the system to be partitioned into several Server Groups, all explicit Server Groups plus the default Server Group.
Both System Views co-exist, there is no either/or. They can be perceived as overlays applied to the overall set of Dispatchers, one is simple and consists of a single Server Group whereas the other is more complex consisting of more than one Server Group.
Depending on the request at hand, Dispatcher may use either view to perform its routing task. This emphasizes the importance of Server Groups for load balancing and service allocation in general. Eventually Server Groups are the prerequisite to allow the use of Advanced Routing, which will be discussed later.
Server Groups definitions get stored in the Content Store and only CM can supply information about a Dispatcher's assignment to a Server Group. The definitions get managed implicitly through the Server Group properties of the Dispatchers in a system mentioned above. If a certain SG X is not referenced by any Dispatcher anymore, that Server Group X will be deleted in Content Store automatically. The list of Server Groups could be perceived as a “SELECT DISTINCT” on the Server Group property of all Dispatchers in a system.
One important thing is to not mix up Server Groups with folders. IBM Cognos Administration offers the possibility to group Dispatchers in folders. This however does not imply a Server Group. This is an organizational grouping only. By using folders one can apply properties to multiple Dispatchers at once. Dispatchers will inherit property values from the parent object, like a folder, by default unless stated otherwise.
Dispatcher Cluster Information
To be able to assign requests to the requested service and the “best suited” instance of that service and of course route the requests correctly, a Dispatcher requires comprehensive information about:
- which Dispatchers are up and running in the IBM Cognos BI system
- the Server Groups those Dispatchers belong to
- the services registered at each Dispatcher
- the running state of each service instance
This collection of crucial system state information is called the “Dispatcher Cluster Information”. It is centrally managed by the Content Manager and updates or enquiries must be directed to the active Content Manager service instance.
As was briefly mentioned before, each Dispatcher registers itself with Content Manager Service upon its start-up. During this registration process the Dispatcher will inform CM about the services it hosts and their running state (enabled/disabled). CM will add this information to the Dispatcher Cluster Information gathered so far. Next the Dispatcher queries CM (via Content Manager Service) for a copy of the now updated and most recent Dispatcher Cluster Information. Obviously this contains all registered Dispatchers and all service instances registered to them so that eventually the Dispatcher after start-up has information about all other Dispatchers and their registered services. This local copy of the Dispatcher Cluster Information must be kept in sync with the CM of course.
This happens by means of a keep-alive ping in the form of a light-weight CM query which is sent by every running Dispatcher every 30 seconds independently. The purpose of this query is two-fold. First and foremost the Dispatcher signals to the CM that it is still up and running. This concept enables CM to account for any Dispatcher to become unavailable for whatever reason, either intentionally (stopped) or unintentionally (crash). This would result in CM updating the master copy of the Dispatcher Cluster Information which would then propagate to all other Dispatchers within the next 30 seconds automatically because of them probing for changes via the keep-aline ping. Secondly, by this query a Dispatcher polls for possible updates of the Dispatcher Cluster Information to keep its local copy in sync. If the keep-alive ping signals the need to sync changes a configure call is sent to CM which pulls the most recent copy of the Dispatcher Cluster Information and forces a local reconfigure which means Dispatcher and all services registered to it will update their configuration accordingly.
Secondly, by this query a Dispatcher polls for possible updates of the Dispatcher Cluster Information to keep its local copy in sync. If a change is detected a configure call is sent to CM which pulls the most recent copy of the Dispatcher Cluster Information and forces a local reconfigure which means Dispatcher and all services registered to it will update their configuration accordingly.
Next to this keep-alive pattern there is only one other way to update the Dispatcher Cluster Information. There is an admin command which is sent by Dispatchers to CM resulting from a running state change of a particular service. This occurs when an Administrator uses Cognos Administration to start or stop a particular service on a Dispatcher. Once the admin command is sent to CM the master copy of the Dispatcher Cluster Information will become updated and propagated to other Dispatchers of the system by the keep-alive queries.
The Dispatcher Cluster Information of a specific Dispatcher can be viewed by using the following URL: http://<INTERNAL_DISPATCHER_URI>/p2plbDiag
The user accessing this URL requires the “canUseAdministrationPortal” capability.
Figure 2 - Cluster View output of /p2plbDiag URL
Load Balancing is a concept which can be applied whenever there are multiple instances of the same IBM Cognos BI service available within a given Server Group. Instances of the same service in different server groups are not eligible for load balancing.
Not all requests are subject to load-balancing though, the following sections will have more on this.
Load-Balancing is controlled by the “Load Balancing Mode” setting of a Dispatcher (refer to Figure 1). It can be set to either “Weighted Round Robin” or “Cluster Compatible”. As a best practice all Dispatchers in a system should use the same mode to prevent unexpected results. It's not a sharp requirement though, if one is knowledgeable about the implications it is technically possible to have different settings on different Dispatchers.
A mode of “weighted Round Robin” implies that IBM Cognos BI Dispatchers should route applicable requests. The routing will facilitate a variant of the Round Robin method to distribute requests among qualifying (in the same server group) instances of the requested service. While Round Robin is a simple straight forward assignment of requests, it doesn't respect possible differences between the service instances. Since IBM Cognos BI is based off an SOA, those instances could, and likely are, located on different physical boxes, which could have very different resources available to them (amount of heap, number of CPUs). This leads to the understanding that not all instances of a service are equal in their potential capacity. A service running on a multi-core CPU box with lots of RAM available might be able to handle more load than a service running on a single CPU box with 2GB of RAM. To account for this, a weight has been added to the Round Robin algorithm. This weight impacts how many requests a particular service instance will obtain. Actually, the weight is assigned at Dispatcher level and not at the service level, so it applies to all services hosted by that Dispatcher. This is reasonable as they all share the same hardware. The weight is defined in the property Processing Capacity (refer to Figure 1) of a Dispatcher. With this capacity taken into account Round Robin works like this:
Example: 4 Dispatchers with different Processing Capacity's defined.
- Dispatcher A, Weight 1
- Dispatcher B: Weight 2
- Dispatcher C: Weight 1
- Dispatcher D: Weight 4
Requests will be assigned to Dispatchers in this order: A, B, B, C, D, D, D, D, A, B, B, C, D, D, D, D, …
This is only true statistically though, there is no strict assignment according to the sequence outlined above. Over time, on average though, the above distribution of requests will be achieved.
As of IBM Cognos BI 10 there is an additional tuning parameter for the Weighted Round Robin algorithm called the In-Process Request Factor (IPRF). This factor lowers the weight of a Dispatcher with every request it is currently processing.
The IPRF is disabled by default and has to be enabled explicitly by specifying it as an advanced property to either a Dispatcher (applies to all services registered to this very Dispatcher) or a Service (applies to requests for this service only). Best practice is to use a value of 2.0, a value of 0.0 implies no IPRF. The Administration and Security Guide provides details on how to specify this parameter.
The reason for adding this factor on top of Weighted Round Robin was that there is no notion of how “busy” a given Dispatcher, to be precise the services registered to it, are and thus how much of the (hardware) resources available to the services are in use. The Processing capacity setting only indicates a potential maximum. If there are several requests being processed by services registered to a given Dispatcher already, it may be unreasonable to assign another request although Weighted Round Robin might have calculated to it. The IPRF allows to take this into account.
The second possible value for load balancing mode is “cluster compatible”.
In “cluster compatible” mode the load balancing is trumped by the fact that priority is given to local (to the Dispatcher which is currently handling the request) instances of the requested service. Whenever a request for service X arrives, the Dispatcher will try to assign it to a local instance of that service first. Only if that fails, the fall-back of weighted round robin will be used. Therefore this setting gives priority to local instances and ignores capacity which is only evaluated in a later step.
One would want to enable this mode when using external load balancing by either hardware or software to distribute the load between nodes (Dispatchers).
For example this is recommended when deploying IBM Cognos BI in an IBM WebSphere cluster. In that case IBM WebSphere software components decide to which node/Dispatcher a request is routed. However, that routing is not based on IBM Cognos BI service availability or load but simply based on Dispatcher availability and load. Since in a cluster all nodes are expected to be identical they should all run the same services and hence it would be counterproductive to re-route the request again. Never the less, if a service doesn't exist locally the fall-back of weighted round robin is still applied.
Conversations and Affinity
One of the most important concepts to be aware of for IBM Cognos BI is its use of contextually related sequences of client requests and Service responses to implement a certain operation. These sequences are called conversations.
A most prominent example of a conversation in IBM Cognos BI is the interactive execution of a report. This usually starts by a click on a report in Cognos Connection which triggers the execution. Next several prompts could be presented to the client, for selecting a data source, for providing prompt values. Once completed the report is executing, a browser client will receive execution updates through the “your report is running” page. Once the report has completed running, the user can for example browse through the report, given it's HTML output format. The overall operation is report execution which consists of several steps. During the course of this conversation the client sends multiple requests and receives multiple responses.
A conversation always starts with a primary request. This request starts the operation and is destined for some particular service. After the initial response a client can send secondary requests which drive the conversation. The important thing is, that almost obviously these conversations perform best, if all requests, primary and all secondary requests, of a conversation get served by the very same instance of a service. This is because of the context conversations may create. To refer back to the example of the report execution, this would transfer to a data base connection and cursor to be opened, some data cached locally etc. Yet, if for whatever reason a secondary request of a conversation cannot be handled by the very same service instance which handled the previous requests of that conversation, it will not fail. There is rather an overhead of re-establishing the same conversation context at another service instance. Again, reflected to the example used before, this implies opening another data base connection, re-running the SQL, reading response data to eventually get to the same state in the conversation. In summary, secondary requests will work on any instance of the requested service but at a cost of lowered overall performance since the system needs to repeat work done previously.
This makes routing conversation requests a priority for the Dispatcher routing! A measure for how important it is to route a request to a particular instance of the requested service, and thus a particular Dispatcher, is called request affinity.
A request's affinity depends on the Service it's destined for and whether it's part of a conversation or not. Affinity is set implicitly by the IBM Cognos BI system following three simple rules:
- requests to Content Manager Service are absolute affine
- primary requests of a conversation have no affinity
- secondary requests of a conversation have high affinity
The affinity is stored in a value which is part of the SOAP action header of a request. There are 5 different values of affinity in IBM Cognos BI:
- (none) – Non affine, sometimes labeled “low” affinity
Request can be routed to any available instance of the targeted IBM Cognos BI service.
- high – High affinity
The request should be routed to the service instance running at the Dispatcher specified in the nodeID parameter. That parameter is mandatory for high and absolute and control affinity requests and hence will be set in the request's SOAP header. If the requested Dispatcher is not available then the request will be treated as being non-affine.
- session – Session affinity
This is the same as high affinity except the nodeID is optional. If no nodeID is specified, it is treated as non-affine.
- absolute – Absolute affinity
The Dispatcher must route the request to the service instance running at the Dispatcher specified by the nodeID. If the specified Dispatcher is not available, the request will fail and a SOAP Fault is returned.
- control – Control affinity
Same as absolute affinity, however this is reserved for system operations involved with executing reports such as cancelling a report.
The IBM Cognos BI Dispatchers will do their best to adhere to the preference expressed by affinity, it is explicitly acknowledged by the routing process to be detailed later.
Going back to conversations now, there is another feature of conversations to be described. When a request cannot be served within a configurable time limit, a conversation can become asynchronous (“async”).
After the client sent the request, it will wait for a result thus locking up the client and all involved resources at client and server side. This is reasonable for some short amount of time (several seconds) only. If the target service cannot provide a result at this time it will respond with a “still working” type message and the client switches over to asynchronous mode. In asynchronous mode, the client must send regular “heartbeat” requests to drive the conversations forward. If these heartbeats don't reach the service instance processing the request, it will assume the client has shut down. This will terminate the conversation and the target service will quit processing, all resources at client and server side get released. This explains why heartbeat requests are secondary requests using absolute affinity. The same happens when the client sends a cancel command, only that it uses a secondary request of control affinity. If the conversation is driven forward as expected, the client can process other things in between the heartbeats. The same applies to the resources involved with the conversation on the server side like Dispatcher threads. Overall this allows for better overall system performance.
Once the target service finally completes the processing of the request its status changes to “ready to provide result” and the client can now retrieve the result by sending further secondary high affinity requests. Once the overall operation is complete, all results have been retrieved, the client will send a last secondary request to gracefully terminate the conversation.
Overall conversations yield many benefits. As described, the conversation concept allows for interaction at every sate of a conversation, the client can chose to end the conversation or drive it further. At the same time conversations help to save overall system resources through asynchronous mode and client availability awareness. Last but not least conversations help to avoid hitting client or network time-outs when executing lengthy operations.
There is one more reference to make. It was mentioned before that not all requests are subject to load balancing. With the concepts of conversations and affinity introduced now, the explanation to that previous statement is as easy as: Load balancing only applies to non-affine requests, in other words primary requests of a conversation. One will see this reflected in the routing process to be described shortly.
In a previous section the concept of Server Groups was introduced. They are the basis for an important customization feature to the overall routing concepts discussed next.
Every Dispatcher is part of a Server Group at any point of time. By default there's only a single, implicit default Server Group which all Dispatcher in a IBM Cognos BI system are a member of. IBM Cognos BI Load Balancing only happens within a Server Group, which means that under normal circumstances, once a request has logically been assigned to a Server Group, there is no way it's going to leave that very Server Group.
For example consider the ReportService used for interactive report execution. All instances running on Dispatchers with the same Processing Capacity value are considered equal regarding the resources they offer. In addition there is no way to account for possibly differences regarding data base connectivity. This however may not be sufficient for all use cases.
One might want to ensure that certain report executions get processed by particular Dispatchers. Some reasonable examples are:
- Reports executed by executives should be processed on dedicated hardware.
- The system might be geographically distributed. Reports called by clients from a particular office should be processed by geographically local/close servers to minimize network latency and traffic.
- A certain type of database connectivity only exists/doesn't exist on particular Dispatchers. For example, if a given system has mix of Dispatchers running on Windows and UNIX, those ReportService instances hosted by the UNIX based Dispatcher could not access Microsoft SQL Server as there is no supported database client for SQL Server on UNIX. One has to ensure that reports run against SQL Server get processed by an instance of ReportService hosted by a Dispatcher running on Windows.
- Reports against IBM Cognos PowerCubes require the cube files to be available locally. Instead of copying them to multiple machines or use network storage one could desire to process all reports based on particular cubes on machines which have the file available locally.
By Advanced Routing one can route non-affine requests for most Services to specific Server Groups based on so called Routing Rules. Once they are assigned to a Server Group they are subject to Load-Balancing in that Server Group.
The restriction to “most Services” applies because not all Services support Advanced Routing. Amongst those who do are all services which execute queries against defined data sources like (Batch-)ReportService, PowerPlay Service, (Relational) Metadata Service and Query Service. (N.B. some of the named Services only exist as of IBM Cognos BI 10.1 or 10.2).
Routing Rules map so called Routing Sets to Server Groups. Routing Rules get defined globally for the overall IBM Cognos BI system. They get stored in a single list and are managed by Content Manager through Cognos Administration.
Figure 3 - "Specify Routing Rules" Button in Cognos Administration
Matching of requests against the Routing Rules is handled by CM (not Dispatcher!). The rules are parsed in sequence and the first match fires and stops further evaluation of the rules. To emphasize, although a request may match rule #1 and #3, it will always be treated according to rule #1 as the first match fires.
If a request should be subject to Advanced Routing, then the client which send the request is responsible for calling CM to evaluate Routing Rules and put the Server Group Information into the request passed to a Dispatcher. It's important to note that in this context “client” really means client service. This is because whenever a request for execution of a query against some defined data source is sent to the executing service it will have been sent by another “client” service.
For example, if a report is called interactively via Cognos Connection it will be the Presentation Service (XTS) which queries CM to learn whether the report destined for execution is subject to any routing rules. If it is, CM Service will have responded with a Server Group name which Presentation Service will then add to the request it passes to its local Dispatcher for processing. For scheduled reports the same role of client service would be adopted by the Monitoring Service (MS). There are many other similar scenarios, in every case though there will be a client service which is responsible for populating the target Server Group attribute of a request. The Dispatcher simply adheres to that, it does not add or evaluate Routing Rules.
Figure 4 - Definition of Routing Rules in Cognos Administration
Finally, the definition of which query executions to apply Advanced Routing to is defined by Routing Sets. A Routing Set is a name, a label applied to a set of objects of a certain type. There are three types of Routing Sets, distinguished by the type of objects in the set:
- Package Routing Set:
Is composed of Packages. Report executions based on data from the defined packages will be routed to a specified Server Group.
- Group Routing Set:
Is composed of security Groups. Report executions run by a user who is a member of one of the groups in the set will be routed to the specified Server Group.
- Role Routing Set:
Is composed of security Roles. Report executions run by a user who is a member of one of the Roles in the set will be routed to the specified Server Group.
A Routing Set can contain elements of a single type only. It is not possible to define a set containing a package, two groups and a role for example. These would have to be configured as three different sets.
An Administrator adds objects to a set by specifying the name of the set in the respective section of the objects properties. If a set of this name does not exist already it will be created implicitly, like Server Groups definitions. Refer to product documentation for details on how to assign Groups, Roles or Packages to a routing set.
Requests will be checked against the defined Routing Rules by CM based on request attributes. If a matching rule is found, CM will return the specified Server Group to the client service which is supposed to add it to the request's attributes. Dispatchers will route the request to any Dispatcher in that specified target Server Group. In the target Server Group Load-Balancing may take place and finally the request is assigned to an instance of the requested service. If there is no instance of the requested service available, the request will fail with an error stating that there is no service available to handle that type of request.
Advanced Routing is very versatile and can greatly improve overall load handling or traffic shaping by routing to dedicated resources.
The Routing Process
Finally, now that we introduced all the required concepts, we can define the routing process.
Note: This description is conceptual and does not necessarily describe the exact rundown of steps for each action. To comprehend the concepts described in this document this is sufficient though.
Step 1 - Identify Handlers
Dispatcher processing is driven by a static list of so-called handlers. When Dispatcher receives a request, it queries each handler to find out whether it can handle the received request. There are handlers for handling features like authentication sequences, load balancing, etc and services like Presentation Service. Without going into detail, it's sufficient to comprehend that each handler defines a sequence of steps, possibly involving calling other handlers, processing a received request. Overall, the sequential probing of each handler implicitly builds an execution plan for Dispatcher on how to handle the request, that is run it by every handler which signaled it will handle this particular request.
As an example consider one of the first operations Dispatcher runs when it receives a request: verifying whether the session this request is a part of is already authenticated. If it isn't then the authentication handler will be used. This handler first saves the original request then runs through a number of steps to drive authentication and only after if that completes successfully, the original request is retrieved back allowing other handlers to resume processing of the original request.
For the next steps it's safe to assume that the session has been authenticated.
Step 2 - Identify Target Service
The next step will be to identify the target service which can handle the request. The target service is identified based on the following information in the order presented.
- The SOAPAction HTTP header
In this case the SOAPAction header will contain references to specific schemas which imply the service to use.
- Service Mappings
Each dispatcher has a static list of SOAPAction -> Service mappings from which to derive the target Service.
- The “b_action” URL parameter
The HTTP GET or POST command to the Dispatcher can contain the b_action parameter, its value will indicate the target service.
E.g. b_action=xts.run implies the Presentation Service (XTS)
- The PATH_INFO for the request
Some URLs' path indicate a target service.
E.g. /cgi-bin/cognosisapi.dll/gd/… indicates a specific action (retrieving a pre-rendered chart or Diagram) which is handled by a specific handler/service.
- If the request specifies none of the above, the request is forwarded to the IBM Cognos Connection home page, which implies Presentation Service.
After this step, the request is authenticated and the target service is identified.
Step 3 - Handle Content Manager Requests
If the target service is Content Manager Service, the request is routed to the active Content Manager, regardless of any Advanced Routing, Server Groups or Load-Balancing. This is crucial for the product to work.
Step 4 - Handle Absolute and Control Affinity Requests
In case the submitted request has absolute or control affinity it's obsolete to perform any advanced routing or load balancing. The request is forwarded to the Dispatcher indicated in the NodeID of the request. If that fails, the request fails and an error is written to the log and possibly pushed back to the client.
Step 5 - Deduce Cluster View
In this step the Dispatcher constructs a set of potential target Dispatchers running available instances of the requested service. This set is called “Cluster View”.
If the actual request is tagged, that is it specifies a target Server Group because it's subject to Advanced Routing, the Cluster View is derived from the Advanced View and thus will contain only Dispatchers from the target Server Group.
If the request is untagged, which means it does not specify a target Server Group, the Cluster View is derived differently, depending on the actual product release.
- For IBM Cognos BI prior to version 10.2.1
the Cluster View is based on the Advanced View and only contains the Dispatchers from the “local” Server Group, the same Server Group the processing Dispatcher is a member of.
- For IBM Cognos BI as of version 10.2.1
the Cluster View is derived from the Simple View which means it contains ALL Dispatchers from the default Server Group, which essentially means all Dispatchers of the system.
The implications of this subtle difference will be explained shortly.
Step 6 - Respect affinity
Next Dispatcher will try to respect affinity. The absolute and control affinity requests were handled in a previous step so any request dealt with in this step can only have high or session affinity. The Dispatcher will forward to the requested Dispatcher if it's within the Cluster View constructed in the previous step. If this fails, because either the requested Dispatcher is not in the Cluster View (unlikely) or the requested Service is not available on that Dispatcher due to load (some other request could have used up all free resources in the meantime) or failure, then the request is simply demoted to non-affine. Processing continues with the next step which effectively allows some fault tolerance.
Step 7 - Process Load-Balancing
Now that the Cluster View has been established and only non-affine requests remain to be assigned Load-Balancing is evaluated.
First of all, the load-balancing mode is checked.
If it is set to “cluster compatible” the Dispatcher will try to assign the request to a local instance of the requested service, which however must be within the Cluster View, thus ignoring load balancing. This means, that although the request could get load balanced a local instance would be given priority. This still respects Server Groups. The Cluster View only contains the Dispatchers form the deduced Server Group. If the Dispatcher processing the request is not within the Cluster View the assignment will fail. Same is true if the local Dispatcher doesn't offer the requested service. In case of failure to adhere to cluster compatible, the fall-back is used, which is to process the request according to weighted round robin like any other request.
Note: Using Server Groups when the load balancing mode is set to “cluster compatible” is counterproductive though. Clustering expects the nodes of a cluster to be identical. Server Groups are a concept to model the exact opposite, technically this will work though.
Eventually, if load balancing mode is set to “weighted round robin” or the assignment due to “cluster compatible” failed, the request is load-balanced amongst the instances of the requested service offered by Dispatchers within the Cluster View.
If there is no instance of the target service available at all, the request will fail and an error message is returned to the client indicating that there's no target service available in the configured Server Group.
Appendix A – Known Challenges
Routing Change in IBM Cognos BI 10.2.1
IBM Cognos BI 10.2.1 introduced a subtle change in the routing behavior which will go unnoticed by most but which may lead to unexpected behavior for systems using Advanced Routing.
Effectively the change implies, that untagged requests will be load-balanced across the whole system, instead the local Server Group only.
This makes no difference for systems which did not add any explicit Server Group because for those the Advanced View and the Simple View are identical. For systems which defined at least one additional Server Group, which is only required when aiming to use Advanced Routing, the change can lead to a different behavior though.
Before version 10.2.1 untagged requests, which means either those not subject to Advanced Routing because they did not match and Routing Rule or those requests which have been demoted due to a failure in affinity adherence, would remain in the Server Group they have been sent to. Assuming in such a system, there are at least two Server Groups, one defined explicitly plus the default Server Group, a request could potentially hit any Server Group. For example the system uses two externally load-balanced Gateways, each offloading requests to a Dispatcher in a different Server Group. If a request came in to the default Server Group and remained untagged, because none of the defined Routing Rules applied to it, it would never get load-balanced to any other Server Group by IBM Cognos BI following the “what hits a Server Group, stays in a Server Group” mantra. This was a behaviour which , although not explicitly designed, was leveraged by several clients to control the processing of those requests.
Now as of IBM Cognos BI 10.2.1 the behavior changed, so that in the above example the untagged request would be load-balanced across the whole system, potentially ending up being served by a service instance from any other Server Group. To project on a use case example: although the request came in to the “non-privileged”, low resource Server Group it could be served by a service from the “privileged”, high resource Server Group now.
With this change it becomes even more apparent, that the Best Practice for using Advanced Routing should be followed which is to define Routing Rules such, that EVERY request is subject to Advanced Routing and effectively becomes tagged.
In practical terms, this might translate to defining a catch-all Routing Rule as the last entry in the Routing Rules table which applies to all requests which have not been matched so far. With such a catch-all in place, the behavior in 10.2.1 is exactly the same as for versions before IBM Cognos BI 10.2.1.
FRONT Dispatcher requirements
There is an infamous behavior which is quite often is perceived as an issue or defect in setups where single Sign-On has been configured and clients access Dispatchers directly without a Gateway.
Typical examples of setups where the Gateway component is omitted are those where IBM Cognos BI is deployed to an application server like IBM WebSphere and web tier application server plug-ins are used. Another example is when for reasons of product conformance (no suitable Gateway available like lack of 64bit MOD implementations) or infrastructure requirements hitting the Dispatcher for entry point is reasonable. Often external solutions take over the load balancing task which balances the client requests between multiple Cognos entry points either.
In those setups on every n-th access or even sporadically the SSO fails and XML is returned to the client instead. This is however the result of current product design and solvable by a simple configuration change. One needs to remind of the definition of a front Dispatchers mentioned earlier.
What happens is (assuming default configuration values for Dispatcher Servlet Context-Root) this:
- Request to http(s)://<host>:<port>/p2pd/servlet/dispatch" hits the
Dispatcher. This would be the arbitrary URL being used by an initial request of a new
The Dispatcher accessed is therefore the FRONT Dispatcher.
- Since there is no other indication of which service is being called, this request is assigned to XTS (Presentation service).
- This is a primary request (not a secondary in a conversation) with no affinity which is therefore subject to load balancing.
- Assuming default configuration, “round robin” is set for load balancing mode.
Now if there is more than one instance of Presentation Service available in the current Server Group (by default all Dispatchers are members of the default Server Group), the request might be sent to any of the Dispatchers in that very Server Group hosting one of the available instances of Presentation Service. This implicitly implies it can very well be sent off to a remote instance on a different Dispatcher!
- The handling Dispatcher (the one which hosts the instance of XTS which the request got assigned to) checks for an authenticated session. The client session is yet not authenticated therefore the authentication sequence is triggered which includes passing the request to Content Manager Service. The original request is saved (like pushed to stack).
- The request plus some extra info is sent to CAMAAAsyncService (as of IB Cognos 10, in previous versions it would have been sent to CMService).
- CAM-AAA determines which authentication provider to pass the request to. Either based on an URL parameter (CAMNamespace) or based on the fact that there is only a single active authentication provider available for authentication.
- The request is assigned to the authentication provider identified to handle the authentication in the previous step. The provider checks for the presence of logon data sufficient to process the authentication request with the configured provider. This will come out negative, otherwise there wouldn't be an issue.
- The authentication provider now triggers SSO which means it has to throw a "SystemRevoverableException" which technically is a response using HTTP status code 599 and some specific data.
- The response is returned to the sender, which is the handling Dispatcher from step 5.
NOW: If the handling Dispatcher is not the same as the FRONT Dispatcher being accessed (in 1.) what happens is
- The handling Dispatcher deducts it is not the FRONT Dispatcher for this session which received the original request. Therefore it is not responsible for handling the response received from the authentication provider (step 9.). Instead it will transform/leave the response in it's original format (SOAP, which is XML) and pass it over to the actual FRONT Dispatcher for this session.
- The FRONT Dispatcher considers the response received from his fellow Dispatcher to be a normal request response which it therefore is supposed to pass to the client; which it does.
This is why the XML appears in the browser clients (because they know how to render it) or being returned to other client types (clients like Framework Manager).
The desired behavior is only achieved if the handling Dispatcher (that is the one hosting the instance of Presentation Service which handles the request) is the same as the one being accessed (the FRONT for this session) in 1. In that case what happens is
- The handling Dispatcher deduces it is the FRONT Dispatcher for this session and therefore responsible for handling the response.
- The Dispatcher investigates the response code for anything special. It finds response code HTTP 599 which triggers special SSO handling code to be run.
- The SSO code investigates the SOAP (XML syntax) request and finds the
SystemRecoverableFault in here. This will invoke the matching handler to act on this.
(will only do that for XTS and CM Service requests).
Within this handler, Dispatcher reacts to the SystemRecoverableException by first retrieving an environment variable value from its local CGI environment and then repeating the authentication request to the authentication provider, now carrying the retrieved variable value in a signed parameter.
- Once the authentication provider receives that 2nd request with the attached variable value, authentication will be run. If successful response contains the requested content and the cam_passport cookie and session identifiers are set as cookies. If the authentication fails an UnrecoverableException is thrown by the authentication provider which will "bubble up" like the first response as described below. Dispatcher will catch that exception and revert the client back to the login page in that case.
From above explanation it evolves that to prevent the XML responses to be send back to the client one has to ensure that initial requests for an unauthenticated session get handled by Presentation Service instances on the FRONT Dispatcher for the session only. Or in other words
- Presentation Service must be run on a FRONT Dispatcher
- Initial requests of a session must be processed by a local instance of Presentation Service on the FRONT Dispatcher of that very session. They must not be routed “away”.
This can be achieved through either of these two configurations
- Ensure there is only a single FRONT Dispatcher in a particular Server Group which runs the only instance of Presentation Service in that Server Group.
- This approach can easily be adopted to define additional Server Groups explicitly in which a single Dispatcher is put which adheres to the above requirement. That way, one can point out specific entry points for specific types of clients as well, e.g. one entry point Dispatcher for Framework Manager clients, a particular set for browser clients etc.
- Implicitly this means, that if no Server Groups have been defined, and multiple instances of Presentation Service exist, the issue will surface without the required configuration change because all Dispatchers are pooled in a single Server Group.
- In case of multiple FRONT Dispatchers in a Server Group, configure the load balancing
mode to be “cluster compatible” on those FRONT Dispatchers.
Mind that if any additional Services over Dispatcher Service, Presentation Service and Monitoring Service which form the very least set of Services a FRONT Dispatcher should run, are configured, the cluster compatible mode applies to those as well, which means local processing trumps load balancing when possible.
Dispatcher Services instances
When Server Groups get defined it's important to ensure that each Server Group runs at least one instance of Dispatcher Service. Otherwise requests may not be routed correctly.
This can happen if, for example, a Server Group is defined which only contains a single node which is an install of a Content Manager component only. By default a CM only install does not run the Dispatcher Service.
The best practice is to define Server Groups in such a way that only nodes which run report executing services like (Batch-)ReportService, Query Service (as of IBM Cognos BI 10) and PowerPlay Service get added to explicitly defined Server Groups and all other nodes remain in the default Server Group, if applicable.
64bit Report Server Execution Mode
As of IBM Cognos BI version 10.1. Refresh Pack 1 (10.1.1) there is a new feature which allows to configure the Report Server execution mode to either 32bit mode or 64bit mode. This basically determines, whether executions for reports based on relational data source packages are handled by the compatible query mode (CQM - legacy, same as in 10.1, C++ 32bit executable) or the new dynamic query mode (DQM - 64bit, Java engine).
Regarding routing it's important to remember, that although DQM is configured, request routing is based on the availability of the (Batch)ReportService and not the Query Service. The latter is not a target Service for load balancing but rather a secondary Service to the (Batch)ReportService. Only if a local instance of ReportService is available, a co-located Query Service will be employed.
For all background processing the central Service is the Monitoring Service (MS). Opposed to active load-balancing by Dispatcher the mechanism for background processing is called “auto-load-balancing” and works in a “pull” fashion.
An instance of Monitoring Service will investigate the system global task queue and match items in there to the locally available so called “target services”. Only if a matching target service is available on the very same Dispatcher (locally) the Monitoring Service will probe if that instance of target Service has yet capacity to take on another task. This is determined by the number of unallocated affinity connections for that service. Their number can be configured in Cognos Administration on a per service level. If there is yet a free connection the Monitoring Service will pull the task from the queue, set up the controls to overlook the execution and pass it to the target service for processing.
That is, only if the Monitoring Service is co-located on the same Dispatcher as for example an instance of Agent Service that very instance of Monitoring Service will take the task item from the global task queue and engage to run it. Implicitly that means, that the Monitoring Service is required to run on a Dispatcher so that the target Services can get assigned work. Target Services are
- Batch Report Service
- PowerPlay Service
- Agent Service
- ContentManager Service
- Job Service
- Index Update Service
- Delivery Service
If they run on a Dispatcher which doesn't run Monitoring Service, they won't participate in working on the tasks.
Another crucial task of the Monitoring Service is to provide data for the Presentation Service when rendering the lists of tasks actually executing, pending etc. Yet, Monitoring Service is accessed through conversations like other services as well and it is subject to load balancing.
For that reason the best practice is to have Monitoring Service enabled on basically every Dispatcher. It won't take away noticeable resources unless it's actually controlling the execution of background processing which it can only do if there are active instances of target services hosted on the very same Dispatcher. This for example implies that running Monitoring Service on a “routing” Dispatcher which only runs the minimal Services like Dispatcher Services and Presentation Service won't do any harm. At the same time Monitoring Service is required on a Content Manager install component only to feed the Content Manager Service background tasks like Deployments. If the Monitoring Service, which is enabled on those installs by default, is disabled, deployments will fail.
Regarding routing, the Monitoring Service should only be disabled on a Dispatcher if there is no target Service and no Presentation Service on this Dispatcher.