Introduction to the project

This Software Group incubation project, codenamed "Botticelli," focused on the front office within capital markets. The front office involves the buying and selling of securities by traders. This project implemented a simple algorithmic trading use case. Algorithmic trading refers to the programmatic placing of buy and sell orders. In algorithmic trading, a quantitative model automatically generates the timing and size of the orders based on certain parameters and constraints. These trading environments require a high-speed infrastructure that can handle large volumes of data to optimize the firm's trades. In the project scenario, the algorithmic program analyzes and processes live market data in conjunction with other reference data and generates orders to buy or sell securities.

The front office infrastructure needed to support algorithmic trading is different from the mid- and back-office infrastructures. The latter are more business process-driven, with guaranteed delivery of messages being a critical requirement to ensure that none of the orders are lost.

Within an algorithmic trading scenario, there are a number of steps that must be executed, and we applied IBM technology to demonstrate each of these steps.

Back to top

Algorithmic trading process requirements

Algorithmic trading requires an infrastructure that provides high-speed access to large volumes of data. The trading platform typically requires various capabilities, including the ability to:

• Accept streams of incoming market data
• Distribute market data to applications/processes in microseconds
• Analyze market data in real time
• Pre-load reference and historical data
• Support business rules
• Provide access for human traders to the market data and other financial information
• Monitor the health of the algorithmic trading environment

Back to top

Botticelli scenario

The Botticelli scenario depicted in Figure 1 represents a simplistic algorithmic trading use case.

The flow is as follows:

• The NYSE and NASDAQ market data is processed through feed handlers, and is made available to subscribers.
• Two subscribers receive the market data— the bargain index algorithmic trading program and the data hub.
• Reference data and threshold parameters are loaded into memory to be used throughout the trading session by the algorithmic trading programs.
• The bargain index program receives the market data over a high-speed connection and does the following:
• Retrieves reference data, such as earnings per share, analyst ratings, and configuration parameters from memory.
• Calculates volume-weighted average price for the trades.
• Performs calculations and generates the orders that it has identified as bargains.
• Determines if the order is an exception (exceeds a threshold on size).
• Sends non-exception orders through the data hub to be routed to the execution venue.
• Invokes business rules to determine to whom to route exception orders.
• Routes the exception orders through the data hub to the appropriate trader.
• The trader desktop subscribes to the data hub and displays:
  • A watch list of stocks (Dow 30 for this scenario), various financial widgets, and exception orders for the trader.
• The monitoring dashboard subscribes to the data hub and displays:
  • Multi-dimensional views of the order data, order status, output from the bargain index program, system latency information, and exception order breakdown.

Back to top

Financial markets front office blueprint

Figure 2 illustrates the suite of IBM middleware that was integrated to provide the capabilities defined in the previous section.

Let's take a look at each of the IBM technologies and products that are being used for these capabilities. Each product described below is associated with one of the lettered components (for example, "A") in Figure 2 .

IBM WebSphere® Front Office (A)

WebSphere Front Office provides over 95 feed handlers covering the major North American, European, and Asian equities feeds, the largest options feed, and some futures and commodities feeds within North America. The feed handlers understand how to interpret the incoming data, normalize it, and make it available to subscribers. The data distribution feature provides various ways to distribute the data, each involving different qualities of service.

WebSphere MQ Low Latency Messaging (B)

WebSphere MQ Low Latency Messaging is a low-latency distribution bus that addresses the needs of the financial markets trading infrastructure. It supports features such as high-performance messaging, multicast and unicast transports, component state replication, fault detection and failover, and consistent ordered delivery.

Low Latency Messaging recently participated in a Securities Technology Analysis Center (STAC) performance benchmark. In terms of speed, at a rate of 50K messages per second, STAC benchmarked WebSphere MQ Low Latency Messaging at eight microsecond single hop latency on Infiniband. And in terms of throughput, in internal performance tests, WebSphere MQ Low Latency Messaging was benchmarked at being able to handle over 13 million, 45-byte messages per second on InfiniBand.

IBM InfoSphere™ Streams (C)

InfoSphere Streams is a high-performance stream processing technology that can be used to rapidly analyze data, news, and video in real time as the data streams from thousands of real-time sources. The Streams platform can analyze structured and unstructured data and can scale to over 125 node (server) instances. Within Botticelli, we implemented the bargain index as a program running on the Streams platform.

Figure 3 depicts the runtime view of the bargain index program. This program is implemented as a one-way flow of data tuples. It starts with the input of market data from WebSphere Front Office, and then splits the data into trades and quotes. The trades are enriched with additional data, and the volume- weighted average price calculation is performed. Orders are eventually generated, and the output is sent to the data hub over an WebSphere MQ Low Latency Messaging connection. The program was created using a stream-processing language that supports declarative composition of operators, which are its basic building blocks. Streams provides code generation for high performance and platform independence. Because of these features, Streams can be a very good technology for an algorithmic trading platform.

IBM solidDB® (D)

solidDB provides in-memory retrieval of data using specialized access methods. These access methods result in data access and storage up to ten times faster than those used to access cached data from traditional disk-based database systems. Within Botticelli, we preloaded the reference data that the algorithmic programs would need into solidDB so that accessing that data in real time would be as fast as possible.

WebSphere ILOG® Business Rule Management Systems (BRMS) (E)

ILOG technology specializes in business rule management, optimization, and visualization. The ILOG BRMS is called JRules. JRules provides the ability to centralize rule management and administration within an enterprise, providing auditability and traceability, which are critical requirements in the financial markets industry. For this reason, we use JRules to handle the routing of exception orders to traders for review.

Within Botticelli, we invoked routing rules created in JRules to determine where to route exception orders. These are algorithmically generated orders that exceed thresholds set on order size. The next article in this series addresses how we integrated Streams and ILOG. (To listen to a recorded ILOG demo that shows how JRules can be used in a Financial Markets firm, see Resources.)

Botticelli also used ILOG visualization widgets within the trader desktop. IBM ILOG JViews Enterprise provides a wide range of graphical interfaces that can be integrated into desktop, Ajax, and Eclipse displays.

WebSphere eXtreme Scale (F)

WebSphere eXtreme Scale is a caching and grid technology that we utilized as a central data hub for the data that we needed to make available to client applications. We cached the market data from WebSphere Front Office, the order data from Streams, and the historical data required by the trader desktops.

We created different receivers that could process the incoming data over multiple types of messaging protocols and made that data available through a subscription manager.

Desktop clients: IBM Lotus® Expeditor and WebSphere sMash (G)

There are different requirements for client desktops based on the volume and frequency of data changes. We created a desktop application that exhibits very low latency using Lotus Expeditor and a Web-based application using WebSphere sMash (that can be accessed by any Internet browser).

Lotus Expeditor is a server-managed client platform that provides a single container to integrate desktop applications, Web applications, and legacy applications. You can contextually link the content of the applications through the use of a property broker. In Botticelli, we also integrated the ILOG JViews visualization widgets for our charts and graphs.

WebSphere sMash is a development and execution platform for quickly building and running dynamic Web 2.0-based applications using SOA principles. It supports scripting languages such as PHP and Groovy, and provides an agile Web application development environment.

IBM Cognos® Now! (H)

In addition to the trader desktop, we identified a need to provide application monitoring capability. Trade desk managers, who are responsible for a trading desk, are interested in:

• How many orders are being generated
• The order breakdown by sector/symbol/time
• How the data feeds are running
• What the order status is

Cognos Now! has a streaming engine that allows it to process feeds of data in near real time. In our scenario the order data is sent from the algorithmic program into the Cognos Now! dashboard. Multi-dimensional views of the order data are presented on the dashboard in a variety of formats. The Alert Manager capability allows the business users to identify data conditions that they would like to be alerted to. Watch points can be used to highlight data conditions within the dashboard tables.

A future article in this series describes the integration between Cognos Now! and WebSphere MQ Low Latency Messaging as part of this project.

Back to top

Integration points

There were multiple integration points that were addressed as part of this effort. As WebSphere MQ Low Latency Messaging is the primary messaging bus, the products in the Blueprint had to be enabled to use it. This required the development of a number of new APIs. The code for those APIs has been contributed to the product teams. There was also work done to integrate the JRules engine with Streams, enabling dynamic refresh of business rules in a running Streams program. Future articles in this series describe how we addressed these and other integration issues. These integration points include:

• Sending WebSphere Front Office data to InfoSphere Streams
• Receiving solidDB data into InfoSphere Streams
• Sending InfoSphere Streams output to other applications using the WebSphere MQ Low Latency Messaging protocol
• Receiving data into Cognos Now! over WebSphere MQ Low Latency Messaging
• Embedding the ILOG JRules engine into an InfoSphere Streams program
• Extracting latency metrics from WebSphere Front Office, WebSphere MQ Low Latency Messaging, InfoSphere Streams, and solidDB

Back to top

Conclusion

This article provided you with an overview of some of the IBM capabilities in the financial markets industry, specifically the front office. We used an algorithmic trading scenario to illustrate how IBM middleware technology can be integrated to provide a sample solution. The same technologies can provide a framework for many other solutions in this domain that require high-speed, high-throughput capabilities.

Acknowledgements

We would like to thank the people who contributed to the Botticelli project:

• Nick Schofield, Wei Tchao, and Wayne Lee developed the prototype that integrates these products and provides a demonstration.
• Folu Okunseinde provided financial markets technical expertise and architectural direction.
• Rajiv Chodhari and Philip Enness provided business direction priorities, financial markets expertise, and customer requirements.
• Jim Caldwell made us aware of this problem domain and provided great management support.

Resources

Learn

• IBM financial markets industry expertise: Learn more about IBM financial market solutions.
  
  
• WebSphere MQ Low Latency Messaging: Learn more about WebSphere MQ Low Latency Messaging.
  
  
• WebSphere Front Office: Learn more about WebSphere Front Office.
  
  
• InfoSphere Streams: Learn more about InfoSphere Streams.
  
  
• solidDB: Learn more about solidDB product family.
  
  
• WebSphere ILOG BRMS: Learn more about WebSphere ILOG Business Rule Management Systems.
  
  
• WebSphere sMash: Learn more about WebSphere sMash.
  
  
• Cognos Now!: Learn more about Cognos Now!
  
  
• WebSphere eXtreme Scale: Learn more about WebSphere eXtreme Scale.
  
  
• "Getting Started with WebSphere eXtreme Scale, Part 1: Understanding WebSphere eXtreme Scale and how it works" (developerWorks, November 2009): Gain a technical understanding of what WebSphere eXtreme Scale is, the features it provides, and the vast benefits it offers.
  
  
• Demo: "Cognos Now! for the financial industry" (developerWorks, February 2010): See how Cognos Now! can be used by financial markets firms in an algorithmic trading scenario.
  
  
• Demo: "IBM WebSphere ILOG Business Rule Management System applied in the financial industry" (developerWorks, February 2010): Understand IBM WebSphere ILOG Business Rule Management Systems (BRMS) capabilities and how they can be applied in the financial market front office.
  
  
• developerWorks Information Management zone: Learn more about Information Management. Find technical documentation, how-to articles, education, downloads, product information, and more.
  
  
• Stay current with developerWorks technical events and webcasts.
  
  

Get products and technologies

• Build your next development project with IBM trial software, available for download directly from developerWorks.
  
  

Discuss

• Lotus Expeditor wiki: Find and contribute to information about installing, administering, and using Lotus Expeditor components.
  
  
• Participate in developerWorks blogs and get involved in the My developerWorks community; with your personal profile and custom home page, you can tailor developerWorks to your interests and interact with other developerWorks users.
  
  

About the authors

Mary has been with IBM for over 25 years. She has held a variety of positions, including programmer, systems engineer, project lead, DBA, and data quality specialist. She has been in SWG Technical Strategy for the last four years. Much of that time was focused on Software as a Service. She now administers the SWG Incubation Program. For the last 18 months she has lead a project called Botticelli, which is focused on positioning IBM middleware in the financial markets front office.

Dr. German Goldszmidt is an IBM distinguished engineer working in Technical Strategy for IBM Software Group (SWG). He is responsible for the SWG Advanced Technology program, which includes the Incubation Program and the SWG-Research Joint Program. Before 2003 he was a Research Staff Member and Manager at the IBM Research T.J. Watson labs. He led the team that developed the first compute provisioning utility (cloud) that efficiently handles varying workloads and infrastructure changes by autonomically adjusting resource allocations. Earlier, he designed and developed Network Dispatcher, a high-performance load balancer that provides reliability and scalability for multiple IBM products and deployments. He has published more than 50 papers in refereed journals and conference proceedings and holds 15 patents. Dr. Goldszmidt holds a Ph.D. in Computer Science from Columbia University in the field of network management.

Rate this article

Comments

Back to top

Table of contents

• Introduction to the project
• Algorithmic trading process requirements
• Botticelli scenario
• Financial markets front office blueprint
• Integration points
• Conclusion
• Resources
• About the authors
• Comments

Next steps from IBM

• Try: InfoSphere Streams
• Buy: InfoSphere Streams

Dig deeper into Information Management on developerWorks

Try IBM PureSystems. No charge.

Experience today!

---

Get started with the cloud-based trial and pattern development kit.

Special offers