Every programmer, systems administrator, database analyst, and so on knows what to do when they face a problem with business process or some inefficiency utilizing systems resources. They reach into their own personal, idiosyncratic bag of tricks, put together through years of experience and tuned for very specialized for problems they have already solved. While this approach has served the industry for decades, increasingly enterprise architects want better. They want sound, reproducible, evidence-based approaches to solve such problems and, even better, to anticipate and avoid such problems in the first place.
The discipline of Operational Systems Analysis, and in particular Systems Optimization has emerged to help provide such discipline. Increasingly, responsible architects are paying attention to formalized means for managing the systems that have become the lifeblood of business. More and more traditional business rules and workflow tools are growing into the formal discipline of Systems Optimization, including IBM® ILOG® (see Resources), whose approach to modeling for optimization is illustrated in Figure 1. The optimization model lists various inputs (such as demands to satisfy; available resources; cost, yields, and recipes; operational constraints and customer preferences; and business goals), one or more mathematical models and optimization engines, and a schedule or plan with metrics (such as minimized costs, maximized yields, specific resource assignments, and best timing for activities).
Figure 1. IBM ILOG optimization model
It is now very common for real-world systems to transmit messages in XML formats through various flavors of Service-Oriented Architecture (SOA). Such systems often express business rules, vocabulary, classification, and descriptive systems in XML forms. Thus frequently, an important input into optimization efforts comes from pattern analysis of XML data sets and message statistics. In support of such analysis, the XML-savvy enterprise architect can find inexpensive and flexible ways to gather basic information to support Systems Optimization. This article discusses conventions and techniques for applying the most effective XML processing practices in Operational Systems Analysis, organized around a selection of example scenarios.
The most basic inputs to any systems optimization are patterns of actual usage. What operations are used the most? Are any patterns of parameters especially heavily used? These might be candidates for optimized resources or processing. In many modern, Internet-based systems, actual usage is represented by semi-structured messages over the wire, and these days it's common to have such messages represented in XML as SOAP Web services or XML messages HTTP POSTed to RESTful endpoints. If you use SOAP-based Web Services, chances are you already have access to specialized tools that operate at the binding level, so I'll focus examples in this article to XML over HTTP in RESTful fashion, a popular system for which off-the-shelf tools are less plentiful. The systems architect needs to understand how to perform analysis in such systems, and, as a bonus, these techniques are also available for SOAP-based systems at the protocol level.
Resource reservation service
For examples in this article, consider a fictional enterprise looking to deploy a resource reservation system, where employees can send requests to reserve resources such as conference rooms and multimedia equipment. Figure 2 is a very high-level diagram illustrating how the various systems (including brower, mobile, and custom applications) interact with the Resource Manager service.
Figure 2. Systems overview for resource manager
You might be able to use network topology tools to analyze usage patterns of the various entry points (for example, to determine when the mobile interface is used more often than the regular Web interface). You can also work yourself into a more intricate analysis of usage patterns by instrumenting the REST layer. Listing 1 is an example of one of the reservation request messages issued as an HTTP POST.
Listing 1. example reservation request message
<?xml version="1.0" encoding="utf-8" ?> <reservation xmlns="http://example.com/enterprise-reservations"> <session id="115384" user="jdoe"/> <request timestamp="2010-03-02T15:39:50-07:00"> <resource id="room123" type="conferenceroom"/> <purpose>Marketing meeting, and free doughnuts</purpose> <period start="2010-03-10T10:00:00-07:00" end="2010-03-10T11:00:00-07:00"/> </request> </reservation>
Analysis using Schematron
Schematron is a good tool for such analysis. Listing 2 is Schematron to generate a report of the request timestamp and resource type.
Listing 2. Schematron to generate a report of request timestamp and resource type
<?xml version="1.0" encoding="UTF-8"?> <schema xmlns="http://www.ascc.net/xml/schematron"> <title>Reservations manager usage analysis</title> <ns uri="http://example.com/enterprise-reservations" prefix="er"/> <pattern name="Basics requests report"> <rule context="er:reservation"> <report test="er:request/@timestamp"> Request timestamp: <value-of select="er:request/@timestamp"/> </report> <report test="er:resource/@type"> Requested resource type: <value-of select="er:resource/@type"/> </report> </rule> </pattern> <pattern name="Validation"> <rule context="er:reservation"> <assert test="er:session"> Request must have a session element </assert> </rule> </pattern> </schema>
I recommend learning Schematron (see Resources), but I'll just explain the major constructs in the listing. The
element is the namespace declaration mechanism for Schematron. A
pattern is a grouping of related rules applied to XML documents. The
rule elements specify business rules about specific
constructs in the XML, given using the XSLT pattern form in the
context attribute. Within rules,
report elements are used to log constructs that meet criteria given in XPath form, and
assert elements are used to specify criteria which the schema designer expects valid XML documents to meet. As a gross rule of thumb, Schematron
assert elements express expected document patterns, while
report elements express patterns of particular interest.
If you instrument the REST API of the reservation manager with the Schematron in Listing 2, you readily accumulate a log of when requests are made and for what resources. You can use the log to optimize peak times, determine whether resources are sufficient to accommodate requests, and more.
In the previous section, you learned the basic idea of applying XML reporting to systems analysis, and focused on a core tool, Schematron, which is already the most popular way to express rules for XML document patterns. In this section, I show a variety of approaches to build on such a foundation to provide more comprehensive analysis in preparation for any aspect of systems optimization. By connecting individually simple but expressive mechanisms together, you can gather commonly-used operational statistics while retaining the power to derive your own, specialized tools which contribute to a competitive advantage.
Active systems modeling
In this scenario, the architect establishes the model of the system such that the systems team can actively monitor its actual behavior. Figure 3 illustrates the work and data flow for active systems.
Figure 3. Active systems modeling scenario
One ready tool for this scenario is Service Modeling Language (SML), a proposed standard by IBM and other companies used to model complex services and systems including their structure, constraints, policies, and best practices. SML is based on an adaptation of W3C XML schema (XSD) with embedded Schematron rules as illustrated in Figure 3. Architects and developers can build on tools to incorporate SML into systems development and maintenance such as the Eclipse-based COSMOS project.
Utilization heat maps
An important input to optimization is some rigorous understanding of what patterns of services and resources see the most actual use, similar to the user experience concept of heat maps (modern statistical tools can produce the equivalent of heat maps). Figure 4 illustrates the work and data flow for heat maps.
Figure 4. Utilization heat maps scenario
In this example, each message goes through the reporting module which aggregates a report, building on Schematron Validation Report Language (SVRL), a set of conventions for generating structured output from Schematron reports. Such output is readily transformed into direct input format for statistical analysis tools, such as IBM SPSS®.
One of the most common maintenance concerns is variations from expected protocol. Such variations might be caused by a gap in the systems model or by a unilateral change in the client layer. Figure 5 illustrates a monitoring process for such cases.
Figure 5. Protocol adaptation scenario with exception notification
If, for example, you construct the Schema using a closed grammar in XSD or RELAX NG (and possibly augmented with Schematron), you can get a closed model which should raise exceptions in case of protocol deviations. Such exceptions can be handled by notification for review by administrators or developers. A similar exception and notification approach works for managing operational compliance (such as to meet regulations).
As an example of a fairly complex analysis approach, consider what happens when more general processing exceptions occur. The system might notify an agent for special treatment. Perhaps a conflict arises about the use of meeting rooms, and the agent calls the requesting parties to negotiate alternatives. The remedy might be equivalent to modified original requests, such as new meeting room requests for the negotiated outcome. In this case, it might be useful to analyze the original request versus outcomes through intervention in order to develop a means to automate intervention and increase productivity of agents. Figure 6 illustrates such a scenario.
Figure 6. Intervention outcomes scenario
You can see many of the same workflow tools from other scenarios, but in this case, a differential analysis tool also compares the patterns of related inputs, supporting sophisticated analysis.
As Web APIs and social media penetrate the firewall directly and indirectly influence enterprise systems design, more of the information fueling systems is in formats amenable to semi-structured reporting tools, an important insight to consider as you prepare for the Monitor phase of Systems Optimization. At the same time, the globally decentralized nature of such modern systems makes it impractical to take a monolithic approach to systems analysis and optimization. Operational Systems Analysis is a very important development in applying discipline for enterprise architects, as long as they realize that it's not possible to impose such discipline upon the real world. It is very important to have a flexible toolbox that you can reconfigure and redeploy to adapt to emerging scenarios. I hope you learned from this article some possibilities, some complexities, and several ways that reporting of XML messaging streams can support continuous improvement in modern systems architecture.
- Schematron Validation Report Language (SVRL): Complement Schematron with SVRL, "a simple report language defined as part of ISO Schematron. SVRL provides a fairly full set of information from validating a document" that you can use as the basis of subsequent transformations.
- A hands-on introduction to Schematron (Uche Ogbuji, developerWorks, September 2004): Learn all about Schematron in this tutorial with detailed examples that illustrate Schematron's use, and offers recipes for common schema needs.
- Service Modeling Language (SML): Learn more about SML plus its constructs for creating models of complex IT services and systems. These models typically include information about configuration, deployment, monitoring, policy, health, capacity planning, target operating range, service level agreements, and so on.
- Systems Optimization using IBM ILOG: Explore ILOG and its advanced mathematical programming and constraint-based optimization tools and engines for efficient planning and scheduling (including ILOG CPLEX®, ILOG CP Optimizer and the ILOG OPL-CPLEX Development System.
- New to XML: If you are new to XML, start exploring XML and all you can do with it. Readers of this column might be too advanced for this page, but it's a great place to get your colleagues started. All XML developers can benefit from the XML zone's coverage of many XML standards.
- My developerWorks: Personalize your developerWorks experience.
- IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
- The developerWorksXML zone: Find more XML resources, including previous installments of the Thinking XML column. If you have comments on this article, or any others in this column please post them on the Thinking XML forum.
- XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks.
- The developerWorks Web development zone: Expand your site development skills with articles and tutorials that specialize in Web technologies.
- developerWorks technical events and webcasts: Stay current with technology in these sessions.
- developerWorks on Twitter: Join today to follow developerWorks tweets.
- developerWorks podcasts: Listen to interesting interviews and discussions for software developers.
Get products and technologies
- The Eclipse COSMOS project: Get SML support for the popular development and architecture workbench.
- Schematron: Add this ISO/IEC standard language for making assertions about patterns in XML documents to your toolbox. Schematron is a very effective means to express business and model rules in systems using XML documents.
- IBM product evaluation versions: Download or explore the online trials in the IBM SOA Sandbox and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
- Participate in the discussion forum.
- XML zone discussion forums: Participate in any of several XML-related discussions.
- developerWorks blogs: Check out these blogs and get involved.