Thinking XML: Use XML pattern tools for systems analysis

Combine the discipline of Operational Systems Analysis with a flexible, adaptive toolkit

Systems optimization is a growing field, especially in adaptive, autonomic systems, but also in traditional information workflows. Much of the material accumulated in the monitor phase is available in some form of XML. Rather than apply complicated, monolithic analysis tools, you can benefit when you apply the pattern dispatch mechanisms inherent in XML. This saves effort and increases flexibility as it supports a library of analysis primitives that you can redeploy for high-level reports as well as fine-tuning. Learn to apply the likes of XPath and XSLT patterns much more broadly in order to support analysis and drive systems optimizations.

Uche Ogbuji (uche@ogbuji.net), Partner, Zepheira, LLC

Photo of Uche OgbujiUche Ogbuji is a partner at Zepheira, LLC, a solutions firm specializing in the next generation of Web technologies. Mr. Ogbuji is lead developer of 4Suite, an open source platform for XML, RDF, and knowledge-management applications, and its successor Akara. He is also lead on the Linking Enterprise Data (LED) architecture, and the Versa RDF query language. He is a Computer Engineer and writer born in Nigeria, living and working in Boulder, Colorado, USA. You can find more about Mr. Ogbuji at his Weblog Copia.



02 June 2010

Also available in Japanese

Frequently used acronyms

  • API: Application Programming Interface
  • COSMOS: Community-driven Systems Management in Open Source
  • HTTP: Hypertext Transfer Protocol
  • REST: Representational State Transfer
  • W3C: World Wide Web Consortium
  • XML: Extensible Markup Language
  • XSD: XML Schema Definition
  • XSLT: Extensible Stylesheet Language Transformations

Every programmer, systems administrator, database analyst, and so on knows what to do when they face a problem with business process or some inefficiency utilizing systems resources. They reach into their own personal, idiosyncratic bag of tricks, put together through years of experience and tuned for very specialized for problems they have already solved. While this approach has served the industry for decades, increasingly enterprise architects want better. They want sound, reproducible, evidence-based approaches to solve such problems and, even better, to anticipate and avoid such problems in the first place.

The discipline of Operational Systems Analysis, and in particular Systems Optimization has emerged to help provide such discipline. Increasingly, responsible architects are paying attention to formalized means for managing the systems that have become the lifeblood of business. More and more traditional business rules and workflow tools are growing into the formal discipline of Systems Optimization, including IBM® ILOG® (see Resources), whose approach to modeling for optimization is illustrated in Figure 1. The optimization model lists various inputs (such as demands to satisfy; available resources; cost, yields, and recipes; operational constraints and customer preferences; and business goals), one or more mathematical models and optimization engines, and a schedule or plan with metrics (such as minimized costs, maximized yields, specific resource assignments, and best timing for activities).

Figure 1. IBM ILOG optimization model
Diagram of IBM ILOG optimization model with input, models and engines, and a schedule or plan

It is now very common for real-world systems to transmit messages in XML formats through various flavors of Service-Oriented Architecture (SOA). Such systems often express business rules, vocabulary, classification, and descriptive systems in XML forms. Thus frequently, an important input into optimization efforts comes from pattern analysis of XML data sets and message statistics. In support of such analysis, the XML-savvy enterprise architect can find inexpensive and flexible ways to gather basic information to support Systems Optimization. This article discusses conventions and techniques for applying the most effective XML processing practices in Operational Systems Analysis, organized around a selection of example scenarios.

Usage analysis

The most basic inputs to any systems optimization are patterns of actual usage. What operations are used the most? Are any patterns of parameters especially heavily used? These might be candidates for optimized resources or processing. In many modern, Internet-based systems, actual usage is represented by semi-structured messages over the wire, and these days it's common to have such messages represented in XML as SOAP Web services or XML messages HTTP POSTed to RESTful endpoints. If you use SOAP-based Web Services, chances are you already have access to specialized tools that operate at the binding level, so I'll focus examples in this article to XML over HTTP in RESTful fashion, a popular system for which off-the-shelf tools are less plentiful. The systems architect needs to understand how to perform analysis in such systems, and, as a bonus, these techniques are also available for SOAP-based systems at the protocol level.

Resource reservation service

For examples in this article, consider a fictional enterprise looking to deploy a resource reservation system, where employees can send requests to reserve resources such as conference rooms and multimedia equipment. Figure 2 is a very high-level diagram illustrating how the various systems (including brower, mobile, and custom applications) interact with the Resource Manager service.

Figure 2. Systems overview for resource manager
Overview diagram of system with applications, a REST wrapper, and resource manager service

You might be able to use network topology tools to analyze usage patterns of the various entry points (for example, to determine when the mobile interface is used more often than the regular Web interface). You can also work yourself into a more intricate analysis of usage patterns by instrumenting the REST layer. Listing 1 is an example of one of the reservation request messages issued as an HTTP POST.

Listing 1. example reservation request message
<?xml version="1.0" encoding="utf-8" ?>
<reservation xmlns="http://example.com/enterprise-reservations">
  <session id="115384" user="jdoe"/>
  <request timestamp="2010-03-02T15:39:50-07:00">
    <resource id="room123" type="conferenceroom"/>
    <purpose>Marketing meeting, and free doughnuts</purpose>
    <period start="2010-03-10T10:00:00-07:00" end="2010-03-10T11:00:00-07:00"/>
  </request>
</reservation>

Analysis using Schematron

Schematron is a good tool for such analysis. Listing 2 is Schematron to generate a report of the request timestamp and resource type.

Listing 2. Schematron to generate a report of request timestamp and resource type
<?xml version="1.0" encoding="UTF-8"?>
  <schema xmlns="http://www.ascc.net/xml/schematron">
    <title>Reservations manager usage analysis</title>
    <ns uri="http://example.com/enterprise-reservations" prefix="er"/>
    <pattern name="Basics requests report">
      <rule context="er:reservation">
        <report test="er:request/@timestamp">
          Request timestamp: <value-of select="er:request/@timestamp"/>
        </report>
        <report test="er:resource/@type">
          Requested resource type: <value-of select="er:resource/@type"/>
        </report>
      </rule>
    </pattern>
    <pattern name="Validation">
      <rule context="er:reservation">
        <assert test="er:session">
          Request must have a session element
        </assert>
      </rule>
    </pattern>
  </schema>

I recommend learning Schematron (see Resources), but I'll just explain the major constructs in the listing. The ns element is the namespace declaration mechanism for Schematron. A pattern is a grouping of related rules applied to XML documents. The rule elements specify business rules about specific constructs in the XML, given using the XSLT pattern form in the context attribute. Within rules, report elements are used to log constructs that meet criteria given in XPath form, and assert elements are used to specify criteria which the schema designer expects valid XML documents to meet. As a gross rule of thumb, Schematron assert elements express expected document patterns, while report elements express patterns of particular interest.

If you instrument the REST API of the reservation manager with the Schematron in Listing 2, you readily accumulate a log of when requests are made and for what resources. You can use the log to optimize peak times, determine whether resources are sufficient to accommodate requests, and more.


Analysis scenarios

In the previous section, you learned the basic idea of applying XML reporting to systems analysis, and focused on a core tool, Schematron, which is already the most popular way to express rules for XML document patterns. In this section, I show a variety of approaches to build on such a foundation to provide more comprehensive analysis in preparation for any aspect of systems optimization. By connecting individually simple but expressive mechanisms together, you can gather commonly-used operational statistics while retaining the power to derive your own, specialized tools which contribute to a competitive advantage.

Active systems modeling

In this scenario, the architect establishes the model of the system such that the systems team can actively monitor its actual behavior. Figure 3 illustrates the work and data flow for active systems.

Figure 3. Active systems modeling scenario
Diagram of active systems modeling based on SML and an analysis dashboard such as Eclipse COSMOS

One ready tool for this scenario is Service Modeling Language (SML), a proposed standard by IBM and other companies used to model complex services and systems including their structure, constraints, policies, and best practices. SML is based on an adaptation of W3C XML schema (XSD) with embedded Schematron rules as illustrated in Figure 3. Architects and developers can build on tools to incorporate SML into systems development and maintenance such as the Eclipse-based COSMOS project.

Utilization heat maps

An important input to optimization is some rigorous understanding of what patterns of services and resources see the most actual use, similar to the user experience concept of heat maps (modern statistical tools can produce the equivalent of heat maps). Figure 4 illustrates the work and data flow for heat maps.

Figure 4. Utilization heat maps scenario
Diagram of heat map setup with a reporting module, XSLT transform, and statistical package

In this example, each message goes through the reporting module which aggregates a report, building on Schematron Validation Report Language (SVRL), a set of conventions for generating structured output from Schematron reports. Such output is readily transformed into direct input format for statistical analysis tools, such as IBM SPSS®.

Protocol adaptation

One of the most common maintenance concerns is variations from expected protocol. Such variations might be caused by a gap in the systems model or by a unilateral change in the client layer. Figure 5 illustrates a monitoring process for such cases.

Figure 5. Protocol adaptation scenario with exception notification
Diagram of protocol adaptation: Reporting module sends exception notice to admin/developer review

If, for example, you construct the Schema using a closed grammar in XSD or RELAX NG (and possibly augmented with Schematron), you can get a closed model which should raise exceptions in case of protocol deviations. Such exceptions can be handled by notification for review by administrators or developers. A similar exception and notification approach works for managing operational compliance (such as to meet regulations).

Intervention outcomes

As an example of a fairly complex analysis approach, consider what happens when more general processing exceptions occur. The system might notify an agent for special treatment. Perhaps a conflict arises about the use of meeting rooms, and the agent calls the requesting parties to negotiate alternatives. The remedy might be equivalent to modified original requests, such as new meeting room requests for the negotiated outcome. In this case, it might be useful to analyze the original request versus outcomes through intervention in order to develop a means to automate intervention and increase productivity of agents. Figure 6 illustrates such a scenario.

Figure 6. Intervention outcomes scenario
Diagram: Reporting module recieves request, then sends exception notices to admin/developer review or intervention agent

You can see many of the same workflow tools from other scenarios, but in this case, a differential analysis tool also compares the patterns of related inputs, supporting sophisticated analysis.


Wrap up

As Web APIs and social media penetrate the firewall directly and indirectly influence enterprise systems design, more of the information fueling systems is in formats amenable to semi-structured reporting tools, an important insight to consider as you prepare for the Monitor phase of Systems Optimization. At the same time, the globally decentralized nature of such modern systems makes it impractical to take a monolithic approach to systems analysis and optimization. Operational Systems Analysis is a very important development in applying discipline for enterprise architects, as long as they realize that it's not possible to impose such discipline upon the real world. It is very important to have a flexible toolbox that you can reconfigure and redeploy to adapt to emerging scenarios. I hope you learned from this article some possibilities, some complexities, and several ways that reporting of XML messaging streams can support continuous improvement in modern systems architecture.

Resources

Learn

  • Schematron Validation Report Language (SVRL): Complement Schematron with SVRL, "a simple report language defined as part of ISO Schematron. SVRL provides a fairly full set of information from validating a document" that you can use as the basis of subsequent transformations.
  • A hands-on introduction to Schematron (Uche Ogbuji, developerWorks, September 2004): Learn all about Schematron in this tutorial with detailed examples that illustrate Schematron's use, and offers recipes for common schema needs.
  • Service Modeling Language (SML): Learn more about SML plus its constructs for creating models of complex IT services and systems. These models typically include information about configuration, deployment, monitoring, policy, health, capacity planning, target operating range, service level agreements, and so on.
  • Systems Optimization using IBM ILOG: Explore ILOG and its advanced mathematical programming and constraint-based optimization tools and engines for efficient planning and scheduling (including ILOG CPLEX®, ILOG CP Optimizer and the ILOG OPL-CPLEX Development System.
  • New to XML: If you are new to XML, start exploring XML and all you can do with it. Readers of this column might be too advanced for this page, but it's a great place to get your colleagues started. All XML developers can benefit from the XML zone's coverage of many XML standards.
  • My developerWorks: Personalize your developerWorks experience.
  • IBM XML certification: Find out how you can become an IBM-Certified Developer in XML and related technologies.
  • The developerWorksXML zone: Find more XML resources, including previous installments of the Thinking XML column. If you have comments on this article, or any others in this column please post them on the Thinking XML forum.
  • XML technical library: See the developerWorks XML Zone for a wide range of technical articles and tips, tutorials, standards, and IBM Redbooks.
  • The developerWorks Web development zone: Expand your site development skills with articles and tutorials that specialize in Web technologies.
  • developerWorks technical events and webcasts: Stay current with technology in these sessions.
  • developerWorks on Twitter: Join today to follow developerWorks tweets.
  • developerWorks podcasts: Listen to interesting interviews and discussions for software developers.

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into XML on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=XML
ArticleID=493204
ArticleTitle=Thinking XML: Use XML pattern tools for systems analysis
publish-date=06022010