Generating compliance documentation using IBM Rational Publishing Engine: Part 3. Reporting over multiple data sources

Part 3 this series of four articles shows how to create reports that take information from several data sources, including software from other companies. It explains how Rational Publishing Engine establishes traces from information in one tool to data managed by another and then combines them into the same document.

Share:

Dr. Einar W. Karlsen (einar.karlsen@de.ibm.com), Solution Architect, North-East Europe, IBM

Author photoDr. Einar Karlsen has more than 25 years of experience in the area of software development tools and methods. He has been with the IBM Rational division for 13 years, supporting customers in the use of Rational software, covering aspects related to demonstrating compliance by using report generation tools, among others. He works in Germany.



29 November 2011

Data source support

IBM® Rational® Publishing Engine supports a large set of Rational data sources so that organizations can generate the compliance-relevant documents with data maintained by them.

Figure 1. IBM Rational Publishing Engine data sources
Architecture with data sources and output formats

Larger view of Figure 1.

Most organizations use a mix of IBM Rational software and other companies' products, so they need to generate reports that contain information from several of those tools. Rational Publishing Engine works with Extended Markup Language (XML) data structures as input, so the schema of the data source must be defined by one or more XML schema definitions (XSDs). The schema defines the queries that serve as a basis for extracting information from a data source, and the XML file contains the actual data being extracted during report generation.

Therefore, you can use Rational Publishing Engine to generate information from data sources other than Rational software, provided that the information can be represented in XML format with an associated XSD schema that can serve as a basis for creating templates. For example, reporting from IBM® Rational® DOORS® data sources works by using the DOORS extensibility interface to export requirement information in XML format (but this happens in the background, so it is not visible to the user).

The XML and XSD definitions can also be retrieved from a Representational State Transfer (REST) service by submitting Uniform Resource Locators (URLs). This is the standard approach when reporting on data in multiple IBM Rational tools. Some of these tools provide a schema discovery wizard that can help you navigate the information source to find the appropriate schema.


Template creation

The end of the second article shows you how to create the main set of elements in a report intended to extract information about Rational DOORS subsystem requirements in table format, as shown in Figure 2. You can use snippets (predefined template fragments) to insert master pages, section headers and the required table in just a few steps. Beyond reducing the development time, snippets ensured that the look and feel of the template is in line with your company's standards and style guidelines.

Now you'll learn how to define this template and other templates, such as the template for rendering project risks maintained in Microsoft Excel (Figure 2). Both templates have the same look and feel, which is enforced by using standard snippets, and both use JavaScript to provide conditional colors for the information rendered.

Figure 2. Reports with conditional rendering
Use of colors to mark areas that need attentionUse of colors to mark areas that need attention

After you have defined the overall template structure with Rational Publishing elements and snippets, there are two steps to complete the template for rendering Rational DOORS subsystem requirements:

  1. First, it is necessary to import the schema for the specific Rational DOORS module.
  2. Second, queries of the schema must be inserted into the template to trigger extraction of information from the module when the report is generated.

You can import a schema by using the DOORS Schema Discovery wizard shown in Figure 3. A template author can use this wizard to select the relevant module, view, and attributes needed for a given template.

Figure 3. Rational DOORS Schema Discovery Wizard view
Wizard to select the DOORS views and attributesWizard to select the DOORS views and attributes

After importing the schema, the template author can start inserting schema queries in the template. For example, the template in Figure 4 starts with a paragraph that renders the section title. A query ($1) of the schema named DOORS_SSR has been inserted to retrieve the module name for use in the title. Other queries have been inserted into the second row of the table to iterate over all requirement objects in the module ($8) and to render the Object Identifier and Object Text for each requirement object in table cells.

Figure 4. Use of queries in the template
Example of queries to render ID and text

Larger view of Figure 4.

Tip:
Rational Publishing Engine provides powerful features for conditional rendering of information. In the template used to generate subsystem requirements, JavaScript has been used to define conditional rendering of the background color of the attribute:

if (Supp_Fullfillment.equals("Partial")) { 
    "FF8000"
} else {
    "FFFFFF"
}

The code fragment specifies that if the requirement is only partially fulfilled, the cell will render with an orange background (color code FF800); otherwise, the background will be black. Figure 5 shows the dialog window for setting the background color using JavaScript expressions, including the Test button, which can be used to test whether the script works as desired.

Figure 5. Use of JavaScript for conditional rendering
Shows the JavaScript code for background color

Larger view of Figure 5.


Reporting data from other companies' software

It is possible to use the Rational Publishing Engine to generate information from data sources in other companies' software, provided that the information can be represented in XML format with an associated XSD schema that can serve as a basis for the schema needed to define queries in a template. When it is available, the XSD schema can be imported into Document Studio and used as a basis for defining queries on the underlying XML data structure.

For example, the template in Figure 6 uses an XSD schema for a Microsoft Excel spreadsheet to establish the data source schema for project risks. The queries in that schema (Row, Risk, Probability) can be used to define the retrieval of information from the XML file. In this example, the risks are rendered in a table. The second row in the table has been defined to iterate over all risks in the XML file (query $1,Root/Row); whereas, the individual properties of a risk are rendered in the table cells (Risk, Probability and so forth).

Figure 6. Template for reporting Microsoft Excel data
Queries in the template to render risk attributes

Larger view of Figure 6.

The actual XML file to be used for extracting the data has been specified in the corresponding document specification in the Rational Publishing Engine launcher, as shown in Figure 7, by assigning the path of the XML file to the URI property.

Figure 7. Document specification for XML data
Setting the URI of an XML data source

There are different ways to produce the XML and XSD files required for reporting Microsoft Excel data.

  • One way is to export the XML file directly from Excel and use the standard XSD schema file from Microsoft (see the Resources section for links). The disadvantage of this approach is that the columns are identified by column numbers rather than by name, thus requiring iteration and filtering to select a specific risk attribute.
  • In the current example, we used a custom Excel add-in to export the XSD schema as well as the XML file. The advantage of this approach is that it generates an XSD and XML file where the fields can be referenced by the Excel column names (see Figure 6).
  • As an alternative to those two solutions, you can use an XML to XSD converter to generate the XSD schema from the XML file.

Whichever way you use, it is important to notice that the command parameter (see Figure 7) of a data source enables the user to specify the name of a program that will be invoked to export the XML file before generating the report.


Cross-tool reporting

One advantage of using integrated tool environments is that it becomes possible to define traces between elements kept in different tools, such as traces from change requests to requirements, from requirements to model elements, from requirements to test cases and from (failing) test results to defects. Traces are key pieces of information that can help you determine which requirements are impacted by a change request, which model elements define the realization of a requirement, which test cases are associated with a requirement, and whether the tests associated with a requirement succeeded -- and if they didn't, what the resulting defects are. The answers usually need to be documented in corresponding reports.

Figure 8 shows two examples of cross-tool reports with traces.

  • The report on the left shows a requirement model in IBM® Rational® Rhapsody® with detailed requirement information from Rational DOORS.
  • The report on the right shows detailed requirement information from Rational DOORS with the related test cases from Rational Quality Manager.

The subsections that follow demonstrate how the key parts of these reports have been defined.

Figure 8. Cross-tool reports
Section with the model element and requirementsSection with the model element and requirements

Cross-tool reporting using identifiers

One key technique for providing traces across tool boundaries is using export and import of information from one tool to the other. For example, Rational Rhapsody can import requirements from Rational DOORS and make them available in a requirement model for further traceability to model elements. Suppose that you would like to develop a template that lists all requirement elements in a Rhapsody model with some of the attributes of those requirements, such as text and priority, managed in DOORS, as shown previously in Figure 8. You would need a template that extracts information from your Rational Rhapsody model (requirement identifiers) and from Rational DOORS (requirement properties), as well as the ability to establish traces from Rhapsody to DOORS.

Figure 9 shows the key elements and queries for achieving this.

Figure 9. Cross-tool reporting template using identifier matching
Condition matching requirement and model element

Larger view of Figure 9.

Basically, the template iterates over all model elements in a requirement diagram (see top container, query $15). For each of these model elements, it iterates over all requirements in a specific Rational DOORS module or view (source DOORSSysReq, query $18) and identifies the requirement where the name of the model element matches the requirement object identifier, using a condition as a filter. When the match is found, the template continues adding the attributes of the requirement object to the table.

Such identifier matching is a basic mechanism for providing cross-tool reporting that can also be used in the context of integration with other data sources. However, in the template described, it will work only if you know the module where the Rational DOORS requirements can be found. Therefore, the associated document specification must define the DOORSSysReq data source to use this module or view. Moreover, report generation creates overhead costs caused by the need to iterate over potentially a large amount of requirements data to find matching element. A filtered view, showing only the requirements that have been imported into Rational Rhapsody, could reduce these runtime costs.

Another technique that could speed up generation is to use native filters rather than conditions: a condition filters all of the data delivered to Rational Publishing Engine by a given data source; whereas, a native filter works directly on the data source, thus ensuring that only the matching requirements are forwarded.

Cross-tool reporting using URLs

A better solution for cross-tool reporting occurs in the context of Rational Jazz™ technology-based products, where the philosophy is to designate links (or traces) between elements in two distinct tools (or even the same tool) by using URLs. One example where this technique is used is the Rational DOORS and Rational Quality Manager report in Figure 10 that outputs information about requirements from DOORS, including information about the Rational Quality Manager test cases associated with the requirements.

Figure 10. DOORS and Rational Quality Manager cross-tool report
Subsection: requirement and test case information

The corresponding template is shown in Figure 11. The template is a refinement of the DOORS.dta template previously used for output of information regarding requirements and their attributes, with an additional container for rendering the external links. The template adds a table for populating information regarding test cases associated with the current requirement. It achieves this by iterating over all external object links for a requirement object (query $9). If the associated URL property of the external link is not empty, it will dynamically connect to the data source named RQM_TC (the schema for test cases). For each test case, it will then generate a table row where the first column contains a hyperlink, the second column the state of the test case (computed using a JavaScript), the third column the weight of the test case and the fourth column the description of the test case. The hyperlink uses the URL as the target of the hyperlink and the test case name as the text to display for the hyperlink. The state is computed using JavaScript, because Rational Quality Manager returns state values as fully qualified names. The JavaScript simply strips off the prefix of the name and returns the simple name ("new,” for example).

Figure 11. Cross-tool reporting template using URLs
Dynamic data source connecting to test case

Larger view of Figure 11.

Dynamic data source configurations are key concepts to facilitate reporting from one data source to another. In essence, they are a subset of a document specification made available inside a template. They allow the template author to define the key properties: which schema to connect to and the actual URL to use.

Figure 12. Configuration of a dynamic data source
Properties tab shows Property and Value columnsProperties tab shows Property and Value columns

The configuration information is not known ahead of time, so the dynamic data source configuration allows data to be configured dynamically at runtime. In this example, the URLs are read from DOORS data, and their numbers and concrete values are not known when the template is created. The properties for the data source configuration element in Figure 11 are shown in Figure 12. The target data source properties have been defined by the template author in the template (see the left part of Figure 12). The Uniform Resource Identifier (URI) property is determined by the value of the module/object/externalLink/URL query.

Notice that, in the template, the username and password properties are empty to avoid hardwiring these properties in the template, because they might vary from user to user. In fact, these properties are set when the RQM_TC data source is configured in the document specification. The properties for the data source in the document specification are given in the right half of Figure 12.

It is important to note that the IBM Rational Publishing Engine has no data source information about any tool. The logic to create templates, in particular ones for cross-tool reporting, relies on the template designer's knowledge of how the point product data is correlated or connected. However, Rational Publishing Engine provides the basic mechanism to support such cross-tool reports in form of dynamic data connections, filters and JavaScript.


Summary

This article has shown how you can use Rational Publishing Engine to generate reports by taking input from a variety of data sources, including Rational tools and commonly used software from other companies. It also introduced several Rational Publishing Engine advanced concepts:

  • Use of JavaScript for conditional rendering of colors that depends on the value of information extracted from a tool
  • XML reporting capabilities to develop templates for reporting on data that comes from other companies' software, such as Microsoft Excel
  • Dynamic data source configurations to create cross-tool reports that use URLs to trace, for example, from a requirement to a test case
  • Identifier matching to create cross-tool reports from a design model element to a requirement

What's next

Part 4 of this series shows you how to test, check, and deploy the documents and templates that you have developed. It also covers a number of more advanced Rational Publishing Engine template authoring features such as regions and hyperlinks, including hyperlinks that will allow you to navigate from a report directly into the data source in case you need to change the information there. Lastly, you will learn how to generate reports interactively, as well as in batch mode.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=776432
ArticleTitle=Generating compliance documentation using IBM Rational Publishing Engine: Part 3. Reporting over multiple data sources
publish-date=11292011