Generating compliance documentation using IBM Rational Publishing Engine: Part 1. Overview and workflow

This is the first in a series of four articles about using the IBM® Rational® Publishing Engine to generate compliance-relevant reports. Part 1 gives a short overview of the software, describes the report development workflow, and provides a sample report as a basis for showing the capabilities of the tool in the subsequent articles.

Share:

Dr. Einar W. Karlsen (einar.karlsen@de.ibm.com), Solution Architect, North-East Europe, IBM

Author photoDr. Einar Karlsen has more than 25 years of experience in the area of software development tools and methods. He has been with the IBM Rational division for 13 years, supporting customers in the use of Rational software, covering aspects related to demonstrating compliance by using report generation tools, among others. He works in Germany.



29 November 2011

Introduction to this series

Compliance is a topic of increasing relevance in the industry, whether it's compliance with International Organization for Standardization (ISO) standards, Software Engineering Institute Capability (SEEI) Maturity Model Integration (CMMI), Mil-Std-498, Institute of Electrical and Electronics Engineers (IEEE), or the V-Model process. IT development teams frequently demonstrate compliance by following specific processes (that might be enforced by tools) and by delivering documents that show that the development has been conducted in accordance with the relevant development standards. Organizations working in an environment with compliance requirements must be able to document that they have designed, implemented and tested the application in accordance with the standards. Usually, this imposes a need to generate documentation across workflows, phases, iterations and tools in the adopted development process.

Most organizations that need to demonstrate compliance are facing the same set of challenges, which pose similar requirements to document generation software. Basically, the organizations need to reduce costs for producing the required documents for showing compliance, and this often involves a significant amount of manual work. Moreover, the reports produced must have a specific layout and include information coming from a multitude of sources (tools and humans). Last but not least, the organizations must be able to prove that the information presented in these documents is reliable, meaning that the information is complete and consistent. One challenge in this context is that a document is typically generated, delivered, read, reviewed, revised and eventually stored in a document management system many times due to changes. This is a very time-consuming, human-intensive and error-prone process. All of this calls for a solution that can generate the documents efficiently and automatically, again and again.

This article series shows how to use the IBM® Rational® Publishing Engine to generate large-scale reports that fulfill these requirements. The articles cover a variety of aspects relevant in this context, ranging from layout, semantic completeness, cross-tool reporting and efficient deployment. You will also learn how you can use Rational Publishing Engine to produce high-quality reports that consist of a combination of handwritten information and information extracted from Rational tools (see the list that follows), as well as tools from other companies, such as Microsoft Excel. The series also covers various techniques for extracting and combining information from several tools into a consolidated cross-tool report, as well as techniques for checking that the information is complete.


Rational Publishing Engine overview

Rational Publishing Engine can be used to create documents from data across a wide range of distributed and diverse data sources. It can access data from all of these Rational applications:

  • IBM® Rational® Asset Manager
  • IBM® Rational® ClearCase®
  • IBM® Rational® ClearQuest®
  • IBM® Rational® DOORS®
  • IBM® Rational® Focal Point
  • IBM® Rational® Quality Manager
  • IBM® Rational® Requirements Composer
  • IBM® Rational® RequisitePro®
  • IBM® Rational® Rhapsody (through Rhapsody REST service or through the Design Manager)
  • IBM® Rational® Team Concert™
  • IBM® Rational® Software Architect (through the Design Manager)
  • IBM® Rational® System Architect

It also provides access to data from other companies' software through Extensible Markup Language (XML) and Representational State Transfer (REST) interfaces.

Figure 1 shows how the Rational Publishing Engine can extract information from numerous sources, such as requirement management, design, test, configuration management and defect tracking tools, to generate a variety of reports, such as analysis, design, test and status documents. The information extracted is defined in terms of templates, and templates can be composed into documents that contain information from a many information sources. The content and the layout of the generated report can be defined according to your organization's needs.

Figure 1. Generation of documents from many sources
Diagram of sources of the information for automated reports

IBM Rational Publishing Engine provides two tools for the two major user roles:

  • The Document Studio application is used by the template author to define new report templates.
  • The Launcher is used by the end user to compose and configure document specifications, using existing templates, and to generate reports from these document specifications.

The Document Studio tool

A template (.dta file) defines the structure of the information to be rendered using elements such as paragraphs, containers, tables and figures. These elements are available in the Palette (top-left view in Figure 2) and can be used to build the template in the template editor in the center of the Document Studio. The template in Figure 2 basically defines a section title, followed by elements to render the Rational DOORS headings, requirement identifier and attribute.

A template also includes information regarding styles, variables and images (PNG, GIF, JPEG, and BMP format) that are used to define the template, as shown in the Outline view (top right). A key component of the template is one or more schemas that define the queries for extracting information from the external data sources. The schemas are available in the Data Source Schema view (bottom left). Last, but not least, the Properties view (bottom right) is available for setting properties of the elements in the template. These properties can control the semantic aspects of the queries (recursion) or they can influence pure formatting aspects (fonts, font size, colors).

Figure 2. Document Studio layout
Screen capture shows views and template editor described

Larger view of Figure 2.

The Launcher tool

The Launcher is used to define a document specification (.dsx file) in terms of references to existing templates, as shown in Figure 3. Templates used in a document specification are shown in the Document Specification view (top left). The templates need to be configured in the Launcher in the Properties view (bottom left) with the information required to connect to an actual data source. For the Rational DOORS data source in Figure 3, this means defining the DOORS module and view, as well as the required user name and password (among other entries). The configuration of a template might also involve definition of actual values for variables used in the template, as well as configuration of the output format, such as Microsoft Word output, including the path name of the Word outline (.dot file) and the path name for the generated Word file.

Figure 3. Rational Publishing Engine Launcher screen
Launcher and document generation notice (pop-up window)

Larger view of Figure 3.

During report generation time, Rational Publishing Engine uses this information along with the queries in the template to extract the required information from the information sources and put it into the resulting report. The report can be generated in Portable Document Format (PDF), Microsoft Word, Hypertext Markup Language (HTML), or XSL Formatting Objects (XSL-FO) format. The choice is configured in the document specification. Word output can be generated in 2003 binary format or the new Microsoft Office OpenXML format used in Office 2007 and later.

The diagram in Figure 4 gives an overview of Rational Publishing Engine architecture and its various components, operations and artifacts. The Document Studio and the Launcher are the central components in defining templates and document specifications and in generating reports. However, for some of the supported tools however (Rational ClearQuest, ClearCase, RequisitePro, DOORS and Rational Design Manager), schema discovery wizards are provided to help the template author connect to the data sources and discover and import schemas. Likewise, wizards are also provided to help the end user connect to actual data sources to configure those sources with the correct parameters.

Other Rational software applications, including Rational DOORS and Rational Requirements Composer, integrate with Rational Publishing Engine by providing wizards that end users can use to configure and generate predefined or user-defined reports from within those tools, without having to start the Launcher, explicitly. These applications also bundle a subset of Rational Publishing Engine called IBM Rational Reporting for Document Generation, which means you can generate documents without a separate installation of Rational Publishing Engine on the machine.

Figure 4. Rational Publishing Engine components
Flow diagram of main components, operations and artifacts

Report development workflow

Development of reports usually follows a specific workflow that involves several standard activities (see Figure 5).

Figure 5. Template creation workflow
Workflow diagram shows activities and their dependencies

These are the main activities, along with pragmatic hints for carrying out these activities:

  1. Gather requirements.
    1. Capture the requirements to the generated report, and then sketch the report to be generated, perhaps in a plain Word document.
    2. Prioritize the requirements (roughly), and determine the amount of information to be extracted.
  2. Design the document.
    1. Determine the overall document structure and the templates needed, given the requirements.
    2. Divide a large document into several templates that can be developed, tested, and possibly reused independently.
    3. Reuse existing templates whenever possible to reduce development and maintenance costs (see Step 8).
  3. Specify the layout.
    1. Define the overall layout to use for the report.
    2. Set up a base template with standard headers, footers and paragraph formats, or reuse existing base templates. Tip: By using external stylesheets, it becomes possible to decouple the logic of the template from the formatting of the document and to define parts of the formatting in Microsoft Word.
  4. Implement the template.
    Implement the template with Document Studio, using a top-down approach.
    1. Document the template by using meaningful names for template elements and documentation fields. Tip: Do not get caught up in details regarding formatting at the very beginning; leave this to later.
    2. Cover 80% or more of the requirements in the first iterations. (Reuse an existing template or template fragments whenever possible.)
    3. Polish the details and incorporate more detailed requirements, one by one.
    4. Consider the relevance of requirements that are difficult to realize (cost-benefit analysis). As an alternative, use scripting to export the information in a form that Rational Publishing Engine can better process (this has a certain cost) in case that the information data source does not expose the data in an appropriate form (which should rarely happen).
    5. Define the document specification and generate the report.
  5. Test the template.
    Testing should be done incrementally and interleaved with template development. After the significant part of the template has been developed, testing with a large amount of data makes sense so you can determine whether there are any performance issues that need to be addressed.
  6. Review the report.
    Review the report and revise the template or change the data sources as necessary.
  7. Deploy the template and document specification.
    1. Install the developed and tested templates and document specifications (on a central file server, for example).
    2. Register the location of the templates if it is relevant for a tight integration with the development tools.
    3. Schedule document generation using a scheduler and the Rational Publishing Engines ability to trigger report generation through command shell scripts if the report needs to be generated automatically at regular intervals.
  8. Register reusable templates.
    1. Identify entire templates or fragments of templates that could be reused in future templates.
    2. Prepare these templates for reuse and put them on a central file server.
    3. Register the template libraries with Rational Publishing Engine, and publish the snippets.

When working with Rational Publishing Engine, it is important to maintain a collection of guidelines for developing templates. The remainder of this article series provides tips that will be valuable for starting to use Rational Publishing Engine to create compliance-relevant reports. You can find other ideas on developerWorks, in articles and in both the Rational Publishing Engine forum and the Rational Publishing Engine Community wiki, as well as on non-IBM sites, such as the Rational Publishing Engine Actual blog by Dragos Cojocari, who is the architect this software (see the Resources section for links).

There is one "master rule" that is important to have in mind before even starting: Automated generation of reports works well only if the data in the tools is organized and stored according to specific guidelines.

For example, if the task is to create a design document, then it is necessary to make assumptions about where in the model the information needed for a report could be found. This, in turn, requires that the model is well-structured, according to model guidelines, and that every team member has obeyed those guidelines. Simple rule: mess in, mess out. Therefore, an organization should have "the end in mind." Start defining guidelines for using the tools to support future reports to be generated in a systematic way. Ideally, the most critical reports and the model guidelines and working model for a tool are defined together in the initial phases of a project.

Simple rule: mess in, mess out.

The sample report in this series

Many compliance-relevant reports have -- from an abstract point of view -- a similar structure. They start with a title page that is followed by approvals, change history, table of contents, list of figures (all standard information or generated automatically) and then an introduction. This is followed by a mix of written and automatically generated information. Figure 6 shows an excerpt of a sample software development lifecycle report, which will be a running example in the rest of this article series.

Figure 6. Excerpts from the sample report
Main sections of the sample reportMain sections of the sample report

Larger view of Figure 6.

In essence, the document contains information coming from humans, third-party products (Microsoft Excel), and Rational software (Rational DOORS and Rational Rhapsody), including Rational Jazz technology-based software (Rational Quality Manager). The subsequent articles in this series demonstrate how such a complex report can be decomposed into smaller, reusable templates and how templates can be developed and published for reuse to reduce the time and cost of developing the reports. This series also demonstrates how you can use Rational Publishing Engine to define a standard layout for all reports and how information can be extracted from several information data sources.

The sample report starts with a title page, which is followed by a written introduction (imported from a Word document), and then has a section with information extracted from Microsoft Excel (the risk list), as shown in Figure 6.

These sections are followed by several sections that containing stakeholder, system and subsystem requirements from Rational DOORS, the use case model extracted from Rational Rhapsody, and sections with information related to test plans, test scripts, test results and defects, all from the Rational Quality Manager.


What's next

The next article in this series shows how you can split a complex document like the sample report into a number of smaller manageable templates, plus how you can design such templates for reuse in various documents to reduce development time and cost. Part 2 also demonstrates how you can use Rational Publishing Engine features to enforce a common layout and style (paragraph styles, headers, footers, and so forth) for all templates that you develop.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=776360
ArticleTitle=Generating compliance documentation using IBM Rational Publishing Engine: Part 1. Overview and workflow
publish-date=11292011