Improve software development project success with better information

Advantages of automated measurement, analytics, and reports

Automated reporting can help you document compliance and eliminate the errors, inconsistency, and wasted time and effort inherent in manual reporting. Automated measurement can help improve processes and streamline project delivery. This article describes how automated reporting and measurement tools, such as IBM Rational Publishing Engine and Rational Insight, help software and systems development teams provide accurate, timely, and appropriate information to decision makers.


Eric Larsen (, Senior Product Manager, Rational Insight software, IBM

Author1 photoEric Larsen is a senior product manager for the IBM Rational Insight, IBM Rational SoDA, and IBM Rational ProjectConsole products. He has been working in the computer engineering field since 1979. When he joined Rational software 10 years ago, he began developing a measure management dashboard application that eventually became the Rational ProjectConsole product. After that, he became a senior software engineer for Rational ProjectConsole, a software development manager for Rational ProjectConsole, and a software development manager for the Rational Portfolio Manager Web UI initiative. More recently, Eric brought his experience with reporting, metrics, and project management tools into the product management position for the Rational Enterprise Reporting product line, which includes the legacy ProjectConsole and SoDA applications, as well as IBM Rational Insight software.

Jonathon Chard (, Systems Marketing Manager, IBM

Author1 photoJonathon has wide experience in standards, technologies, and tools across the entire systems and software development lifecycle, gained from eight years as a consultant within IBM and, formerly, Telelogic. Before this, he spent 12 years as a systems engineering practitioner within the automotive industry. Jonathon studied electronic and electrical engineering at the University of Manchester, in the UK. He received a B.Sc., with honors, in 1984, an MSc in 1985, and a PhD in 1988.

Ben Williams (, Senior Product Manager and Reporting Strategy Lead , IBM

author photoBen Williams is a senior product manager and reporting strategy lead for the IBM Rational brand. Ben joined the IBM Rational organization in 2008 through the Telelogic acquisition. While at Telelogic, he was the sole product manager in the worldwide central engineering business unit, one of the four software business units, which provided l common technologies and capabilities to the three core product divisions, covering requirements management, change and configuration management, and modeling. During this time, he led a team that created what is now known as Rational Publishing Engine, which has become the Rational technology for document generation. Before his product management roles, he was a technical account manager and requirements management specialist for more than seven years. Ben is a keen supporter of the open source movement, having previously been an active member (with Committer status) on the Eclipse Process Framework (EPF) Project and now as a member of IBM's Linux Champion program. He continues to play an active role in the Open Services for Lifecycle Collaboration (OSLC) community, contributing to both the Core and Reporting workgroups.

17 May 2011

Also available in Chinese Spanish

Reporting across organizations presents challenges

The ability of organizations to accurately answer these questions drives success in systems and software development:

  • Are we on target?
  • Are we within budget?
  • Are my projects contributing effectively to my operational objectives?
  • How well are these operational objectives helping to satisfy our business goals?
  • Can we easily demonstrate compliance with customer requirements or industry standards?
  • Are we ready to deliver yet?

However, to answer such questions, teams must have access to accurate, timely, and appropriate project information and the ability to put it in the hands of decision makers. Because data is typically spread across different disciplines, geographically dispersed teams, and tools, team members struggle to quickly get the right information in the right format at the right time.

Adding to the challenge is the push toward making products more instrumented, interconnected, and intelligent, or what IBM calls "smarter." Smarter products rely upon the tight integration of sophisticated software with hardware and electronics. This leads to increased systems and software development complexity and to demanding larger, more diverse teams than ever before. As a result, information has become even more dispersed across physical, organizational, and technological boundaries.

This article discusses the difficulties that organizations face as they gather critical information and explores the use of automation to increase the efficiency and effectiveness of reporting and measurement. It also describes IBM® Rational® software that can help automate reporting and measurement to improve project, program, and organizational management and decision makers' abilities to influence positive outcomes.

Manual reporting can cause problems

Software development organizations typically use existing and familiar tools, such as word processing or spreadsheet programs, to report and measure information. Although these might be adequate methods for local or one-time documentation, they require time-consuming manual effort from many individuals across the software development cycle to assemble and analyze the information for project reporting and measurement.

Manual reporting

Manual reporting is inherently:

  • Expensive
  • Error prone
  • Inflexible
  • Opaque
  • Slow

Having individuals extract and analyze information takes valuable time away from employees' core responsibilities. Moreover, manual reporting and measurement processes are dependent on the way an individual chooses to gather and analyze information. Consistency is essential when organizations update documents throughout a development lifecycle or repeat measurements over time to identify trends. Even with guidelines in place, manual processes leave opportunities for individuals to interpret information or perform tasks differently. As a result, it is difficult to generate consistent reporting and measurement data, which makes it hard to spot the real trends in the information.

Finally, because organizations must frequently generate reports for traceability, compliance, and decision making, sometimes business analysts or developers force reporting mechanisms into processes where they do not belong. For example, a developer who is modeling in Unified Modeling Language (UML) knows that he must create a particular diagram within the model for report documentation. To streamline the report generation process, he adds text boxes to the model to identify which model elements meet which requirement. Although this practice simplifies reporting, the developer now has report-specific data and formatting information embedded in the actual design model, which represents inherent risk to design information integrity.

Organizations need a more efficient, accurate and repeatable way of generating reports and measurements. To address compliance requirements and gain key business insights, they must provide stakeholders and decision makers with the right information at the right time. To do so, they need the ability to quickly publish accurate, timely documentation and real-time dashboards that allow users to recursively drill down through levels of abstraction to get to information relevant to the task at hand.

Automation results in more accurate, reliable data

Rather than manually measuring and documenting business-critical information, developers can use tools specifically designed to automate such tasks. As Figure 1 shows, by using the right software, you can extract information from repositories across the systems or software development lifecycle (spanning requirements management, modeling, testing, and change and configuration management), as well as beyond the traditional development boundaries into other business systems. Automated tools can collect information quickly, easily, and unobtrusively from across the enterprise, delivering accurate, timely reports and enabling better decisions about the right course of action for projects.

Figure 1. Automated reporting can combine information from multiple sources
Diagram of consolidating information

As previously discussed, good tools used for the wrong purpose provide, at best, limited value. However, tools specifically designed for automated reporting and measurement deliver fast, accurate results that cater to specific needs. If you choose tools that offer automated document generation, as well as measurement and development analytics, you also streamline your compliance and decision-making processes.

There are two broad categories of information presentation that are important for software and systems development:

  • Software and systems documentation, such as requirements and design documents, as well as compliance documents.
  • Development analytics, such as requirements coverage, test coverage, defect trending reports, and other such measures that can be aggregated in executive development status dashboards.

For simplicity, IBM refers to both categories as "reporting."

Document generation

Advantages of automated reporting and measurement

Faster and more transparent
Provides access to up-to-the-minute information, with quick, on-demand generation of documents
More accurate
Draws data directly from the source
Less error-prone
Makes manual intervention unnecessary
Protects development data from unnecessary risk
Avoids the need for manual data modification for reporting purposes
Allows a conceptually clearer approach as documents become an artifact of the process rather than driving the process
Enables stakeholders to access information on demand without requiring a subject matter expert's time to assemble information
Transforms information to suit the content and format needs of different consumers

Businesses continue to rely on documentation as a critical communication mechanism, both internally and across contractual boundaries. Development teams are often required to deliver documentation according to specific standards that are imposed either by customers or by industry regulators.

A key benefit of automated document generation is the separation of formatting and content. By creating report templates, you need to format the layout for the information that you need only once. When it's time to update it, you can populate the report template with the latest information. In an automated report, information is also always consistent between versions. IBM® Rational® Publishing Engine includes templates that you can customize (see the last section for more about this tool).

Automation inherently makes a process more efficient. In the case of reporting, it also enables organizations to control documentation production, rather than being controlled by the need to produce documentation. In other words, as developers work on projects, they don't need to worry about how they will document their output. This gives them the freedom to focus on their core responsibilities.

Measurement and development analytics

Measurements are essential to improving processes. As William Thomson (Lord Kelvin) said nearly a century ago: "If you cannot measure it, you cannot improve it." Measurements enable a project team to gather feedback on an aspect of project performance so the team can modify the process to more effectively control that aspect. Measurement is a fundamental aspect of the IBM approach to incremental, continuous improvement. Other commonly used approaches, such as Capability Maturity Model Integration (CMMI), require quantitative management of processes through measurement to drive ongoing process optimization. However, measurement is only part of the story. It is equally important that organizations understand what the data means to various stakeholders.

Automated measurement and analysis tools help organizations turn insight into action, and they bring efficiency and repeatability to the measurement process. The value of repeatable measurements is even greater over time, because analyzing trends further improves the predictability of project outcomes.

Much like automated document generation solutions, automated measurement and analytics collect data from disparate repositories across the development and delivery lifecycle (and in many cases, beyond). After the data is collected, these tools can show trends of that data over time and display the trend in appropriate formats that stakeholders can easily see and understand. By using software for automated measurement and analytics, you can generate different, yet consistent, metrics reports and views for stakeholders at all levels of the business, from developers to the CTO. This helps the development organization contribute to the operational objectives and overall business goals more effectively.

Rational reporting and measurement tools work together

Rational reporting and measurement tools can help you to extract and combine information from a wide variety of IBM Rational development tools, including these products:

As well as extracting information from other Rational tools, Rational reporting and measurement solutions can extract information from nearly any source that either generates XML data or exposes a Representational State Transfer (REST) interface. In this way information from multiple sources can be combined to produce more complete reporting.

"Good tools used for the wrong purpose provide, at best, limited value. But tools specifically designed for automated reporting and measurement deliver fast, accurate results that cater to specific needs."

IBM Rational Publishing Engine

Developers can quickly produce accurate, up-to-date documentation by using Rational Publishing Engine software (also see Resources). It includes prebuilt templates that you can use or easily customize by using drag-and-drop technology.

The Document Studio interface (Figure 2) provides an intuitive workplace for document template design and customization, with easy access to a palette of document elements, data source schema, document element properties, and the template outline. You can build document templates visually by dragging the required document element from the palette to the appropriate point in the template. In this way, you can easily add document formatting items, such as headers, footers, tables (including tables of contents) and boilerplate text

Data source schemas give access to information in the data source tools. You can insert this information into the document by dragging the appropriate schema element into a container in the template. You can apply conditional formatting to extracted data to further enhance the document. For example, a table of requirements could highlight unsatisfied requirements in red. You can use the template outline to navigate around the template and to set the required styles for elements. And, with the Properties window, you can view and modify the properties of selected template items.

Figure 2. The Rational Publishing Engine Document Studio interface
Document Studio screen capture

Larger view of Figure 2.

When the template is complete and the data sources are specified, you can click a Generate Report button, and the report template will gather current information from the sources and transform and populate that data in the appropriate places in the template. It can produce documents in these widely used data formats so that all stakeholders can easily access the reports:

  • Microsoft Word
  • XSL Formatting Objects (XSL-FO).
  • Adobe Portable Document Format (PDF)
  • HTML.

IBM Rational Insight

The IBM Rational Insight automated measurement and development analytics software uses the same flexible REST interfaces as the Rational Publishing Engine tool to extract data from systems and software development data sources. However, rather than publishing a document, the Rational Insight application collects and organizes the data in a central repository. Then it outputs consolidated data, measurements, and trends to customizable dashboards, as shown in Figure 3. This flexibility enables stakeholders and decision makers to obtain the up-to-date and accurate information they need, when and where they want it.

Figure 3. Customizable dashboards to display the right information
Screen capture of a typical dashboard

Larger view of Figure 3.

Rational Insight is a key technology in the implementation of a measured, incremental improvement approach, which provides organizations with a structured way to align their systems and software development capabilities to business objectives. With this technique, organizations define operational approaches to reach business objectives and then apply practices to implement those approaches. Metrics measure and control the implementation and effectiveness of such practices, and the Rational Insight application provides the automation mechanism for the measurement of those metrics. (Also see Resources.)

Automation is about more than just productivity

Use of automated reporting and measurement tools, such as Rational Publishing Engine and Rational Insight, is a key to providing accurate, timely and appropriate information to decision makers. Automated reporting can help you document compliance and eliminate the errors, inconsistency, and wasted time and effort inherent in manual reporting. Automated measurement can help improve processes and streamline project delivery. Ultimately, organizations that automate reporting and measurement help improve the success rates for their projects, gain productivity across their development teams, and improve their ability to meet high-level business objectives.



Get products and technologies

  • Get the free trial download for Rational Insight.
  • Check the Trials and Demos page for Rational software.
  • Evaluate IBM software in the way that suits you best: Download it for a trial, try it online, use it in a cloud environment, or spend a few hours in the SOA Sandbox learning how to implement service-oriented architecture efficiently.



developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Rational software on developerWorks

ArticleTitle=Improve software development project success with better information