Level: Introductory Shawn Hinrichs (Shinrichs@opennetwork.com), Consultant, OpenNetwork Technologies
20 Feb 2004 Derived from a RUC 2003 presentation, this introduction to the real-world use of ProjectConsole utilizes data collected from ClearQuest and TestManager. It discusses determining useful data, the audience for the data, and the purposes of displaying data in a dashboard or report format.
See the
related presentation files
.
Introduction
Open Network Technologies develops software that provides an end-to-end
identity management solution for Web and enterprise identities. This article
covers the following information:
- Identifying useful data
- How to display the data
- Audience
- Lessons learned
- Capabilities
- Benefits
Identifying Useful Data
First, in the discussion of identifying what useful data is, we
focus on Rational ClearQuest defect information, ClearCase activity change sets,
and test script completion pass or fail because those are primarily the tools
that we use in our environment. But ProjectConsole can be used to collect data
from any of the tools in the tool set. ProjectConsole comes bundled with the
Rational Suite products, so any tool in the suite is a candidate for data collection,
plus select third-party products, such as Microsoft Project.
In determining what is useful data, you might discover that what's
useful to you might not be useful to somebody else. There's a time and a place
when you're exploring and learning that you put together what you think constitutes
good data. You create charts and graphs and use the tool, and then you can use
that as an example when you want to start rolling this out for people to use
to see what the tool can do. But seek input because the people who give you
input are more likely to use the tool than if you to just create something and
throw it out there, telling people that it's available. After all, good-looking
reports are not useful if the data isn't of any interest to anyone.
How to Display the Data
There are two different ways to display the data: the dashboard
format and the report format.
Dashboard Format
The dashboard format provides charts, graphs, and tables. Management loves charts and graphs. Most of my upper management
don't want to take the time to read a lengthy text report if they can look at a chart or graph and obtain the data in just a few minutes. With the dashboard
type of collection, the data is collected and stored over a period of time, which makes trend reporting possible. For example, we trend how long something
takes or how many open defects exist by severity level. One or two are open each day, and then you can see a trend over time.
Another advantage is that a user can drill down to reveal more information. Once you find a chart or a graph, perhaps a graph that
shows the state of a defect and the severity, you have the ability to drill down behind that data and retrieve who the owner of that defect is.
For example, perhaps there's a severity two level defect out there and your manager wants to know who owns it.
The data can also be displayed on the fly in different formats. If for one purpose a line graph would be more beneficial or you
have a manager who prefers line graphs, you can create a report so that your manager can see line graphs. If somebody else prefers
the look at a bar chart, then you can, simply by right-clicking and choosing a Display Type, display that same data in a bar chart.
This data can also be published and used in reports to be printed out and distributed. One drawback with dashboard is that the data can
be collected only one time a day. It's rare to have a reason to collect data more than one time a day. The only time that I've found it useful was
if we were entering a very busy time of the project and I wanted to see some progress made between where we were in the morning and where
we were in the afternoon. Version 2003 replaces the old data with the new data, and it will enable you to see your progress. In 2002, it would just
go out and retrieve the data again, and it would have double numbers for everything.
Stacked Bar Chart
Figure 1 is an example of a stacked bar chart that shows a distribution of defects by state and priority.
Figure 1: A Stacked Bar Chart
All of our severity one or priority one defects are in a closed or resolved state. Along the bottom, the state and its color-code
reflect the priority of the defect. There are a couple of red lines under Assigned and Submitted, and this is where, if a manager wanted
to see who has been assigned to resolve this high attention defect, they could take the cursor and hover over the red band in the assigned bar,
do a right-click, drill down, and they would be given an option of owner. They will be able to see who has been assigned to resolve that defect. If
you take the cursor and hover over any part of this graph, it provides the number of defects in that particular state or priority.
Trend Chart
Figure 2 is an example of a trend chart of open defects of priority one or two. In this case, there is a significant spike in the number of defects that were open on about April 15th.
Figure 2: A Trend Chart
A manager might question this and want to find out why all of
a sudden we have severity two defects being opened against our product. After
a little research, they may either find that there was a code drop, the code
was turned over to QA, there was more testing, more people testing the product,
and so thus justify why the sudden surge.
Change from a bar chart to a line graph is very easy. See Figure
3.
Figure 3: Changing a Bar Graph to a Line Graph
If you have a bar chart displayed on your screen, right-click
anywhere on the char and a menu is displayed. If you scroll down and highlight
display type, another menu with selections is displayed. See Figure 4 for
the Line Graph.
Figure 4: A Line Graph
In 2003 they implement the pie chart, and you will see more options
in version 2003.
Figure 5: Automatically Generated Report
In the first column we have the ID, which reflects the enhancement
request number. All of this data was requested by our sales group. I sought
feedback from them and asked them what would be useful to see and what didn.t
they care to see on their report. They wanted to see what the number was. They
also wanted the Headline, which is the second column. The third column reflects
the Description of the enhancement request. Then we put in a Customer Priority
and Qualification column that allowed the sales reps to add information. This
gave them a chance to justify an enhancement by saying that we have this big
customer who says they.ll close in fourth quarter if we can get this feature
into our product. It allows them to provide a little extra information to us.
Then we have the Submitter column and the Date that the enhancement record was
submitted. This is good for follow-up; the sales manager can go back and follow
up with the person who opened the enhancement record, and it also gives them
a chance to see how long the enhancement record has been sitting in the system.
The next column, the Owner column, is definitely a benefit once
an enhancement request has been selected to appear in our next release or a
future release. It is then assigned an owner from the architect team or from
the development team to start working on that particular enhancement and that
person.s name is then assigned as the owner. Then the final column, the State
and Resolution column, tells them if their enhancement request has been looked
at yet, is still setting in submitted state, has been assigned to somebody,
is open and being actively worked upon, or has been declined, and why has it
been declined. As an example in the last line, there is a particular enhancement
request that was actually a duplicate of one that already existed.
Figure 6: A Test Manager Report
Our QA manager often gets the status of whether a testing script
has passed or failed, but she doesn.t really know to what extent a test script
may have failed. If we have 50 verification points within a test script and
if one fails, the whole script fails. This report gives a line-by-line account
of each verification point within Test Manager and tells whether it passed or
failed, so she can see whether this script failed because one step failed or
because 20 of the steps failed.
There is an enhancement to the version 2003 Test Manager that will
enhance this report. The verification point is very non-descript; you don.t
know what the verification point was exactly, but in Test Manager 2003 they
have appended a description, number of characters, that will go into the log
that can now be pulled by this report. It.ll be a little more descriptive so
that you will be able to see what exactly passed and what failed. The final
column gives you the test date and time that it had been run. The first column,
the script column, is a system-generated code that may or may not be useful
depending on how you utilize that in your company.
Figure 7 shows the template that was used to create that Test Manager
report.
Figure 7: Sample Template
The ProjectConsole commands or the SoDA commands are enclosed in
square brackets. The fields that are being retrieved from the product are closed
within greater than or less than signs. Everything else is considered text,
which shows you the power of your customization. ProjectConsole is integrated
with Microsoft Word, so the table in the center of Figure 7 is simply a table
that was created in Microsoft Word, and then the data was placed inside of that
table.
The comprehensive code listing is shown in Figure 8.
Figure 8: Comprehensive Code Listing
This report demonstrates the power of ProjectConsole. I had a development
lead approach me and say that he wanted to see everything that goes into a release
that he is responsible for. He wanted to see the baselines, the activities,
and the defect numbers. He wanted to know who worked them, what files were used,
and the comments that the developers put on each file. The first number of lines
. Project VOB name, Project Name, Stream, Latest Baseline and Activities Headline
. all come from ClearCase. The Defect Number and Owner come from ClearQuest,
and it was quite challenging figuring out how to merge the two to get the data
to come together in a meaningful format so that it all associates and flows
in this way. This is just a snapshot of the report. The entire report for this
release was actually over 200 pages long.
We would like to be able to port something like this to an XML format
so that certain categories are collapsible. If somebody.s not interested in
looking at the change set, they won.t have to; they can just close it down.
Then, it would also be expandable for those who do want to see that information.
Figure 9 shows the template behind this report.
Figure 9: Sample Template: Text Listing
This template appears to be fairly simple and straightforward, but
it probably took six months off and on working with this and working with ProjectConsole
tech support before we were able to troubleshoot all the issues that I was having
with it and actually get this report to flow and generate for my developer.
He was quite surprised when I came back to him a couple months after his request
and said remember that report you asked for? Well, now I have it. He was quite
happy with that.
Audience
Who uses ProjectConsole? Basically, anybody who collects data and
tries to generate reports for upper management or for any purpose in the company
from the Rational tool products is a good candidate to use ProjectConsole, including
the product support engineer, sales engineers, technical leads, and managers.
An example, at Open Network, of those who are currently using this or have it
available to them is the Change Control Board. The Change Control Board is made
up of sales people, development leads, configuration management, and QA managers.
They come from all areas, and it was out of the Change Control Board that I
was able to create that first report, the sales group enhancement request report,
and that.s where their request came from so that they could see the status of
their own enhancement requests. The report that we just looked at was requested
by one of my development leads.
Other ways that the development leads may utilize ProjectConsole
is to assess the workload of the developers that they.re managing. They can
go into dashboard and see a distribution of who is working what defect, what
state they.re in, the priority of those defects, and be able to then balance
the work load or shift the workload if there.s something critical that comes
up.
For managers, the one example that I gave earlier was that of the
quality assurance manager who uses our testing status report. She uses this
on a weekly basis to compile the data to provide to our vice president of engineering.
We also have a report that was created for a product support manager. He manages
an area similar to a help desk, and he has to report on a monthly basis on how
many defects came into the help desk, how quickly they were resolved, how many
are closed at the end of the week, and data such as that. I.ve been able to
help him with that report. He used to do it all by hand, going through the defects
assigned to his group and doing the calculations, but now I can provide him
with a list at the end of the month of all the defects that were assigned to
his group.
Lessons Learned
This is how I introduced ProjectConsole at Open Network. First,
as I was learning the tool, I was creating charts, graphs, and reports that
I thought reflected useful information. After I had a sample and I started feeling
comfortable with the product that I could then generate reports and graphs that
others would request of me, I would contact the individuals and informally show
them the product and ask them if they saw any use for the data that I had or
if they had any ideas that they could give me to create reports that would benefit
them. Then I would create the report to satisfy that need, and that was the
best way to get people to use the product. To emphasize, the input from others
is essential for success when you.re implementing ProjectConsole.
Now the current administration of the product rests solely with
me. We collect all the data on a daily basis, and I.m the single focal point
for people who want reports generated. As the product becomes more widely used
and there is a need for people to be able to create their own reports, we.ll
move into that range, but right now, being in the beginning phases, it.s best
if I just create them. In the fast-paced environment, most people don.t want
to spend the time learning it. They would rather just come to me and ask for
it.
Capabilities
I have to expand the use of ProjectConsole. I went to dig into the
ClearCase data a little more. My vice president occasionally asked me to give
him a count of source lines of code, and I used to run a utility on each VOB
and then clean it up and give it to him that way. Now I can use ProjectConsole
to generate that for him, have it out on the Web site to look at whenever he
needs to, and he doesn.t have to approach me to have that report generated.
We.ve barely begun looking at capturing change set information and associating
it with the ClearQuest defects and activities, and we want to move forward with
that to make that more usable and more available and maybe even taking it down
to seeing changes between builds and baselines instead of the overall release.
In Test Manager, I plan on looking into refining our reporting methods, and
also being able to get down to a finer detail. Eventually, I hope to totally
replace the report that my manager is currently providing to upper management.
Right now she takes the data that I give her, and she still creates a format
that she.s used to and provides to him. I think that I can provide all the information
that she needs in a format she desires within ProjectConsole.
With Rational ClearQuest, I want to look more into using data transformations
so that I will be able to take the data that the product support manager needs
on how long a defect is open through a data transformation and actually be able
to calculate that time for him.
Benefits
ProjectConsole saves an enormous amount of time for those who have
manually collected and compiled reports. My QA manager used to go around to
each tester, and we had a spreadsheet out on a share point. They would have
to put their name next to the script that they ran, and they were supposed to
update it with how complete they were in running that script. Being busy, she
would have to go around and remind them or ask each one of them how far along
they were. They would say something like .About 75%,. and that.s very subjective.
Now, with ProjectConsole, she can go out and see what scripts have been run,
which ones are completed, which ones pass, which ones failed, and the testers
can be left to do their testing.
It.s also very convenient to have all the metrics available on one
Web site that can be accessed by the whole company. We don.t want to have tools
installed on people.s machines. I know my vice president doesn.t want the tools
installed on his machine, and he would much rather just go out and view a Web
site where he can get the data that he wants and not have to worry about how
to maneuver around different tools.
We found that assignments can be made more efficiently by the development
lead by viewing the product workload and the defect workload through the dashboard.
They can more evenly distribute the defects so that defect turnaround time essentially
becomes much quicker because they.re not overloading one person, and if it gets
to that point, you can see it right away. Most of the time, they assign a defect
and they know who is the good person to work on it, but they really don.t look
at the workload that person already has.
Conclusion
Finally, one of the main reasons that I was excited about implementing
ProjectConsole at OpenNetwork was the fact that I work with Rational tools on
a daily basis. I know the value that they have to me and to the people that
I support. Oftentimes, upper management only hears about Rational tools when
it comes time to write the check for the maintenance agreement, and that.s when
they come down and ask me if we really need this tool. ProjectConsole gives
visibility to all of the Rational tools to a higher level, and they realize
some benefits of the products that they.ve chosen to purchase and continue to
maintain.
Questions
Q: Do you use the publishing side to
publish reports and documents?
A: Yes, that.s a reporting format, and
the templates that we use to actually publish the reports and the documents.
Q: Do the reports print in ProjectConsole?
A: Yes they do. From within the interface,
you just choose print.
Q: If you publish a report, does that mean publishing it to
a Web?
A: Yes, the ProjectConsole is Web interfaced,
and you can publish that to the Web.
Q: Can you use third-party software, pull third-party software
into the product and use that?
A: They have the interface currently set
up for Microsoft Project. Other third-party products, anything that can be exported
to a CSV file, can then be imported into and used by ProjectConsole.
Q: Is there a dynamic link?
A: There.s no dynamic link. You would have
to export to a CSV file in order to bring it in.
Q: What lessons did I learn when I was trying to create the consolidation
report?
A: One of the lessons was don.t tackle
a very large database for your first try. It was an important lesson learned
because it ran forever, and by the time I found out that it wasn.t going to
work, I had already invested, 2, 3, 8, 12 hours into it, collecting the data
and trying to generate the report. The other lesson I learned is that ProjectConsole
tech support is invaluable. I highly recommend referring any questions to them.
They.re very helpful. They get back to you in a very timely manner. They.re
very knowledgeable about the product, and they can help you work through the
tough issues.
Q: What do you need to basically get this up and running quickly,
right?
A: ProjectConsole comes bundled with the
Rational Suite of products. Currently in 2002 and also in 2003, if you.re doing
the actual development of ProjectConsole creating the reports and such, you
will use a suite license. Now in 2003, for the people who access that, they
do have a lower-cost license available, a ProjectConsole license, but for licensing,
you will be using a suite license in your development of it. I have also found
that when I started using ProjectConsole, there is a modular tutorial that I
highly recommend. It is the one that is with ProjectConsole. If you want to
show your manager how a Requisite Pro report would look, you can go through
a tutorial, and it will take you step by step and help you create a Requisite
Pro report.
Q: How long do I keep my data in the data warehouse and how does
the response time get as the data warehouse begins to get large?
A: We haven.t hit the point where we have
so much data in there that it.s actually slowed to any noticeable level for
us, and we also haven.t hit the size level where we have had to make a decision
on what to do on the data warehousing. We currently have all of our data still
in the warehouse so that we have all the historical data from the time that
I started with this. I think we.re set up at about ten meg, and then it grows
from there, so it.s probably between ten and twenty meg right now.
Q: Have I pulled anything from Req Pro or Rose yet?
A: No, we.re not currently using Requisite
Pro and not many of developers are using Rose, so I haven.t had opportunity
to create reports using those tools yet.
Q: How long did it take me to get started implementing ProjectConsole?
A: It was actually a very quick startup
time. I went through the tutorial, and it showed me how to set up my data collections
and how to create the graphs and charts. It takes you through step by step,
so it is an easy transition from going to the tutorial to applying it to the
company.s data. I probably had it up and running in less than three weeks.
Q: What is my strategy for pushing this out to the users and
concerns about understating or overstating the data based on their novice knowledge?
A: The answer to the first part of your
question, how do I plan on pushing the product out for people to use, is by
soliciting feedback from them. If I approach them and say that this is a sample
of what I can do and ask them if they see any benefit that they can realize
by having me create a similar report or charter graph for them, in that way
they take ownership of it because they.ve contributed to it, and they.re more
likely to use it. As far as overstating or understating the data, I guess I
would be taking an advisory role on that and just helping them to understand
what they.re requesting and what the uses of that would be and the implication
of that data.
About the author  | | | OpenNetwork partners with technology providers, consultants and integrators who complement OpenNetwork's product vision. Our membership in the Microsoft Gold Certified Partner Program is key to our role as the leader in .NET security. We also partner with many leading J2EE-based vendors. |
Rate this page
| 
