Level: Introductory Denton Tarbet, Project Management and Planning Consultant, Galorath Incorporated
15 Jan 2008 
from The Rational Edge: Read how a Ready-for-Rational partner helps software development organizations better estimate the software project development lifecycle.
From The Rational Edge.
Attend the Galorath Webinar! See link and details at the end of this article.
What is most likely to get you promoted or fired? What causes your palms to sweat the most? Is it that you won't be able to grasp the requirements? Or architect a solution? Or generate code? No. What really gets your heart racing is when a project is finally approved based on your projections for cost, staffing, and delivery dates.
Imagine that you had a solution that would enable you to determine with 90% accuracy the actual cost and delivery date for your software project. Following a consistent Software Development Lifecycle Process and employing tools designed to streamline and automate that process will ensure that many more of your software projects are delivered on time, on budget, and as specified ... really.
Galorath Incorporated is an IBM® Rational® partner. We have invested more than two decades developing solutions to help government and commercial organizations plan and manage complex projects. Our SEERsolutions combine an intuitive interface, extensive project-applicable knowledge bases, sophisticated project modeling technologies, and rich reporting features to expedite the planning process and keep projects on track. Government agencies, Fortune 100, and Forbes International 500 companies currently depend on SEER by Galorath for mission-critical projects including financial systems, airline operations, global positioning systems, and the international space station.
This article provides an introduction to processes and technologies for planning and managing the software development lifecycle.
The 10-step estimation process
Today, most software organizations incorporate one or more of the following to evaluate software projects: 1) ballpark/rough order of magnitude/SWAG estimating, 2) top-down/constraint-driven estimating, and/or 3) bottom up/design-driven estimating. Despite the difference in planning methodologies, what all of these approaches have in common is that they tend to be executed as one-off practices dependent on the availability of scarce, over-committed personnel and manual or minimally-automated processes such as spreadsheets and other homegrown tools. Such one-off practices are, by definition, inconsistent, unpredictable, and highly susceptible to human error. Individual planners possess diverse capabilities and project histories; they may be overly optimistic or pessimistic; they may be influenced by internal politics or other factors unrelated to the project; or they may simply overlook some of the less obvious project elements.
Applying an established, proven estimation process that is well integrated with development processes helps ensure that project plans are credible and achievable, meet or exceed customer expectations, and support other management activities by providing accurate and timely planning information. The following 10-Step Estimation Process was developed by Daniel Galorath and Michael Evans (Software Sizing, Estimation, and Risk Management, Auerbach Publishing, 2006), based on more than two decades of research focused on software estimation best practices.
Step 1: Establish estimate scope and purpose
Define and document expectations. When all participants understand the scope and purpose of the estimate, you clear up contradictory assumptions, establish a baseline for measuring the effect of future changes, and head off potential misunderstandings.
Step 2: Establish technical baseline, ground rules, and assumptions
Identify the functionality included in the estimate. If detailed functionality is not known, document ground rules and assumptions, and what is and isn't included in the estimate. Issues of Commercial Off-the-Shelf (COTS), reuse, and other assumptions should be documented as well.
Step 3: Collect data
Identify the data required and the information holders. Provide participants with questions and clear definitions beforehand. During interview confirm data is realistic and valid. Build and refer to repositories of project histories. Be sure to capture uncertainty.
Step 4: Determine size and scope
If circumstances force you to compromise on your estimating methodology, spend the bulk of the time you have on sizing the project (the use of sizing databases and tools can facilitate this process). This is arguably the most critical step in determining the ultimate success of a project. Sizing and scoping drives every aspect of the project, constrains the actions that can be taken in the development or upgrade of a product, and limits available options.
Step 5: Prepare baseline estimate
Estimating a project should not be thought of as a one-time activity. As projects progress, unknowns become knowns, and new and unexpected issues and opportunities arise. Life happens. Refreshing estimates and documenting estimate deltas throughout the development process ensures that 1) the evaluation of trade-offs is based on increasingly reliable data, 2) the organization has a "most accurate" estimate at any given point in time, and 3) estimation accuracy improves over time.
Step 6: Quantify risks and risk analysis
Risk can be characterized in terms of a loss of time, quality, money, control, or understanding. The loss associated with a risk is called the risk impact. Risk cannot be eliminated, but it can be planned for. Risk, by itself, will not threaten the success of a project if it is recognized and addressed in a timely fashion.
Step 7: Validate and review
Validating and reviewing your estimate provides a systematic confirmation of your estimate's integrity. This step enables the organization to be more proactive in adjusting project variables, and justifies confidence that project data is sound, methods are effective, anticipated results are accurate, and focus is properly directed.
Step 8: Generate a project plan
Generating a project plan employs the project estimate as a basis for allocating and communicating specific cost, resources, and timelines in a detailed function and task-oriented work breakdown structure.
Step 9: Document estimate and lessons learned
Each time you complete or update an estimate and upon completion of a project, you should document the information that proved pertinent and record lessons learned. By doing so, you document a "best effort" process, capture actual results with which to calibrate your estimation models.
Step 10: Track performance (measurement and feedback)
In-process information should be collected continuously and compared against the original plan and baseline. If performance varies consistently or significantly, corrective options should be evaluated and implemented to get the project back on track. Performance feedback should also be reviewed and project data retained to refine the estimating process, itself.
Parametric/predictive software modeling
While following a structured estimation process will go far in ensuring that estimates are accurate, much greater gains can be achieved by coupling this approach with technologies designed specifically for project estimation and governance. Parametric/predictive modeling software enables users to assess what is known about a project and to model or simulate what is unknown based on extensive and meaningful, real-world comparative data.
Unlike most project management tools, which focus on automating features or workflow, parametric, predictive modeling tools help organizations model and optimize project feasibility and ensure that projects meet established delivery guidelines. Parametric modeling takes its name from the project parameters or variables that are modified during the project simulation process. Parametric models are built from a set of mathematical equations. These may be standard equations found in reference books, proprietary equations developed by consultants or vendors, or some combination of the two. In order for parametric models to have any validity they must be based on or proven using actual project data. It is the sophistication of the data analysis methods and the extensiveness of the underlying project data which determines the effectiveness of a modeling solution.
Parametric methods are very useful for subjecting uncertain situations to the rigors of a pre-defined and proven mathematical model. They can usefully embody a great deal of prior experience and are less biased than human thought processes alone. While specific features vary from vendor to vendor, robust estimation tools will provide an open architecture for integrating departmental productivity solutions and enterprise applications, and incorporate the following application elements:
Interface: An intuitive interface for defining and describing projects. Users can generate a new project from an existing project "template" or by adding and defining individual work elements. A series of pop-up windows and annotations guide users through the process of defining project scope, complexity, and technologies.
Simulation/modeling engine: Sophisticated, sector-specific mathematical models derived from extensive project histories, behavioral models, and metrics.
Knowledge bases: Knowledge bases serve as a virtual "in-house expert," providing default project definitions, values, ranges, and calibrations based on comparable project histories. Enables users to develop first-look estimates when very little information is known, and to refine those estimates as details become available.
Output: A variety of charts, graphs, and reports for quickly summarizing and presenting project outcomes and alternatives as well as work-in-progress.
Figures 1 and 2 show an example of the parameter input GUI and results page output generated by one such estimating tool, SEER by Galorath.
Figure 1: SEER input
Click to enlarge
Figure 2: SEER output
Click to enlarge
Today's leading parametric modeling solutions enable users to describe a project in varying levels of detail; obtain an estimate for project cost, effort, and duration; analyze and test trade-offs by changing one or more project variables; and provide extensive reporting capabilities, including specific percentage probabilities for achieving project objectives and how those probabilities could be improved by small changes in underlying constraints and assumptions. Top-shelf tools may provide embedded software sizing "wizards" to assist in software sizing, the single most significant driver of development cost, effort, and schedule.
Additional parameters will likely include:
- Project platform
- Development methodology
- Governing standards
- Staffing levels, capabilities and experience
- Development environment complexity
- Requirements stability
- Confidence level
- Target/host systems
- Schedule considerations
- Reusability
- Integration
- Labor rates
- Maintenance levels
- And more
Project modeling in action: A decade in the life of the Social Security Administration
The Social Security Administration is an enormous organization with enormous responsibilities. In 2007, the SSA distributed over $585 billion in retirement benefits to 50 million Americans. Headquartered in Baltimore, Maryland, the SSA has a workforce of approximately 62,000 employees in ten regional offices, six processing centers, and 1,300 field offices. As millions of baby boomers reach retirement age, the demands on the SSA will only get more pressing. Keeping up with the spiraling number of Social Security recipients requires multiple, complex computer systems and specially-designed software capable of tracking the earnings and benefits of over a hundred million people. These systems are written and tested by the SSA's Office of Systems.
In August of 1997, the Office of Systems decided to deploy a parametric modeling solution (SEER by Galorath). At the time, the SSA had approximately 100 programs in use with another 30 to 40 in development. Almost all were written in COBOL, a language which at that time ran 80 percent of the world's businesses.
1
SSA was striving to comply with a new process standard, the Capability Maturity Model (CMM), and to centralize its software estimation capabilities.
Dennis O'Mailey, now retired, was the leader of the SSA's Software Measurement team, the group responsible for estimating potential software development projects and tracking the progress of existing projects. "With more than 600 programmers developing and maintaining many systems, we had a need for an accurate software estimating tool," he said. "The general feeling was that the estimates we had were not good."
At the time, individual project managers were responsible for creating their own estimates. "A good estimate depended on the knowledge and experience of the project manager," said O'Mailey. "Project managers who were on a project for many years were more accurate than newer ones. But on many projects we missed things by orders of magnitude."
Initially, the SSA deployed parametric modeling on just a few dozen projects; they used the tool to determine maintenance costs and whether or not to continue development. But the SSA was also working to bring the organization into compliance with CMM Levels 2 and 3. "CMM was important because it meant your software development operation was mature and that you had a standard methodology for developing your systems," O'Malley stated. CMM required that project effort and cost estimates be developed according to a documented procedure. SEER enabled the SSA to meet these requirements.
Today, the software development shop at the Office of Systems employs about 3,200 analysts, developers, database modelers, and other systems people who, in addition to producing software for Social Security, also design and build applications for other federal agencies. "There are still some COBOL projects," according to Jim Denney, O'Malley's successor at the SSA, "but now they are more than likely to be Web-based languages. We've got Java and JavaScript, HTML, Visual Basic, and Cold Fusion." Recently, his organization was hired to develop the software necessary to implement the Medicare Part D Prescription Drug Coverage Initiative. "Rather than outsource it to a private vendor," said Denney, "they contracted for Social Security to build it for them. By inter-agency agreement we are reimbursed dollar for dollar for every hour of software effort we put into the Medicare program. We ended up hiring about 600 extra people just to support it."
When he took over for O'Malley, Jim Denney saw additional opportunities for the solution. "We asked ourselves, should we farm estimation out to a hundred different project managers or have a dedicated group of estimating people who could become very adept at seeing how one project related to the next," says Denny. Today, estimates are a centralized service provided by the Software Measurement team.
During a typical year the 90 or 100 most critical SSA projects are estimated using SEER parametric modeling solutions. The process begins with a preliminary, bottom-up analysis using Microsoft Project. Denny's team takes a more top-down approach, using function points to build a second estimate. "Our standard is for a project manager to have two estimates, one from the bottom up, the other from the top down, like the difference between inductive and deductive reasoning," Denny says. "The project manager is then responsible for doing a cost-check comparison between the two estimates. What we are striving for is that the two be pretty close. This is where a seasoned project manager can confirm that he's on the right track." SEER also plays a role in the management of resources. "We try to identify over-burn and under-burn on projects by using a concept called fungibility," Denny states. "Resources that are assigned to a project in under-burn mode can be re-directed to a project that needs them more. That's one of the benefits of estimating: Parametric modeling gives us visibility into resource utilization, allowing us to do our jobs more quickly, more efficiently, and more accurately."
With an open software architecture, SEER enables individual organizations to refine their estimation tools by extending native Knowledge Bases with organization-specific project data. "About five years ago we gave Galorath developers information on 50 completed projects that they then incorporated into the SEER knowledge base. We now have a very robust project-tracking system because every person is required to record on a weekly basis what projects they worked on and what tasks they did."
"The real benefit of parametric modeling is helping project managers measure effort and schedule early in the project," O'Mailey says. "The key is to know early in the game and throughout the life of the project how long the project will take and what resources it will require. If you manage your project from a good plan -- and an integral part of that plan is a good estimate -- there is going to be significant cost savings."
Rational-SEER-Rational: Integrating the Software Development Lifecycle
A standard Software Development Lifecycle Process automates project design, estimation, resource allocation, and tracking. The seamless integration between IBM Rational Software Modeler, SEER for Software, and IBM Rational Portfolio Manager supports this process, end-to-end:
Step 1: Evaluate a preliminary concept -- go or no go?
With SEER for Software you can define your project at a conceptual level using extensive and extensible project knowledge bases aggregated from thousands of completed projects. In a matter of seconds, SEER can simulate your software project based on this extensive project history and calculate an initial, rough order of magnitude estimate for cost, effort, duration, and risk.
Step 2: Develop the software design
Once you have determined that you want to move forward with a project, IBM Rational development tools can help you capture business process design and system requirements, develop use cases, and translate use cases into executable code.
Step 3: Develop detailed estimates
Export your Rational Software Modeler, or Rational Software Architect, or Rational Software Developer designs to SEER to refine your initial estimate and perform extensive trade-off analyses, modifying project variables -- alone and in combination -- to determine how best to execute the design, as shown in Figure 3. Provide executives and business leaders with specific percentage probabilities for achieving project objectives, and how those probabilities could be improved with small changes in the underlying constraints and assumptions.
Figure 3: SEER for Software (upper screen) and IBM Rational Development tools (lower screen) scan help you evaluate multiple options to refine and optimize your software design.
Click to enlarge
Step 4: Assign Resources and Manage Portfolio
Export your finalized SEER estimate to IBM Rational RPM, to centralize project and program data, and ensure that resources and priorities are managed optimally across the organization and over time and that no resource is double-booked or under-utilized, and where resource conflicts will occur if projects are delayed (see Figure 4).
Figure 4: The SEER for Software Project Monitoring and Control (upper screen) and Rational Portfolio Manager (lower screen) allow you to centralize project and program data, and track project performance and progress to ensure that resources and priorities are managed optimally.
Click to enlarge
Step 5: Hit project deadlines
SEER enables users to measure actual project performance and progress against expected outcomes (schedule, resources, and defects). Where mid-course corrections are required, users can determine which corrective actions are most likely to get projects back on track without seriously compromising project objectives. Project results can be used for feedback to improve processes, and can be incorporated into databases and repositories of project histories.
Projects delivered. On time. On budget. As specified.
Working in concert, IBM and Galorath solutions provide a seamless solution which enables software development organizations to automate and fulfill their planning processes:
- Providing a complete, seamless solution across the software development lifecycle
- Helping organization understand and optimize cost, time-to-market, and functionality trade-offs
- Providing best-in-class estimation capability (cost, schedule, effort, risk, and reliability)
- Enabling standardized estimation best practices across the enterprise
- Encouraging collaborative decision-making by providing a common language for lines of business and technology providers
- Supporting governance initiatives by providing a project baseline for measuring compliance to projected cost, schedule, and staffing objectives
- Ensuring that project processes and results improve over time
Notes
1 Gartner Group, 1997
Resources - Participate in the discussion forum.
-
Attend the webinar
Integrated Project Lifecycle Modeling: IBM Rational and SEER by Galorath
Date: 01/23/2008 Time: 12:00pm - 1:00pm (GMT-05:00) Eastern Standard Time (America/New_York) By: Global Rational Community (GRC)
Today, most software organizations incorporate one or more of the following to evaluate software projects: 1) ballpark/rough order of magnitude/SWAG estimating, 2) top-down/constraint-driven estimating, and/or 3) bottom up/design-driven estimating. Despite the difference in planning methodologies, however, what all of these approaches have in common is that they almost always rely on the availability of scarce, over-committed personnel and manual or minimally-automated processes, such as spreadsheets and other homegrown tools. Such one-off practices are highly susceptible to human error.
This webinar will include a brief demonstration on the SEER software modeling solution to show you how quickly a rough order of magnitude estimate can be created, as well as give you a glimpse of the drill-down power of a good software modeling solution.
Register for this webinar here. https://events.webdialogs.com/Portal/WipEvents/register.php?id=360e00c90e7d4f82aeff738aa0e7da1f&sourceID=
- A new forum has been created specifically for Rational Edge articles, so now you can share your thoughts about this or other articles in the current issue or our archives. Read what your colleagues the world over have to say, generate your own discussion, or join discussions in progress. Begin by clicking HERE.
-
Global Rational User Group Community
About the author  | 
| | Denton Tarbet's professional background includes more than twenty-five years as a director of engineering and senior consultant in information technologies, software architecture, cost estimating, and earned-value analysis. He has extensive experience managing software planning and cost estimating projects, and has developed and calibrated software models for NASA, the Department of Defense, and numerous commercial and government organizations worldwide. He lectures on a wide range of project management topics, including systems and software project management, configuration management, and software quality assurance and testing. He holds a Ph.D. in Systems Engineering and a Master of Science in Operations Research from the University of Houston. |
Rate this page
| 
