Guide to Running Software Development Projects
© Copyright International Business Machines Corporation 2003. All rights reserved.
Most software projects fail. In fact, the Standish group reports that over 80% of projects are unsuccessful either because they are over budget, late, missing function, or a combination. Moreover, 30% of software projects are so poorly executed that they are canceled before completion. In our experience, software projects using modern technologies such as Java, J2EE, XML, and Web Services are no exception to this rule. This article outlines the top ten factors for improving the success of your software development projects. Industry leaders, such as the Standish Group, also document critical success factors for software projects.
Project Success Factors
1. Recruit skilled and experienced people - Today's environment is more complex than ever. Tools like WebSphere® Studio help; but in the end, inexperienced people produce, at best, only mediocre results and, in most cases, fail because they do not understand good project management and the best ways to apply new technologies. An excellent Project Manager and Architect or Technical Lead provide joint leadership of the project. They set the tone of the project and have a vast impact on its ultimate success. If you have this type of resource, treat them well, very well. The project manager and technical lead should interview the other team members and they should have the final say about who joins the team. The rest of the team also needs to have above average skill and inexperience. Poor performers need constant attention and usually never "get it". In the end, they usually drag the team and slow down the progress. However, that does not mean the team cannot have junior level people. Often, they are more motivated to do well and shine given the opportunity. For example, in a team of 20 people, there might be 2 leaders, 6 senior resources, 9 mid-range resources, and 3 junior resources. This team of 20 people is broken down into 4 or 5 small sub-teams each with a team lead. IBM Software Services and IBM Global Services (IGS) have experienced Project Managers, Architects, Technical Leaders, and Consultants who can help with your project.
2. Use leading edge, not bleeding edge technology - Many Fortune 500 companies have successfully used mature technologies, such as J2EE and the WebSphere product family, for software projects that have had huge impacts on the way the company did business. In some cases, it is necessary to apply a leading edge technology that helps gain a distinct advantage over competition. However, there are risks with such a strategy and in this case, it is even more important to have excellent people on the project. Because there are few people with this kind of experience, it is important to get outside expert help. Projects that employ bleeding edge or untested technology should consider research projects. It might be useful to perform early proof of concepts on an emerging technology. However, it is unrealistic that a project based on such a technology can deliver in the same way or with the same costs as a project using more mature technology.
3. Use the appropriate development process - The nature of modern software projects demands a spiral-based development process, such as the Rational Unified Process (RUP), one of the iterative IGS methods, or even an agile methodology, such as eXtreme Programming. A spiral process has multiple phases that successively decrease the project risk. At the end of each phase is a go or no-go decision. In the early phases, prototyping is used to explore new technologies for the team or a user interface. For example, the RUP defines roles, tasks, and artifacts for each phase that serve as reminders of what the project team needs to think about for the project. The most important factor for any project is not so much which process is used, but how well it is applied. The Project Manager and Technical Lead need to believe in and understand how to tune the process to the problem at hand, and how to execute the process using best practices. A process provides guidance and reminders on what needs to be done. On the other hand, straying too far from the principles of a process is also a recipe for disaster. The companion article, Best Practices for Software Development Projects, has more detail.
4. Provide the right tools - Any software project needs the appropriate tools that provide productivity aids for the team. Tools include the right hardware as well as design, programming, and test productivity aids. The cost justification of these tools is relatively easy. For example, suppose an IDE such as WebSphere Studio Application Developer saves a programmer 5 hours a week, and that on average, the programmer costs the company $50/hour. It is easy to see that the return on investment (ROI) is worthwhile. The same applies to ensuring that the team uses the latest and fastest PCs for development and appropriate test environments are provided for Quality Assurance, user acceptance, and deployment testing. Training in the new tools or techniques is also essential to ensure that they are used to their full advantage.
5. Use source control management - Use a source control management (SCM) system when the project begins. All documents, not just source code, are under version control of the SCM system. This allows the team to go back and view the history of the project and to retain copies of previous versions of all project related documents, such as use cases, architecture and design documents, and test scripts and plans. I recommend using an enterprise level SCM product, such as Rational ClearCase/ClearQuest.
6. Apply sound estimating techniques - Most projects overshoot their estimated schedules from anywhere from 25% to 100%, but some projects have achieved schedule prediction accuracies within 10%. Without an accurate schedule estimate, there is no foundation for effective planning. However, in the early stages of a project, estimates of time and effort are fuzzy. These estimates should include contingencies and might be quoted as +100%. Software development is a process of gradual refinement and so it is with estimation. Estimates become more precise as the project progresses. At the end of a project, the actual time and effort are known. Most software engineers tend to underestimate, so it is natural to expect some growth in the cost of a project. When estimating a schedule, be careful not to include too much schedule compression. There is a point when the team cannot achieve the tight schedule and in the end, will miss it by a wide margin.
7. Breakdown effort into mini-milestone tasks - Mini-milestones are smaller versions of milestones. Major milestones are the end of a phase or increment. To achieve that point, a project needs mini-milestones along the way. Mini-milestones take less than 1 to 2 days effort and are measured in hours. The advantages are improved status reporting, fine-grain control of knowing if a mini-milestone is missed, improved motivation because every day or so a mini-milestone is achieved, and reduced schedule risk. To avoid problems in any project, it is suggested that mini-milestones are used from the beginning of a project. The best method of documenting and tracking mini-milestones is using a spreadsheet. The project plan from a tool, such as MicroSoft® Project, is better used for higher-level tasks. Naturally, only the current phase is broken down into mini-milestones. Successive phases are broken down just-in-time. Although developers consider mini-milestones an inconvenience, this problem offsets team leads and individual developers the ability to define their own milestones and to spread the load of project management and tracking. Often a task defined by the technical lead becomes larger once a developer breaks it down into mini-milestones. Sometimes the technical lead can suggest alternative approaches that are faster or more maintainable, and on other occasions, agrees with task breakdown and has to get more time assigned to the task. This early mini-milestone planning work prevents a potential disaster.
8. Track all project hours - It is important to track the time spent by everyone on the project, not just the hourly paid consultants and contractors. The advantages are that the hours for an individual are compared to the planned hours. Steps are taken if that individual has been diverted onto other tasks. Also, the actual hours are compared to the estimated hours, which in turn provides feedback into the estimating techniques for the next project phase or the next project. An estimate of completed hours for a mini-milestone also gives a heads up on any overruns so they can be corrected. Using the mini-milestone technique requires time and effort from the technical lead, team leads, and individual developers. At least once a week, the status of each developer's efforts is rolled up using the spreadsheets so that the project manager can update the percentage complete on the each of the higher-level tasks. This spreads the project management load to other team members. Tracking project hours take a more work, but they provide very effective project management.
9. The only constant is change - For most projects, the requirements change less than 5% per month. These changes occur for many reasons, such as someone failed to ask the right questions at the right time, the problem being solved has changed, the users changed their minds or perceptions, the business environment changed, or the market changed. Feature creep is the most common source of cost and schedule overruns. In the early stages of a project, there is a large amount of churn in the requirements. At some stage (usually at the end of the second phase), the requirements need to be settled and essentially locked. A change management process is then implemented by a "change board" consisting of representatives from each of the areas involved in the project, such as business, marketing, development, QA, user documentation, support, and project management. The change board is responsible for assigning the change to the right people, getting clarification on the change, and sizing estimates from all parties. With enough information, the change board decides whether to accept or reject the change. Once a change has been accepted, it is added to plan and the schedule is changed. A project with changes may be delivered later than the original plan without changes, but it is still successful because it met the revised schedule and stakeholder expectations. A project that changes by more than 5% after the change board is started indicates that either it was poorly defined or it is out of control and will probably fail.
10. Project leadership - It is important that company executives support a software project with a single executive responsible and accountable for the outcome. This key executive not only provides the vision, but also helps support the team by getting and controlling the resources needed for the project. It is also important that this executive does not meddle or micro-manage the team. The executive trusts the team to deliver.
This article provided a list of ten factors that help improve the success of a software development project. By following these guidelines, you can have a better chance of completing your project within budget and within the scheduled timeframe, maintaining a high performance team, and minimizing feature creep. See Mike's companion article on best practices, Best Practices for Software Development Projects.
- Best Practices for Software Development Projects
- IBM Software Services Web site
- McConnell, Steve, Rapid Development, Microsoft Press, ISBN 1-556-15900-5
- Cantor, Murray, Software Leadership, Addison-Wesley, ISBN 0-201-70044-1