In this installment in the series about the discipline of application architecture, you explore the skills, tools, techniques, and milestones related to application development methodology. A well-rounded application architect must be able to apply a number of methodologies to the development of an application. The choice of method may be dictated by the organization or the nature of the project. Balancing disciplined command and control rigor and flexible, agile techniques is a critical component of a successful application development project.
The role of an architect is to translate the need for automation (requirements) into an appropriate structure (design) and the means of construction (methodology) within resource limitations (time, financing, skills). In previous articles in this series you learned about the basics associated with the requirements and design patterns. In this article you learn how to apply the appropriate techniques to defining the application development methodology.
There are many software development methodologies published in books, packaged as products, or maintained by standards bodies. If you design software for a large organization, there may be an established standard methodology practiced informally or as a mandatory formal method.
A useful and effective development methodology should include a spectrum of techniques that range from loosely managed to completely specified. The type of technique to use and the amount of process rigor to apply are driven by the amount of risk involved in the application design, skill set, technology maturity, scale, complexity, and criticality. At one end of this spectrum, the development team, working closely with the stakeholders, crafts an application solution to some set of identified and prioritized functions. Most of the rigor can be derived in the deployment aspect of the process, using continuous testing and integration of the application components, as shown in Figure 1.
Figure 1. Agile method schematic
Moving along the spectrum, a unified process methodology breaks up the development effort into specific disciplines (design, code, test, and so on), as shown in simplified form in Figure 2. Input from and feedback to stakeholders is more stylized and formal, as well as less immediate and continuous. This type of method is applicable when the requirements are well understood, with little or no reusable design and implementation assets available.
Figure 2. Formal method schematic
At the other end of the process rigor scale are product-line architectures where the application design, construction process, configuration, and deployment are pre-developed and customized using a set of variability points. This is similar (as indicated by the term) to production-line engineering. The materials, parts, and assembly processes are standardized. A simplified view of this type of method is shown in Figure 3.
Figure 3. Product Line method schematic
In a project with any scale at all, it becomes obvious that no single methodology, no matter how carefully tailored in any number of dimensions, will be optimal for all parts of the application. Some components of the architecture have fuzzy or unknown requirements, as in: "We need something to organize and prioritize delinquent accounts." Other components are instances of well-known and understood functional elements: "We need an application database maintenance interface." In even the most primitive development shop, the implementation of the latter starts by copying the last instance built or some existing archetype.
An organization or development team evolves reusable components and designs over time. As this happens, the development method for some application types moves across this spectrum from experimental creative processes to formalized reuse and variation.
Learning the ins and outs of the various methodologies is extremely valuable, but be sure that you also know their strengths and weaknesses. Also, apply the techniques that are appropriate to the application and the context in which you are performing the development. The architecture itself serves as the primary vehicle for dividing the application into structures that reflect a separation of concerns. That separation allows the architect to analyze the characteristics of each component independently of the others. The methodology to apply is just one of these characteristics and should be identified and supported by the architecture analysis.
So, you need some kind of application development methodology if you hope to succeed in building applications of an appreciable size. What exactly is involved in a methodology? How can you compare and contrast the various methods that are available? To address these questions, you can create a model of a methodology that defines the types of things that can be specified in a method, their attributes, and possible relationships. Many examples of this type of model exist, including the Unified Method Architecture, which is part of the Eclipse Process Framework (EPF). You can also look at attributes of the methodology itself, such as its profile, or the range that the method addresses in term of roles, application life cycle, and the activities that are specified.
Agile development methodologies are lightweight processes that seek to minimize the time lag between identifying requirements and the delivery of working code. High-level requirements are captured as general statements of desired functionality or as user stories that describe in narrative form how the system is used to achieve a goal. Requirements are formalized as test cases for the implementation so that development progresses toward passing the test cases, thereby delivering the required functionality with as little process overhead as possible.
Agile development processes are usually executed in small bites of a few user stories or requirements at a time. Scheduling is basically a time box method that defines the period of time for an iteration and the requirements that should be completed within that time frame. Each iteration should be large enough to deliver at least one significant functional aspect of the application but not so large that it dilutes the team's focus.
Depending on your organization, you as an architect are not ultimately responsible for the day-to-day management of the application development project. However, you should have a clear understanding of the issues and requirements related to management of the project budget and schedule. Part of the architect's role is to deliver the required functionality within the available time frame and finances.
You should be able to support accurate planning and scheduling with the help of the architecture by analyzing the complexity and risk factors inherent in the components. You can also specify particular skills or skill levels to specific components, which helps with accurate resource scheduling.
For more agile methods, project management consists of prioritizing requirements and facilitating team development cycles. In this case, the architect adds value by analyzing the impact and complexity of the requirements.
The iterative in iterative development refers to the fact that these methods involve repeated execution of the kernel processes. Depending on the specific methodology, the kernel may be more or less structured, such as the workflows in each IBM® Rational® Unified Process® (RUP®) iteration or the sprint in Scrum. Each iteration results in tangible progress on the project deliverables. After each iteration, the results are examined, and the plan is adjusted for the next iteration.
Methods like the Unified Process and related products such as RUP define a process framework based on phases and iterations. The development life cycle is defined as a sequence of phases. Each phase results in a critical milestone version of the system, as well as an evaluation of the phase and planning for the next phase. Within each phase, a set of parallel workflows define the activities performed and the artifacts that are used, created, or modified by each activity. The type of phase calls out which workflows are mandatory or optional and a weight that indicates the amount of effort expended on that workflow relative to the others. The distribution of work shifts from requirements analysis to design to implementation, testing, and deployment as the phases progress. However, all of the workflows are executed to one degree or another, even if only to tweak artifacts based on feedback from other flows.
Within a phase, the set of workflows may be executed multiple times, in iterations. Each iteration should have a clear duration based on a time period or milestones achieved. An iteration may further modify the distribution of work to focus on a particular area, such as a prototype.
The choice of a development method and the types of resources that you apply to a specific application component should depend partly on the amount of risk that component contributes to the project. Risk involves many things, including unknown requirements, new technologies, complex algorithms, performance and response time requirements, or critical correctness and reliability.
Identifying the high-risk components of your application design allows you to address those risks as early as possible. You may decide to evaluate off-the-shelf software for particularly risky elements. However, for components that provide a competitive advantage or use proprietary algorithms, this may not be possible. Deciding how much risk is acceptable depends on your environment and the competitive value of the application to the business. In general, you can rarely justify attempting a high-risk project where the payoff is not extremely high, such as a unique competitive feature. You can ameliorate risk by factoring your project and refactoring the high-risk areas. Eventually, you will have a clear map of your exposure and be able to see where to apply exceptional skills or effort.
Process definition and tailoring
No one process or even methodology works for everyone all the time. At some point, explicitly or implicitly, you tailor the process to suit your needs. You should be familiar with the basic methodology metamodel and be able to define and customize method elements. Define development processes for your projects by defining a process that takes into account the different characteristics of the application components.
There are many more tools than I can possibly even hint at here. I've identified a few examples that should get you started.
The EPF is a method definition framework. The framework defines a process and content metamodel for documentation of a particular method. The composer tool is an Eclipse plug-in that allows browsing, authoring, and publishing content. Using EPF, you create a process definition from existing elements in the method content, which consists of role, work product and task definitions, and associated guidance (checklists, templates, and so on). You can further tailor elements beyond what is defined in the method library by using descriptors to add information.
You can use the framework to author new method content or extend existing configurations. You can then use this new content to define a project process and publish it as full-featured Web content. Enterprise architects can use the tool to create new configurations. Configurations constrain or extend existing library content by omitting, adding, or specializing the method content that supports a specific project context.
IBM Rational Method Composer is a product version of this framework with extended functionality, built-in method content for RUP, and integration with the IBM Rational tool suite.
Rational ClearQuest® is a workflow, process automation, and issue tracking system. The product comes with a large set of prebuilt workflows and can be customized using a designer tool. ClearQuest integrates with the other products in the IBM Rational tool suite and offers a Web-based interface as well. Integration with IBM Rational ClearCase provides version and configuration automation for process artifacts.
Rational ClearCase® is a version and configuration management system that can support large-scale enterprise asset repositories and version control. The ClearCase product is integrated with IBM Rational tools, Microsoft® Visual Studio®, and even plugs version control functions into the native file system. You can create sophisticated rules, controls, and reporting functions within the asset repository to support configuration promotion processes and to integrate with other systems.
IBM Rational Software Architect
You can use IBM's Rational Software Architect to structure the artifacts and processes relative to the development of an application. You can create project templates that contain the generic structure of the solution, annotated with guidance on how to use the included pieces. IBM publishes modeling guidelines for structuring these templates using a library package to hold generic artifact components that you can copy into your model and customize.
As with tools, there are too many techniques to cover in this article. Here are a few to get you started.
Extreme Programming (XP) is a lightweight, agile method that emphasizes the delivery of working, tested code as quickly as possible. It is mostly suitable for small teams but can be scaled by suitably factoring the application into smaller components. The primary motivation for this method is to minimize the amount of process overhead (sometimes called ceremony in agile literature) that takes the team away from productive work delivering functionality. XP practitioners use test-driven development (TDD) and continuous integration to keep the development focus on meeting the requirements with working code.
TDD puts the test before the code. Application requirements are expressed as test cases that reflect high-level requirements and business rules that are possibly extracted from user stories. The test cases are written first, and implementation proceeds by writing code to satisfy each test case. Tests are run whenever code is committed to the project build. This integrates with the continuous integration technique.
Continuous integration means that the entire project (or some suitable subproject) is rebuilt on a frequent, periodic basis or when code is committed or integrated. If the project builds successfully, all the tests are run against the resulting system and reports are generated for review by the entire team. Anything that breaks the build becomes a high priority.
Scrum is a lightweight, agile methodology for managing software development projects. Business requirements are expressed in general terms and maintained as a prioritized list. The development team works without interruption for short durations (usually two to four weeks). Each interval is called a sprint. At the start of the sprint, the team evaluates the highest-priority requirements in terms of risk and scope and determines a set of requirements that will be implemented, tested, and possibly deployed at the end of the sprint. Daily meetings, called scrums, are informal reviews of progress and issues that impede progress.
During the sprint, the team is left alone to self organize and complete the required tasks. Any issues are raised at the daily meeting to be addressed by the project manager. Work for the day is planned. After each sprint, the progress is evaluated and the next sprint is planned. The requirements are re-prioritized based on new capabilities that have come up and on lessons learned during the sprint.
Use Scrum when you need to produce regular releases, to break a large project into smaller ones, or when requirements are volatile.
Crystal is a family of methodologies that are associated with a range of project contexts defined by the criticality (financial, legal, or reputation consequences of failure), project team size, and project priorities. The methodologies are coded by color and cover a range within each of the project dimensions of size and rigor. These ranges are identified through actual practice using the methodology on actual projects. Ranges of applicability can then be adjusted to reflect a successful use in a project that stretched the methodology's range. When no existing method applies, new constructs are added to form a new one with its own color and range.
You can use Crystal in your own work to achieve some very useful effects:
- Using the Crystal methods out of the box as methodologies.
- Using the Crystal mechanism to characterize method configurations according to your own organization's priorities, experience, and skills.
- Maintaining your own classifications for existing methods to optimize development productivity over time by choosing the most efficient method for new projects.
- Using the Crystal method classification techniques to evaluate existing or proposed methodologies.
The Open Unified Process (UP) is an iterative methodology with minimal but extensible content. UP emphasizes architecture and modeling in the development process and is in the process of being adapted to model-driven development.
RUP is an incarnation of the Unified Process methodology that is packaged as a product. RUP contains extensive content, deliverable templates, process templates, roles, process and artifact guidance, and so on that you can tailor to almost any project context. RUP is delivered as browseable, reusable content. Organizations can build content that describes their specific process and deploy across any size team.
Team Software Process and Personal Software Process
The Software Engineering Institute (SEI) supports the Team Software Process (TSP) and the Personal Software Process (PSP) as part of an engineering approach to software development. The goals of TSP/PSP are to make software development more manageable, measurable, accountable, and predictable. SEI provides a large body of background material, case studies, books, training, coaching, and consulting in support of these processes.
How can I talk about milestones in the context of a process or methodology when milestones occur as part of the process? Basically, I'm talking about the higher-level process within which the application development process is nested. This is usually some product or portfolio management process by which projects are spawned. At some point in the feasibility and planning stages of this process, the rough architecture of the system should be developed and used to analyze the effort, risk, and resources necessary for design and implementation. It is at this point that specifying and tailoring the development process can help.
Analyze the components of your architecture to assign characteristics that drive the choice of resources and methodology. Apply your best resources to the difficult or poorly understood components. If you are going to apply new technology, allow for investigation and prototyping. Identify components that are instances of generic implementation patterns and can be managed more tightly.
Understanding the many methodologies available and their characteristics is critical to knowing how to best construct your designs. You should familiarize yourself with the major methodologies and understand the tradeoffs in each. Learn to tailor the method to your needs. Choose the correct method for the type of project and the development environment. Learning to apply the right method in the right way at the right points in the architecture will greatly improve your results.
Learn
-
The Agile Alliance maintains resources related to agile software development techniques.
-
Scott Ambler is the author of several books describing the application of agile methods to various development domains, such as database modeling, documentation, object modeling, and project management.
- The Software Engineering Institute maintains a large body of information on the Team Software Process, the Personal Software Process, and software product lines.
-
Alistair Cockburn is the creator of the Crystal family of methodologies and maintains a wiki that serves as a reading list and map to other resources about the Crystal methods.
-
There are a number of resources available for the Unified Process, including variations for various development environments:
- The Agile Unified Process (AUP)
- The Enterprise Unified Process (EUP)
- Basic Unified Process (BUP)
- Open Unified Process
- Rational Unified Process
-
developerWorks Architecture zone:
Get the resources you need to advance your skills in the architecture arena.
-
Browse the technology
bookstore for books on these and other technical topics.
Get products and technologies
-
Eclipse Process Framework is a process metamodel, associated application program interfaces (APIs), and an Eclipse tool for defining and tailoring method content.
-
Download trial versions of Rational ClearCase and Rational ClearQuest now.
-
Download IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
Discuss
- Participate in the discussion forum.
-
Check out developerWorks
blogs and get involved in the developerWorks community.
Christopher Caserio is an IT Architect with a leading insurance company. He has been involved in the development and design of software systems for more than 25 years, playing many roles during that time. He started Objex Consulting in 1993 to provide enterprise architecture and software design and development services to clients. You can reach Christopher at cppc@acm.org.
