Systems engineers often focus on the system currently under development, without much concern for the larger enterprise it supports. This article explores the connection points between the enterprise architecture and the system architecture, and discusses how the enterprise architecture both provides input to and constrains system development. Its goal is to help systems engineers gain a deeper understanding of how their efforts on projects that create or modify systems are both constrained by, and can modify, the architecture of the enterprise those systems support. In today's business-driven enterprise, there is a direct relationship between the enterprise's business capability and the functionality implemented on projects.
The changing landscape of systems development
Today's enterprises are moving away from stovepipe systems that provide isolated functionality, and toward a much more integrated system of systems where services are leveraged to provide robust, efficient operations. As a result, systems within the enterprise are more tightly integrated, and efforts to modify them are more complex. The systems engineer working on a project can no long focus entirely within the system being modified, but must also be aware of how the system interacts with other systems in the enterprise.
Figure 1 provides an overview of this change of emphasis. Previously the business architecture was supported by independent stovepipe systems. There was a discrete handoff from the enterprise architect to the systems engineers, with its associated problems. Today, systems development is much more business-driven. There is a stronger need for financial accountability of IT and system expenses, and expenses must be tied back to their business benefit. Thus, the alignment between business and development is critical. There is more of a continuum between the involvement of the enterprise architect and the systems engineer, leading to greater business/IT alignment and contributing to technical governance across the lifecycle -- enterprise architects stay around longer and systems engineers get involved earlier. Finally, the implemented services support data collection and monitoring during operation. Analysis of this feedback drives future changes.
Figure 1: Aligning architecture and engineering early in the lifecycle
When discussing systems at different levels it is important to explicitly define terms that would otherwise be ambiguous given their diverse meanings and usages.
One definition of an enterprise is a business organizationi. The organization could be a part of one company, an entire company, or even a collaboration among multiple companies. In order to simplify the discussion, let's think of an enterprise as one company. While there are a large number of ways to look at an enterprise, in order to reason about the evolution of the enterprise it is useful to consider it as a large-scale system. Like a system: a) It has a reason for being, in that it provides certain value to its stakeholders; b) It has funding that enables its operation; c) It performs some actions, satisfying a set of requirements; and d) It is made up of components -- workers and lower-level systems, which collaborate to accomplish its functionality. (Throughout this article the term "component" is used to refer to a part of the whole that collaborates with other parts.)
There are many definitions of enterprise architecture. You will often see enterprise architecture capitalized as a proper noun (e.g., the discipline of Enterprise Architecture), though as you can see, we're not using it that way here. We're referring to enterprise architecture simply as the description of the architecture of the enterprise of interest. The discipline of Enterprise Architecture ties together business, strategy, process, method, and components from a number of different perspectives. These perspectives are defined by, and vary by, the different approaches to Enterprise Architecture. Enterprise Architectures are produced by Enterprise Architects. The responsibilities of an Enterprise Architect go far beyond what is discussed here.
So the purpose of an enterprise architecture, as defined in this article, is to describe the components of the enterprise, their relationships, how they collaborate, and their interactions with the "outside world." An enterprise architecture provides the direction for the implementation of the enterprise components. Implementation of the components results in a change of state of the enterprise.
A system is a group of items forming a unified whole, serving a common purposeii. The common purpose is the system's reason for being. One or more stakeholders recognize a need that the system fulfills. Thus the purpose of the system is to satisfy a set of stakeholder needs -- the system requirements. These requirements include what functionality is exhibited, and also how the functionality is exhibited given required qualities and constraints (i.e., non-functional requirements). The system satisfies its requirements by performing some set of actions. The actions result in satisfying the stakeholder needs. As the system is a group of items, the system's actions are actually accomplished by the collaboration of these components.
It is important to note that the components can be anything: hardware, software, or people. People that participate in a system as collaborating components are called workers. Some of the components can themselves be considered systems in their own right, and are often called subsystems. While, in fact, workers can also be considered systems as described above, we do not treat them as such, but rather as atomic objects, as we are not concerned here with how their parts collaborate internally.
The title Systems Engineer has been applied to individuals doing just about any activity associated with engineering a system. We've seen "Systems Engineers" responsible for everything from planning, to requirements elicitation, to architecture capture, to integration and testing. Many of their activities go beyond the traditional Systems Engineering discipline. In this article, we focus on the role of the systems engineer to ensure that the result of the development effort will "fit in" with the rest of the enterprise and operate in a consistent fashion. Essentially, it is their responsibility to create or update the architecture of the system while conforming to any constraints mandated by the larger enterprise.
A program is an initiative undertaken to change the state of the enterprise, to provide some new or improved capability. Its purpose is to move the enterprise from an as-is state to the to-be state by modifying some portion of the enterprise, adding and/or modifying enterprise components. Programs are executed by implementing one or more (usually more) projects. Note that programs can have a scope that is much smaller than the entire enterprise. However in order to simplify the discussion here, we will only discuss programs that have an impact on the overall enterprise.
A project is a development activity with a specific focus, start, and end, which is intended to deliver some measurable result of value that contributes to a capability. It is common for a project to be focused on the introduction of a new system into the enterprise, or the modification of an existing system, though its scope could be larger or smaller. Projects have their own goals and budgets. They are normally combined with other projects as part of a program (see Figure 2).
Figure 2: How programs and projects affect the enterprise
As-is enterprise-level description
Whether documented or not, every enterprise has an architecture, which consists of components and their relationships and collaborations, often captured in pictures, diagrams, documents, models, etc. In addition to the architecture, the enterprise also has a set of requirements it needs to satisfy. There are also tests to determine how well the enterprise satisfies its requirements.
Again, whether documented or not, every enterprise has its requirements and tests. When a new release of an enterprise component is being deployed, some amount of testing will be performed to ensure that the component satisfies its requirements, including that it does not break any higher-level functionality due to the way it interacts with other components. If these tests discover any problems, they must be tracked as enterprise defects until resolved. (The problem could be the newly released component, or unexpected behavior from an interfacing component.)
Thus, we see that these artifacts, when they exist and are combined, form a full description of the key elements of the as-is enterprise (see Figure 3):
- Requirements (and their drivers such as motivation and goals)
- Architecture (including design and implementation)
Figure 3: Enterprise as-is artifacts
Programs change the enterprise
As defined above, the purpose of a program is to move the enterprise from the as-is state to a to-be state. Many times, this includes creating a set of artifacts that describe the to-be state. However, if the current state is well-documented, it is not necessary to re-document the portions of things (requirements, architecture, and tests) that are not changed by the program. It is only necessary to update the existing as-is artifacts with the changes prescribed by the program. These changes are deltas that need to be applied to the as-is artifacts in order to describe the desired to-be state.
Rather than starting from scratch, program artifacts should describe the changes from the as-is state. This assumes that the as-is state is well-understood (captured). If that is not the case, all is not lost. Since it is necessary to document the to-be state anyway, the artifacts created can become enterprise-level as-is artifacts after the program is completed.
Programs can range in scope from modifying one aspect of the enterprise to transforming the entire business. Thus, it can easily be out of the scope of a single program to generate the entire set of as-is artifacts for the enterprise. Rather, each program can generate the artifacts for the portions it changes. As more programs touch more areas of the enterprise, the gaps are filled in. This approach avoids the problem of needing to wait for a complete set of as-is artifacts before any change programs can be initiated. This approach can proceed until the remaining gaps in the enterprise as-is artifacts are relatively small and can be addressed by a separate effort to directly close the gaps. In order to achieve a complete and consistent representation of the enterprise, all enterprise programs must use standard conventions to represent both the as-is and to-be artifacts (or at the very least convert their artifacts from/to the standard convention), and in fact must begin to build on the artifacts produced by the previous programs. Otherwise it would be extremely difficult to correlate between the artifacts created by different programs and achieve a single consistent representation of the enterprise. Further, the as-is artifacts must be maintained as a single consistent repository. How the repository is constructed, whether it is a single file or a federation of databases, is not important. The important point is that it is maintained and can be accessed as a consolidated consistent representation.
If (or once) the enterprise as-is artifacts are available, the program should start with these artifacts and capture the changes that are needed to implement the program. This includes deltas to the requirements, updates to the architecture, and modifications to tests commensurate with the requirement changes. The requirement deltas capture the desired changes to expected behavior. These desired changes, even if they are initially communicated informally and refined later, drive the architecture deltas. Both the requirements deltas and the architecture deltas can drive changes into the set of tests.
As the program is executed (that is, its projects are implemented), the tests can be performed to verify that the requirements are satisfied, and to detect any defects in the implementation, the requirements, or the tests themselves. Normally any defects discovered would be resolved within the scope of the program. However, some defects may not be resolved within the scope of the program, and thus become additional, enterprise-level defects.
At the conclusion of the program, the enterprise is in the to-be state defined by the program. As this is the new as-is state for the enterprise, the enterprise level as-is artifacts need to be updated. This is straightforward as the program has already produced all the required changes to the artifacts. See Figure 4 for an illustration of the flow of artifacts. (Note that since multiple programs can be executing concurrently, at this time the enterprise may actually have additional changes implemented beyond the scope of this program. Those additional changes will be merged into the enterprise as-is state by the conclusions of those programs.)
Figure 4: Flow of artifacts between program and enterprise-level
Projects implement the program
Programs define a set of changes that create or modify some end-to-end capability. In order to achieve the new capability, it is usually necessary to create new systems and/or modify multiple existing systems (possibly by acquiring a new application or by changing a process). It is common to define and execute multiple projects, one for each impacted system, in order to achieve the total program objectives.
Each project has a specific scope it is intended to accomplish. That scope is directly related to the changes to the architecture required to implement the new capability. That is, the program defines what new functionality is required of the impacted systems in order to implement the capability, and each project implements the new functionality for its system(s). While it is possible for a project to implement changes that support more than one program, this is significantly less common and thus will not be addressed here.
It is most common for program-level requirements to be implemented through a collaboration of multiple systems. In these cases, a program-level design is created to show how the systems collaborate. This design assigns responsibilities to each of the involved systems. The responsibilities correspond to the roles they play in the collaborations. This design satisfies the program-level requirements deltas. The result is a set of derived requirements on each of the systems. However, there are times when a program-level requirement is implemented entirely in a single system. In these cases, the program-level requirement is allocated directly to a single system. See Figure 5 for an illustration of these relationships.
Figure 5: Program to project requirements flow
But here again, in executing the project we do not need to start from scratch unless the project is creating a new system. If the scope of the project is to modify an existing system, then the system has an as-is state. It has requirements it satisfies, an architecture, tests that have been performed, and probably some open defects. As described above for the as-is enterprise artifacts, if these artifacts are not available, they can be built up over a number of projects. Just as is true for the enterprise artifacts, the systems artifacts need to be maintained in a repository and in a consistent format in order to leverage them effectively.
The as-is system artifacts, like the as-is enterprise artifacts, include requirements, architecture, tests, and existing defects. So, if a system has existing as-is artifacts, then rather than starting with a blank slate, the project should create its to-be artifacts as changes to the existing as-is artifacts. Just as the program provides updates to the enterprise artifacts, the project provides updates to the system artifacts. See Figure 6 for the relationships between system as-is artifacts and project artifacts.
Figure 6: Flow of artifacts between project and system-level
Putting it all together
The above provides an end-to-end flow for the evolution of both the enterprise and systems, including their as-is artifacts, through the execution of programs and projects. Admittedly, this is a simplified view, assuming just one step between the enterprise and its systems. Clearly there is the possibility for additional steps, with intervening levels and their artifacts. However, the same approach applies. The approach can be consistently applied to each intervening level of decomposition, with appropriate decisions made as to which mechanism (program, sub-program, project, sub-project, etc.) updates these intervening levels. Figure 7 provides the complete, end-to-end flow for this simplified view.
Figure 7: End-to-end flow from the enterprise to systems
Some organizations have mature practices managing their as-is artifacts for the enterprise and system levels, and are reaping the benefits of leveraging these artifacts. Others are just getting started, and have visions of future benefits. However, the goal and approach are the same in all cases. In order to effectively manage the evolution of the enterprise, it is necessary to have an understanding of its current requirements and functionality, and how it achieves that functionality (its architecture). Furthermore, it is necessary to understand how well it performs currently, in terms of tests and existing defects.
It is not efficient to recreate these artifacts on each program. Rather, organizations desire to reuse existing knowledge and evolve the artifacts as the enterprise evolves. Thus, the organization always has an accurate view of the current state, is able to effectively plan the evolution of the enterprise, and reduces the total effort expended by leveraging the artifacts on each program. Even if it is impractical to capture a complete set of as-is artifacts for the entire enterprise, organizations still derive benefits from capturing the artifacts for a portion of the enterprise -- the larger the portion, the greater the benefit.
Without an accurate view of the current state, each program must a) create a representation of the current state from scratch by examining the enterprise before progressing with the work of the program; b) attempt to construct a representation of the current state by successively applying modifications implemented by prior programs, or c) forego trying to represent the current state and attempt to modify an unknown architecture, hoping that the modifications will not have unintended consequences. We have seen organizations attempt all three of these options, only to painfully learn that maintaining an accurate view of the current enterprise state is necessary for their efficient operation.
It is equally true that organizations greatly benefit from the reuse of system-level artifacts. An enterprise has many systems and thus the benefits of reuse are multiplied by the number of systems. The complexity of most systems has grown far beyond the ability of any single individual to maintain completely in his or her mind. Artifacts capturing the requirements, architecture, tests, and defects of a system are a necessity for communication. The benefits of leveraging these artifacts from one project to the next include: greater consistency, fewer errors, and overall reduced effort.
We have illustrated how the architecture of the enterprise provides the basis for its evolution by programs. Individual programs and the entire organization benefit from specifying the program in terms of changes from the as-is enterprise state. These changes drive and constrain the work done on projects. Projects evolve systems from their as-is state, but do so only to satisfy the allocated and derived requirements provided by the program. To complete the cycle, projects and programs must apply their changes to the as-is system and enterprise artifacts so that they are accurate for the next modification efforts.
IBM® offers a number of products and methods to support the Enterprise Architect and the Systems Engineer throughout the lifecycle. If you would like to know more about these specific products, this link is a good place to start. As for the methods, Figure 8 illustrates how some of the methods apply across the lifecycle.
|Model Driven Systems Development||Service-Oriented Modeling & Architecture *1||Component Business Modeling *1||IBM Global Services Method *1|
|*1 IBM Global Business Services Only|
Figure 8: Methods supporting Enterprise Architects and Systems Engineers throughout a lifecycle
The simplified view presented in this article provides a basis for a common understanding across the organization. By clearly illustrating the relationships we've described, systems engineers are better able to understand how their efforts are constrained by, and contribute to, the overall enterprise architecture. Furthermore, we have illustrated the direct relationship between the enterprise's business capability and the functionality implemented on projects.
There are a few tangential topics that bear mentioning, but whose full treatment is beyond the scope of this article:
- Service Oriented Architecture (SOA): By establishing the as-is enterprise artifacts, the architecture in particular, the use of common services throughout the enterprise becomes clearer. As a result, it is easier to analyze the architecture and define programs to implement SOA.
- Distributed Development: By articulating the artifact types and relationships on programs and projects, the assignment of responsibilities and the criteria they must satisfy becomes clearer. This enables effective distribution of responsibilities to geographically dispersed development centers, within or outside the company.
- Standard Environment: Though not required, a standard environment (common process and tooling for development, change management, configuration management, and metrics collection and reporting) can be utilized across the projects and programs, yielding additional benefits. In addition, the common environment would support the required common representation of artifacts across programs.
We would like to thank Ernest Vogel, Cindy VanEpps, and Arvind Sathi for their contributions to the construction of the original artifact flow. In addition, the following have improved the quality of this article with their comments: Pete Eeles, Jos Jennekens, Jim Densmore, and Fred Mervine.
- Webster's Ninth New Collegiate Dictionary, 1985 Merriam-Webster Inc., ISBN 0-87779-509-6.
- Learn about other applications in the IBM Rational Software Delivery Platform, including collaboration tools for parallel development and geographically dispersed teams, plus specialized software for architecture management, asset management, change and release management, integrated requirements management, process and portfolio management, and quality management.
- See the Rational Software Platform for Systems page.
- Learn about Enterprise Architecture.
- See the page on Enterprise Architecture: Accelerate change - create a shared view of your business and IT landscape.
- See the Rational Enterprise Architecture Management page.
- Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
- Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Additionally, some "Getting Started" courses are available free of charge.
- Subscribe to the Rational Edge newsletter for articles on the concepts behind effective software development.
- Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.
- Browse the technology bookstore for books on these and other technical topics.
Get products and technologies
- Download trial versions of IBM Rational software.
- Download these IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Tivoli®, and WebSphere®.
Dig deeper into Rational software on developerWorks
Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.
Keep up with the best and latest technical info to help you tackle your development challenges.
Software development in the cloud. Register today to create a project.
Evaluate IBM software and solutions, and transform challenges into opportunities.