Best practices for using composite baselines in UCM

Composite baselines, introduced in ClearCase v2003, can help software practitioners and configuration managers to better manage their software components and subsystems. This whitepaper discusses some best practices for using composite baselines to greatly simplify baseline management and enhance the reusability of components and subsystems.

Jim Tykal (jim.tykal@us.ibm.com ), SCM Specialist - Worldwide Technical Marketing, IBM, Software Group

Jim Tykal is an employee at IBM



02 July 2004

Overview

This whitepaper is intended for software practitioners and configuration managers who would like to utilize composite baselines to better manage their software components and subsystems. A basic understanding of the Unified Change Management process (UCM) is assumed.


Background information

Component

In software engineering, and specifically in the IBM® Rational Unified Process® (RUP®) methodology, a component is a set of related elements that are developed and released together. This might be a library, .dll, .jar, an executable, or any set of assets that are released as a unit.

Within IBM® Rational® ClearCase® UCM, a component is a set of elements that are related by being located in a specific directory tree. Since this concept is more restrictive than the general definition of a component, we'll refer to a UCM component object as a Configuration Management Component, (or CM component for short).

CM components have a limited scope and meaning; the intent is to identify a set of elements within ClearCase. A CM component is represented by a single directory tree of elements in a VOB. The root of the directory tree is called the Component Root Directory Element. The root directory may be the root of a VOB (in which case the component includes all of the elements in a VOB), or it may be a directory one level down from the VOB root (which allows several sub-vob components to be contained in a VOB).

A CM component
Figure 1: A CM component

The use of components reduces the complexity of configuration management, because source code can be managed in logical units consisting of many elements -- rather than on an element-by-element basis. Using components also promotes code sharing and reuse because they track the set of elements that depend upon each other.

Rootless component

In v2002, UCM introduced rootless components: components that do not have any associated elements. Since a rootless component is just an object in an UCM PVOB, there is no component root directory element.

A rootless CM component
Figure 2: A rootless CM component

At first glance, a component that does not collect source files may not appear to be useful. Later in this paper, however, we will illustrate that rootless components reduce configuration management complexity and achieve other benefits when used with composite baselines.

Baseline

Just as a component represents a collection of elements, a baseline represents a collection of versions within a component. A baseline identifies at most one version of each element in a single component.

A baseline is a snapshot of a component at a particular moment in time. It comprises the set of versions that are selected in the stream at that time. When a new stream is configured, baselines are used to specify which versions will be selected in that stream. Baselines are immutable, so that:

  • A particular configuration can be reproduced as needed
  • Streams using the same set of baselines are guaranteed to have the same configuration

Therefore, the set of versions included in a baseline cannot be modified.

A UCM baseline
Figure 3: A UCM baseline

Baselines created in the same stream are in an ordered relationship to each other. Within a single stream, for example, an old baseline is referred to as an ancestor of a newer baseline, and the new baseline is called a descendant of the old baseline. The closest ancestor of a baseline is its predecessor. The foundation baseline of this stream, created in a different stream, is the predecessor of the first baseline created in this stream.

An example of baseline predecessors and descendents
Figure 4: An example of baseline predecessors and descendents

In Figure 4, baseline BL1 is the predecessor of BL2 (and BL2 is a descendant of BL1). When BL2 was created, there were new versions of str.h, txt.h, and file.h, but for the files imp.h and num.h, BL2 shares the BL1 versions. Similarly, BL3 is a descendant of BL2; the other two baselines, BL1 and BL2, are predecessors of BL3. BL3 captures changes made after BL2 was created, but it still uses the BL1 version of num.h and the BL2 version of txt.h. Version 4 of txt.h is checked out, so it isn't included in BL3. Another way of looking at the relationship between baselines is that a descendant contains its predecessors-all of the changes captured in BL2 are also included in BL3.

Baselines have three purposes in UCM ClearCase:

  • Record work done and store milestones. Each new baseline incorporates the changes made since the previous one.
  • Define stream configurations.
  • Provide access to delivered work.

The relationship between a baseline and a component is very similar to the relationship between a version and an element. For instance, baselines exist in streams while versions exist on branches, and both baselines and versions have predecessors.

Stream

Baselines and streams have a mutual relationship: baselines are produced by streams, and streams use baselines for their configuration. A stream is configured with a set of baselines, called its foundation, which defines which versions are selected in that stream. Views attached to the stream see the versions of elements that are selected by the foundation baselines, plus any changes that have been made in the stream.

A UCM stream
Figure 5: A UCM stream

Note that a stream's foundation needs to include a baseline from every component (both modifiable and non-modifiable) that it needs to access.

Project

In the context of UCM, a project contains the configuration information needed to manage a significant development effort. A project consists of a set of development policies and a set of streams, which together provide the context for development.

Every UCM project has a single integration stream, which configures views to select the latest versions of the project's shared elements. A traditional project also includes multiple development streams, which allow developers a private workspace to develop and test their changes -- without worrying about disruptive changes from the rest of the project team. An integration stream is connected to the project, but development streams are spawned from another stream, in a parent-child relationship. Therefore, integration streams may have child streams but no parents, and development streams have parent streams and may or may not have child streams.

Each development team organizes their projects to meet their own requirements, but generally the project organization falls into one of two categorizations: release-oriented and component-oriented.

Release-oriented projects

Many project teams think of their work in terms of product releases. First, the team works on Release 1 of the product. When the team is ready to begin work on Release 2, they branch off from Release 1 and begin new development work in the Release 2 stream. Likewise, when Release 3 work is ready to begin, it will branch off Release 2, and so on.

A cascading release-oriented project
Figure 6: A cascading release-oriented project

Figure 6: A cascading release-oriented project

After Release 1 ships, patches for the release might be developed in their own project, which branches off of Release 1 but delivers its work to Release 2, so that the bug-fixes can be incorporated into the new release.

The cascade of branches (as shown in figure 6) that results from this project organization can cause some difficulty, so a slightly different project layout is generally recommended for release-oriented projects. In this layout, each project delivers its release back to a "main" project. The projects can then be branched off of the Main_Proj integration branch (as shown in figure 7) instead of cascading from the previous release.

An improved release-oriented project
Figure 7: An improved release-oriented project

A release-oriented project must have modifiable access to all of the components that compose the final product. Developers working on one component of the project need to coordinate their work consistently and frequently with developers working on other components.

Component-oriented projects

Other development teams organize their work by reusable assets. These teams use projects to create components (which may be composed of several CM components). Low-level or core components are used to construct mid-level components or subsystems, and so on, until the highest level components are integrated into a product.

In configuration management, the key attribute is that all the code of a subsystem is released together. To the development team, the important factor is that the subsystem represents assets that can be easily reused. Designing a product or a family of products based on subsystems makes it easier to divide the development effort into relatively independent sub-efforts. This division can help simplify planning and risk management.

The key aspect of component orientation is that each project has access to two classes of components: custom components and shared components. The project does its development work on the custom components: it is the only project to modify these components. The shared components, on the other hand, are used, but the project doesn't plan to modify them because that would break the sharing model. Accordingly, these components are generally specified to be "non-modifiable" in the project.

The goal of a project is to produce a set of baselines that represents the integration of the shared components into a subsystem.

A component-oriented project
Figure 8: A component-oriented project

In this example in Figure 8, project Alpha uses CM components A and C to produce a set of baselines that define the Alpha subsystem, while project Gamma uses CM components B, C, and D to implement the Gamma subsystem. Project MLib uses the subsystems created by the Alpha and Gamma projects, represented by the baselines these projects produce, and additionally does custom work in CM component E.

The sub-teams in a development group with this type of project organization are restricted as to which components they may modify. Because the lower-level components might be shared, the changes have to be made in a central, compatible manner, in the project dedicated to that component. If the MLib project needs new functionality in the Alpha component, that work has to be done within the Alpha project.

One aspect of a component orientation is that each project has more freedom in the selection of baselines of shared components. Because these components are not modified by the project, the project should theoretically be able to change from one version to any other version of a shared component. The component-oriented project works well when components are loosely coupled with well-defined interfaces. That is, if a particular development task falls within one subsystem, this approach works well. If a development effort crosses many subsystems, then it will require coordination among the various projects in which the work will be done.

Mixing project strategies

The strategy for project organization is usually dictated by the organization and philosophy of the development team. UCM supports both release-oriented and component-oriented projects; it is also possible to mix the two strategies. Some projects can be dedicated to producing certain components, and other projects can be dedicated to integrating components into a release. Of the two, UCM's support of component-oriented projects is more complex, because additional constraints are placed on the use of composite baselines and more burden rests on the project leaders. This will be discussed in detail later.

Rebasing

Changing the foundation baselines of a stream is done through the rebase operation. As mentioned previously, baselines provide access to delivered and tested work. In order to use code that has been included in a baseline, a stream rebases to pick up the desired baseline.

In a rebase, the user selects one or more baselines to add or drop from the configuration of the stream. Just as a view can select at most one version of an element, a stream can include only one baseline for each component. This is because if more than one baseline were allowed in a stream's foundation for a particular component, the selection of versions in that component would be ambiguous.

During the rebase, the specified baselines replace the current baselines, if any, for their components. Changes that have been made on the stream are merged into new versions, if necessary. Note that while deliver always involves merging elements, rebase only involves a merge if elements that have been modified in the stream also have new versions selected by the new baseline.

The relationship between the old baseline and the new baseline for a component defines the direction of the rebase. If the new baseline is descended from the old baseline, the rebase advances -- the stream is picking up work done in other streams from the same starting point. Similarly if the new baseline is an ancestor of the old baseline, it is said to revert -- the stream is moving back to an earlier baseline. If two baselines either share an ancestor (but both contain significant development work), or if the relationship between them cannot be established, the direction is sideways.

A single rebase operation might involve many baselines; the rebase direction is determined on a baseline-by-baseline basis. Thus in one rebase operation, a stream might advance for one component, revert for another, and rebase sideways for a third.

Advance

Advancing is the main use case for a rebase. Most development streams over the life of a project advance from early baselines to later baselines that integrate work performed in the project. Streams are usually allowed to advance.

Restrictions on advancing rebases occur when the descendent baseline isn't from the parent stream. It is possible at rebase time to specify a baseline from any stream. A stream may select a baseline that is a descendent of its foundation and happens to be in a stream that is not its parent stream. For example a development stream could rebase to a baseline created in a sibling development stream -- in this case, the rebasing stream could pick up work -- which has not been delivered to the parent stream. If several streams attempt to deliver the same work, there will be confusion at merge time (alternate target delivery is a special case, and policies can be set to allow a stream to deliver work done on another project).

A common use case wherein an advancing rebase might occur is when using a build stream, as illustrated in Figure 9. After a baseline for a milestone has been created, the work to stabilize the build is done on a stream dedicated to this task. When the build is ready to be released, the development streams can pick up a baseline on the build stream, rather than from the integration stream, because the integration stream might have received more activities delivered by other developers in the meantime.

Example of a build Stream
Figure 9: Example of a build Stream

However, the development streams are not allowed to rebase to the trans2_deploy baseline, even though it is a descendent of their current foundation baselines, trans1. If this rebase were allowed, then Dev_stream1 or Dev_stream2 might deliver the build stabilization work before Build_stream does. Therefore the rebase is not allowed until after Build_stream delivers its work in trans2_deploy (represented by the curved arrow) to the parent of the development streams, the Rel_1.0 integration stream.

After the work in Build_stream is contained in the parent stream, Dev_stream1 and Dev_stream2 can then rebase to trans2_deploy.

Note that the deliver operation changes the relationship between baselines in a stream. In this example, when a new baseline trans3 is created in the Rel_1.0 integration stream, it will be a descendant of trans2 as with any baseline, but it will also be a descendant of trans2_deploy from Build_stream. Because trans2_deploy was delivered to the integration stream, trans3 will contain trans2_deploy, which is a requirement of the predecessor/descendent relationship.

Revert

Reverting occurs when a stream needs to remove some bad changes. If a baseline is discovered to have some serious problems, a stream can revert to an earlier baseline. Streams are only allowed to revert, however, if there are no changes in the component involved; if changes had occurred, ClearCase would have to "unmerge" those changes.

If a stream must revert even though it has made changes, the developer will have to explicitly remove the new versions before rebasing.

Sideways

Sideways rebase occurs when the baselines have no ancestor or descendent relationship to each other. This occurs when rebasing from one import baseline (or a descendant of it) to a different import baseline (or descendant). Rebasing to an imported baseline is almost always a sideways rebase. The exception to this occurs when the imported baseline is an ancestor of the current baseline, changing the direction to revert. Imported baselines have no predecessors, so they are only related to their descendent baselines. The other kind of sideways rebase is when the new and old baselines are cousins -- that is, share a common ancestor -- but neither is an ancestor of the other.

One use case for sideways rebasing is a project that uses third-party software, such as a set of compilers or other tools. The project does no development on these tools, but from time to time it receives a new release. The new release is put into a VOB, certified and, if it passes, it is labeled. The label is then imported into a baseline; rebasing to this imported baseline is a sideways rebase.

Example of a sideways rebase-rebasing to an imported baseline
Figure 10: Example of a sideways rebase-rebasing to an imported baseline

Without a relationship between the baselines, ClearCase has no guidance on how to merge the changes. This means that ClearCase has to impose one of the following restrictions on sideways rebases. Either:

  • The component is non-modifiable and there are no baselines (-ident baselines) on it

Or

  • The new baseline must be contained in the parent stream (as with advancing).

Managing projects with many components

In any project, the project leader has to maintain the set of baselines that are recommended by the project's integration stream. This means creating baselines regularly, testing the baselined code (perhaps in a build stream), and making the baseline recommendations. The project leader should be responsible for ensuring that each recommended set of baselines from the various components are compatible with one another. Furthermore, he needs to maintain a history of the recommended baseline sets, in case the project encounters a situation where it had to return to an earlier set of baselines. To help track which baselines have reached certain levels of certification, UCM provides promotion levels. A baseline can be marked BUILT, TESTED, REJECTED, RELEASED, and so on; the list of promotion levels is customizable. Some projects use baseline-naming schemes to help keep track of which baselines work together (baseline naming was made much more flexible in v2003).

As the number of components in a project increases, the cost of managing baselines grows significantly. Additionally, larger development efforts often divide their work into components -- large components consisting of more than one CM component. The status of a large component, or subsystem, is expressed as a set of baselines on CM components, increasing the number of consistent baseline sets the project leader has to keep track of. Furthermore, if subsystems are built from other subsystems the problem becomes progressively more difficult to manage.

The challenge of managing sets of baselines is met through the use of composite baselines.

What is a composite baseline?

Introduced in ClearCase v2002, composite baselines are a mechanism for grouping baselines into a collection. One baseline is designated as the composite, and other baselines become members of the composite.

In Figure 11, the baselines A1 and C3 are members of the composite baseline Alpha1.

A composite baseline
Figure 11: A composite baseline

The member relationship implies that the versions recorded in A1 must be released with the versions in Alpha1, together with the versions represented by baseline C3. Membership implies a code dependency, so a member baseline is said to be a dependent on the composite baseline. Member baselines may themselves be composite baselines, so the chain of dependencies forms an acyclic directed graph (with no limit on the depth of the member relationship), as represented in Figure 12.

A composite baseline that includes composite baselines
Figure 12: A composite baseline that includes composite baselines

Why use a composite baseline?

Using composite baselines greatly simplifies baseline management, because only one baseline needs to be specified for an entire set of baselines. The composite baseline represents the closure of all member baselines. For example, an integration stream can build a composite baseline that has as its members the current baselines of all the components in that stream. The administrator only needs to recommend that single composite baseline, not all of them. In addition, the development streams off that stream can rebase to that single baseline to get them all. Thus, including the Alpha1 baseline (shown in Figure 11) in a stream's foundation set will mean that the stream will select versions from component A based on the member baseline A1, and versions from component C based on member baseline C3.

In this way, composite baselines can be used to model components or subsystems that are made up of more than one CM component. Composite baselines can also represent a subsystem, and allow projects to combine lower-level subsystems into higher-level subsystems. At the lowest level, CM components are included into a composite baseline, then higher-level components are represented by composite baselines composed of a combination of composite baselines and baselines on CM components, as indicated in Figure 13.

Composite baselines can represent component hierarchy
Figure 13: Composite baselines can represent component hierarchy

In Figure 13, the Mathlib1 composite baseline has been re-drawn to highlight the fact that the Mathlib subsystem is composed of the Alpha and the Gamma subsystems, and a CM component, F.

Notice that Figure 3 represents the dependency tree for the Mathlib1 baseline: all of the objects pictured are baselines, so the diagram must be understood in the context of a stream. This example ilustrates a component-oriented project. The Mathlib1 baseline is the product of the integration stream of the Mathlib project. Other projects in this example produce the other components -- the Alpha and the Gamma baselines.

Creating composite baselines

Creating the composite -- editing baseline dependencies

You can create Composite baselines using either the command line or the GUI interface. However, they must be introduced in an integration stream. Baseline dependency relationships are based on the stream context -- they are not propagated in a deliver operation. In deliver, changes to individual components are merged into the target stream, but nothing about baseline relationships is communicated. The rebase operation is the way that a stream receives information about baselines, and it is naturally the only way for a stream to pick up a composite baseline.

To create the first composite baseline, the component that will hold the composite needs to be included in the integration stream's current configuration. If necessary, rebase the integration stream to the appropriate baseline of the component first, then add the baseline dependencies.

From the command line, baseline dependencies are created by making a new baseline on a component and specifying the components whose baselines are going to become members of the composite. As with any other baseline, this must be done in the context of a view connected to the integration stream creating the baseline.

For example:

cleartool mkbl -component Alpha -adepends_on A,C ALPHA_BL

This command creates a composite baseline on component Alpha. Baselines of components A and C will be members of the new composite baseline; the current baselines on A and C are used, unless there have been changes to these components.

Refer to the ClearCase mkbl manual page for more specific details and examples.

Using the GUI to create a composite baseline is more straightforward since it utilizes a drag and drop interface to build the composite baseline dependencies. Select Edit Baseline Dependencies from the integration stream's context menu. In the Edit Baseline Dependencies dialog box, drag the components whose baselines will be members under the component that will have the composite baseline.

Creating a composite baseline in the ClearCase GUI
Figure 14: Creating a composite baseline in the ClearCase GUI
(click here to enlarge)

In this example, baselines on components B, C, and E have been made into members of a composite baseline on component Gamma. Note that B, C, and E are still visible at the top level of the Edit Baseline Dependencies dialog box because these components are still in the foundation of integration stream. For more details on how to handle these baselines, see the section titled Bootstrap Projects.

At any time, a new composite baseline can have member baselines added or dropped.

Note that in both the command line and the GUI, it appears that components are manipulated to create composite baselines. The relationship, however, is actually made between baselines and not components. It is tempting to say that component B is dependent on component Gamma, but the dependency relationships have a limited scope. The dependency may last for the life of a particular project, but it might be different for other projects, so the dependency graph will change over time as components are added or dropped. For that reason, the relationship has to be made between versions of components -- baselines -- rather than the components themselves.

Furthermore, it also appears that the command line interface and GUI handle editing dependencies differently. However, the operations are the same. Options to the mkbl command can change the baseline relationships and create baselines. The GUI has two menu options -- Make Baseline and Edit Baseline Dependencies. The latter also allows you to change baseline relationships and create baselines. The former menu option behaves the same as the command line mkbl (without the add/drop baseline dependency options).

Creating a Composite Baseline Descendent

When a successor baseline to a composite is created, the new baseline inherits the members of its predecessors -- unless components are explicitly added or dropped. When a new descendent baseline is created, ClearCase checks the dependency graph of the composite baselines and creates new baselines as needed. There are three reasons to create a new baseline:

  • Changes were made in the component since the last baseline was created
  • Changes to the dependency graph (a member added or removed)
  • Changes in a member component

A change in one of the leaf nodes of the dependency graph will be propagated up to the root as new baselines are created, to include the new member baselines.

For example Figure 15 illustrates a situation in which code changes are made in component C, and then a new baseline is created on component Alpha. ClearCase examines the baseline dependency graph and finds the work done on component C, so it creates a new baseline (C4) to capture the changes. Next, because baseline C3 was replaced by C4, and C3 was a member of B1, ClearCase makes a new baseline (B2) to record the relationship with C4. Finally, because B1 was replaced, a new baseline on Alpha is needed.

Example of how a composite baseline descendent is created
Figure 15: Example of how a composite baseline descendent is created

The baselines created in the hierarchy are required because the composite baseline descendent represents the stream's configuration.

Using rootless components in composite baselines

As described previously, a new member baseline might mean one of three things when a descendant of a composite baseline is made:

  • Changes in the component
  • Changes in the baseline dependency graph
  • Changes in a member component

Because of these different possible meanings, a composite baseline fulfills multiple roles: identifying a set of versions in its component, collecting the set of member baselines, and changes in member components. It can be expensive (in terms of performance) to determine which reason caused the creation of the baseline.

However, a rootless component does not have an associated root directory and thus contains no elements. Therefore, a rootless component need not be concerned with the role of identifying a set of versions: a new composite baseline created on a rootless component can only indicate changes in the baseline membership. A composite baseline on a rootless component, sometimes called a pure composite, is simply an aggregation of baselines.

Consider two different composite baselines that link component A with component C (Figure 16).

Different ways of coupling baselines in a composite baseline
Figure 16: Different ways of coupling baselines in a composite baseline

In the directly coupled case, A1 is a baseline on component A and also a composite baseline that defines the dependency relationship between the components. We would need a new baseline of component A if an element of component A changed, or if a new baseline was created for component C. Note that this arrangement expresses the direction of the dependency between the two components: it says that something in A references, or otherwise depends on, something in C and not vice-versa.

On the other hand, when baselines on A and C are coupled through a pure composite baseline (on the rootless component Alpha), no information is implied about the direction of dependency between the two components. The dependency only expresses that the two baselines have been used together at the same time in the stream. However, there is no indication whether A depends on C or vice-versa, or both.

With a pure composite, a new baseline on Alpha has one meaning only: that there was some change in the membership of the composite baseline. We will see why this is significant in the section titled Best practices for composite baselines.

Conflicts in composite baselines

Composite baselines promote component reuse by making it easier to include large components and subsystems in a project. As the number of shared components rises, though, the potential increases that different subsystems will have a baseline conflict.

If a composite baseline contains only baselines of independent components explicitly named as members, there can't be any baseline conflicts. However, higher-level projects that use subsystems are likely to include in their foundation sets composite baselines that have members in the same component. Conflicts arise when the members are not the same baseline for a particular component.

For example, suppose Alpha1 and Gamma2 are both composite baselines that include a baseline on component C. Figure 17 illustrates this example with baseline C3, a member of Alpha1 and C1, a member of Gamma2:

Example of conflicting composite baselines Alpha1 and Gamma2
Figure 17: Example of conflicting composite baselines Alpha1 and Gamma2

When Alpha1 and Gamma2 are put together in the Mathlib project, as illustrated in Figure 18, ClearCase cannot tell which baseline on C to use. It is a basic axiom of ClearCase that a view must have an unambiguous rule for selecting versions of an element. In UCM, the unambiguous rule means that a stream can only use one baseline to select the versions in a component. ClearCase will block rebase operations (and baseline recommendations) that would result in conflicts.

Baseline conflicts when using composites
Figure 18: Baseline conflicts when using composites

ClearCase forces the user to specify explicitly a baseline for the component in question -- in this case, component C -- in order to resolve the conflict. This chosen baseline is said to override the members of the composite baselines in conflict. A baseline explicitly included in the foundation set of a stream -- whether it resolves a conflict or not -- will override any baseline of that component that is implied by a composite baseline.

Note that it is easier to avoid conflicts in release-oriented projects than in component-oriented ones. In the former, the baselines of all the components are typically selected (by a task stream or development stream) from the same stream (the integration stream,for instance). The highest-level composite baseline on that stream will necessarily be self-consistent, by the very definition of composite baselines. In other words, all the member baselines of the composite built in one stream must be consistent -- only one version of any element in any component will be selected at any time.

In Figure 19:, the project leader chooses baseline Cx to resolve the conflict on component C. As a result, the baseline C3 in Alpha1 and the baseline C1 in Gamma2 are ignored, and Cx is used to select versions in component C.

Using an override to resolve a baseline conflict
Figure 19: Using an override to resolve a baseline conflict

Notice that the project leader is not required to choose one of the conflicting baselines. Another baseline of the conflicting component can be chosen as the override. For new streams, it can be any other baseline. For rebase, it must satisfy the rules governing the direction of the rebase. The project leader must select a baseline that is compatible with the other baselines in the project's baseline set. In the example in Figure 19, the project leader could have chosen C1 as the override instead of Cx, but he would have to be sure that the configuration selected by Alpha1 was compatible with C1. With the selection of the override Cx, the Alpha and Gamma subsystems will have to be checked to ensure that they are compatible with Cx.

Note: Overrides stay in effect until either the overriding baseline is explicitly removed from the foundation set (for example, with cleartool rebase -dbaseline), or the foundation set is replaced completely (as happens when rebasing to the parent stream's recommended set: cleartool rebase -recommended).

The selection of overrides is a decision that UCM highlights for the project leader, but the decision cannot be automated. The project team has to determine the correct override in each instance of a conflict.


Best practices for composite baselines

Bootstrap projects

As we saw in the example of editing baseline dependencies in the GUI, when a composite baseline is created it doesn't affect the foundation of the stream in which it is created. After we create the composite baseline on Gamma, the foundation of the stream still contains baselines on components B, C, E, and Gamma. In order for development streams to have an appropriate configuration, then, the integration stream needs to recommend only the composite baseline. Still, it can be confusing to have all of the member baselines in the foundation of the integration stream.

One way around the confusion is to set up a special project to "bootstrap" the composite baselines. The bootstrap project should include all of the components in its foundation. When we create the composite baseline in the bootstrap project, a development project can then start with this initial composite baseline as its foundation.

A bootstrap project creating composite baselines on gamma and mathlib for other projects to use
Figure 20: A bootstrap project creating composite baselines on gamma and mathlib for other projects to use.

In a bootstrap project, the integration streams and the development streams have similar foundation sets. The project leader doesn't have to keep track of the individual baselines, since the projects can start with a single, all-inclusive composite baseline. Follow-on projects won't be confused when trying to select baselines to include in their foundation.

Bootstrap projects are useful if you know the baseline relationships at the start of development, and do not expect them to change much after that.

Use rootless components for composite baselines

When you use a rootless component to create a pure composite baseline, a new composite baseline has only one meaning: there has been a change in the baseline dependency graph. This allows ClearCase to provide greater configuration flexibility with pure composites than with a composite baseline based on a regular component. For this reason, it is recommended that you use pure composite baselines whenever possible.

We can see an illustration of the significance of pure composite baselines in the Mathlib project. Consider a composite baseline composed entirely of baselines on regular components. Suppose that the project has some development in the custom component F, and the project lead wants to create a baseline to capture it.

The Mathlib project has changes on component F
Figure 21: The Mathlib project has changes on component F

When the project lead tries to create a new baseline, the operation inspects the baseline dependency graph on Mathlib and finds the changes in F. However, ClearCase also sees that the Alpha and Gamma composite baselines have changed because their members C3 and C1 are no longer visible, since they are overridden by Cx. Therefore new baselines will be created on components F, Alpha, Gamma and Mathlib.

Status of the Mathlib composite after a new baseline is created
Figure 22: Status of the Mathlib composite after a new baseline is created

Baselines are created on Alpha and Gamma even if they are flagged as non-modifiable in the project. No changes were made explicitly in these components. However, in the context of the Mathlib integration stream, Alpha and Gamma contain changes based on Cx. At first, this may not appear to be a real change, but it represents a change because the project lead chose to use Cx (override) in the configuration of the Mathlib subsystem. The Mathlib subsystem will be tested using Cx and cannot be released with C3 or C1, because it wasn't built with these baselines.

Notice that after Mathlib2 was created, M_Alpha2 and M_Gamma3 include the overriding baseline Cx. There isn't a conflict between M_Alpha2 and M_Gamma3, so when another stream rebases to Mathlib2, it will not need to resolve conflicts in component C. This, of course, has no effect on the stream in which Mathlib2 was built; the override of Cx is still needed there.

It is very important to understand that in this system, the baselines M_Alpha2 and M_Gamma3 are necessary. This is because the very meaning of Mathlib2 is that all the baselines implied by its members represented the state of the stream at the time Mathlib2 was made. If testing and validation occurred at that time, then this might be an important baseline-for instance, one that can be released. Also, since the validation occurred with Cx, not C1 or C3, the members of Mathlib2 must represent that configuration.

The new baselines of Alpha and Gamma have a very real effect on the Mathlib project: the project is now constrained to only advancing on these components -- they can no longer revert or rebase sideways.

This becomes a problem if, for example, the various projects in this development team (Alpha, Gamma, Mathlib) are delivering their work to a centralized project Main_Proj as shown below in Figure 23.

Mathlib and projects producing subsystems delivering to a main project
Figure 23: Mathlib and projects producing subsystems delivering to a main project

If the Mathlib project starts with a baseline in Main_Proj (such as Alpha1), then, after it creates the M_Alpha2 baseline it cannot pick up the most recent baseline from the Alpha project, (such as trans_2). Rebasing from Alpha1 to trans_2 is a sideways rebase because neither baseline is an ancestor of the other. A sideways rebase is forbidden by ClearCase because of the change on component Alpha, represented by the baseline the Mathlib project placed on Alpha.

As discussed previously, he restrictions on sideways rebase do not apply if Alpha, Gamma, and Mathlib are rootless components. Since the rootless components cannot contain changes, ClearCase recognizes that there cannot be a conflict on these components and so the rebase rules are relaxed.

The Mathlib subsystem where Alpha, Gamma and Mathlib are rootless components
Figure 24: The Mathlib subsystem where Alpha, Gamma and Mathlib are rootless omponents

The rules are still applied as the rebase operation walks the dependency graph, so each composite baseline should be created on a rootless component. That way, the rebase is only blocked when there are code changes in one of the leaf components -- when there are code changes that could be lost.

Note: Release v2002 and v2003 require a patch to work correctly in this situation.

Logically, the rootless components represent the subsystems. Alpha and Gamma are the components being consumed to produce the Mathlib subsystem. The fact that these components are rootless can be seen as reflecting the fact that subsystems are logical constructions.

Migrating to rootless components

If a project such as Mathlib has already created a composite baseline with regular components, it is not difficult to change the project to use pure composite baselines.

The project leader can create rootless components to stand in for every component that has a composite baseline. In the Mathlib example, these might be Alpha_nr, Gamma_nr, and Mathlib_nr. Then new composite baselines can be made based on the rootless component, and include the original component as a member, as shown in Figure 25.

Changing a composite baseline from a rooted component to a rootless component
Figure 25: Changing a composite baseline from a rooted component to a rootless component

Strategies for avoiding rootless components

Not all projects or project orientations need the flexibility provided by using rootless components. They are needed when baseline conflicts occur and the project architecture might require a project to rebase sideways, picking up baselines from a variety of source streams and projects.

A project that does not include overlapping composite baselines, for example, will never have conflicts. In development teams where projects are release-oriented, a project is likely to only consume baselines from the previous project, and so the rebase flexibility will not usually be needed.

In place of rootless components, the project leader could ensure that the subsystems stay in sync with respect to the shared components. When a new baseline is created on component C, all of the consuming projects would release a new baseline for their subsystem based on the new baseline. This prevents baseline conflicts, although it is likely to be feasible only for projects with a small number of shared components.

Use build streams to stabilize composite baselines

Because composite baselines aggregate baselines from several components, it can be more difficult to make small, isolated changes in an integration stream, such as those that are needed to address stabilizing a release as it approaches a milestone.

For any project, the task of testing a release in preparation to publishing its baselines can be difficult if the testing is based on the integration stream. After a specific activity has been delivered, the project leader may wish to build and test the product. For example, critical bugs have to be fixed before a baseline can be published, but in the meantime there may be other activities that have been delivered, which then need to be tested. The project leader could lock the integration stream to prevent these additional deliveries, but this stops all development work that isn't related to stabilizing the release.

To solve this problem, create a stream dedicated to the build stabilization task. The project leader lays down an initial baseline of the untested release, and uses that baseline as the foundation of a build stream. The project leader can then control what changes occur on the build stream. Fixes can either be made directly on the build stream, or delivered to it from a development stream. Fixes implemented in the build stream will be uncontaminated by ongoing deliveries to the integration stream.

Example of a build stream
Figure 26: Example of a build stream

When the product on the build stream is stable, you can create a baseline in the build stream and deliver it to the integration stream. The build stream baseline can then be recommended by the integration stream, and the development streams can rebase to it.

Composite baselines in release-oriented projects

When projects are organized in a release-oriented manner, use composite baselines to shorten recommended baseline lists. Shorter lists make it easier to keep track of which baselines were recommended at different times. The composite baselines can be used to represent major subsystems in the product or, in the simplest case, a project can consolidate all of its baselines into a single component, which it recommends and delivers to follow-on projects.

Additionally, using a single composite baseline to represent the final product -- collecting baselines from all of the components-reduces the occurrence of baseline conflicts. If a project uses a single composite baseline, the project leader can implement a process rule that forces all builds to occur from the top-level component, which reduces the probability of a developer inadvertently introducing conflicts.

Developers can still choose to use overrides when they need to access materials that are not included in the recommended baseline set. For example, if a developer needs a bug fix that hasn't completed the process for baseline recommendation, the developer can pick up the appropriate baseline for the specific component containing the fix, even though there isn't a baseline conflict.

A little care needs to be taken when using overrides in this way, because it is possible to introduce a conflict into a stream. Conflicts can occur when more than one composite baseline is overridden-but only when the overridden baselines are composites. Conflicts cannot be created when a single composite baseline is overridden, or if baselines of CM components are overridden.

Use composite baselines to model components in a component-oriented project

When a project team organizes its work by component, so that each component or subsystem has its own project for doing development work, significant savings can be realized by modeling the subsystems with composite baselines. Projects that integrate the subsystems can rebase to a single baseline to pick up a subsystem. If rootless components are used to represent the subsystems, then streams that consume its baselines will have more flexibility to later rebase to other composite baselines of that subsystem.

Composite baselines can be used to model subsystems, which are logical components built of smaller components. The components used in constructing a subsystem fall into two classes: components that are shared, and components that contain the work particular to that subsystem. The important property of the shared components is that they are non-modifiable. If a subsystem requires changes to a shared component, it reduces the ability to share that component. The modifiable, custom components store the code needed to integrate the shared components, and to provide the unique functions of the subsystem.

Returning to the example of the Mathlib subsystem, Figure 27 illustrates potential conflicts in shared components:

The Mathlib subsystem
Figure 27: The Mathlib subsystem

Mathlib is composed of the shared components Alpha and Gamma, and the custom component F. It doesn't matter whether Mathlib is a product, a .dll, a lower-level library, or some other component.

In the Mathlib example, a component-oriented approach to development can be more prone to baseline conflicts than a release-oriented approach. The use of a low-level component in multiple subsystems will often create conflict in still a higher-level subsystem. It is probably not practical to try to avoid such conflicts by coordinating all the projects so that only a single baseline of any low level component is in use by any project at one time.

Also, if development teams need complete freedom in selecting baselines of the components that they consume, then conflicts will be more likely, because of the difficulty in ensuring that a random baseline of one component, like Alpha, will work with a random baseline of another component.

While it is tempting to think of Mathlib as "consuming" baselines of Alpha and Gamma, it is not a true producer/consumer relationship. As was noted in the section titled Use rootless components for composite baselines, Mathlib isn't using the Alpha1 baseline; it's using Alpha1 plus the override, Cx. Therefore Mathlib is not actually using the product of the Alpha project. The Alpha project does produce Alpha baselines, but in a situation with conflicts, the baseline has to be seen as a guideline for projects that need Alpha rather than a rule.

Because overriding conflicts is a normal occurrence in component-oriented projects, there is a burden on administrators to make sure that they are selecting compatible baselines as overrides. This requires coordination among the leaders of the various projects, since the system cannot make such decisions automatically.

Selecting an override in a component-oriented development organization, therefore, needs to be a careful and deliberative process. The project leader needs to be aware that, in selecting an override, they are potentially rejecting the decisions made by the project that produced the component. There might be specific reasons why Gamma was built with the C1 baseline, and it may break when a different baseline is used. Often, using a descendant of C1 will work, but ClearCase doesn't restrict the selection of override baselines. There is no guarantee of the relationship between C1 and the override Cx.

As the ship date for a component approaches, the use of overrides can be destabilizing to a product because they represent significant code differences. Changes need to be managed by coordinating the work of the projects producing each subsystem. As the release date draws near, the teams come into sync with respect to the lower-level components, so that conflicts are reduced.


Summary

Composite baselines provide the capability to represent complex subsystems using a single identifier, and retain the flexibility and control of managing the lower-level components and subsystems independently. Composite baselines enhance the reusability of components and subsystems by making it easy to identify and include specific versions of those components in higher-level systems. As the size and complexity of the system increases, however, so do the possibilities of conflicts between subsystems using different baselines from shared components. Being aware of the restrictions and taking them into consideration when planning project hierarchy and baseline strategy will greatly enhance a development team's ability to reap the benefits that composite baselines provide. Reusability and simplified project administration will minimize the additional effort required to overcome baseline conflicts.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Rational software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Rational
ArticleID=5134
ArticleTitle=Best practices for using composite baselines in UCM
publish-date=07022004