Be sure the register for the Clearvision Webcast on March 25, 2008, where you can ask the experts about concepts described in this article. See the link below.
Software Change and Configuration Management (SCCM) considers the full software development lifecycle, from the initial emergence of changes (requirements, customer enhancements, defects) through implementation, build, testing, and eventually the formation and post-event management of a software release. Software Configuration Management (SCM) focuses purely on the process of tracking and controlling changes in software (and is therefore a component of SCCM).
A variety of SCCM standards are in use by the main software providers; however, the foundations stones for effective software development are clear and understood by all. Multiple products exist to enable effective implementation and day-to-day running of an end-to-end SCCM strategy and process. The bad news is that today, despite many commercially available and free open-source solutions, many companies continue to use spreadsheets and paper-based systems.
This article discusses relatively recent changes to the SCCM marketplace, in particular the rise and success of the open source SCM product Subversion. The intended audience for this article are SCCM decision makers/architects who define and enforce process and policy within their organization. SCCM product budget holders are also likely to find points of interest here.
The history of software change and configuration management
The basic principles of source code management were originally documented in the early 1960s by Professor Leon Pressor. The first serious source code management solutions were not available until the 1970s, however. Early solutions were created to help individuals manage their personal workspace for small quantities of files; modern tools can handle thousands of users in multiple locations with many more multiples of files and complexity.
One of the largest changes in the commercial market was the launch of ClearCase in 1992 by Atria Software (which was subsequently purchased by Rational Software and ultimately by IBM®). ClearCase was fundamentally different with its Dynamic View approach and eventually led IBM to develop (and purchase) multiple tools to manage the end-to-end software development process such as IBM Rational® ClearQuest®, IBM Rational RequisitePro®, IBM Rational Build Forge®, etc., and to create one of the more popular SCM methodologies, the IBM Rational Unified Process® (RUP®).
In the current economy, every organisation is closely examining costs and looking for possible reductions. At the same time developers want to use easier and leaner change and configuration management solutions with less overbearing process, restriction, and control. Subversion seems to meet the SCM goals of both parties: no license cost, simple to use, quick to learn, and it is inexpensive to administer and maintain.
Subversion does have some disadvantages though, which are described below. More important, Subversion is only half of the story, because the other key component of an effective software development process is change management. Regardless of which SCM strategy an organisation adopts, one of the most valuable and in many cases straightforward integrations is between change management and version control tools. Recording a change set within a change management tool enables or simplifies the following capabilities:
- Continuous integration, with one standard process
- Full audit, traceability, and detailed history of change
- Improved communication and visibility throughout the organisation (including communication to customers)
- Reduction in duplication of effort -- e.g., the same defect recorded multiple times
- Automated builds
- Automated and detailed release notes
- Improved testing -- i.e., each requirement can be specifically tested upon release
- Compliance with standards such as ISO and Sarbanes-Oxley, etc.
- Management and balance of workload across individual developers
- Improved reporting and management information
Although not trivial, it is generally considered simpler to migrate software and data into new configuration management tools such as Subversion than to migrate change records and processes into a new change management tool.
The success of open-source software is, without a doubt, having some effect on traditional vendor solutions. Smart providers such as IBM embrace open-source solutions to extend their products and provide flexible and fully functional development environments to help meet the needs and requirements of software development organisations.
The software development industry is clear that change and configuration management tools add value, improve software quality, and reduce ongoing costs, particularly as development teams grow larger, span multiple locations, and develop increasingly complex products. To support effective software development, it is no longer sufficient for these tools to stand in isolation; they need to be integrated, flexible, and adaptable.
Of all the tools used within the software development process, the simplest of integrations is between change management and configuration management. Moreover, this integration generally reveals more benefit than any other (greatest return on investment).
Achieving the ideal SCCM solution
The aims and key requirements for any SCCM system will vary depending upon the role and tasks of the individual user:
From a developer perspective, the ideal SCCM solution must
- Be unobtrusive and easy to use.
- Allow developers to work against one or more assigned activities.
- Give them the ability to have more than one active work item at any one time, but not have each work item interfere with another.
- Associate the resolution for one change record with other change records which are also resolved by the same fix.
- Make their code available for testing quickly and easily.
- Record requirements, and make them clearly understood (XP/Agile) or defined (RUP).
- Manage merging -- i.e., only integrate the files which actually need to be merged.
- Work from a "to do" list of items assigned specifically to each developer (also allowing the manager to prioritise work items).
From a Development Team Lead/Project Manager perspective, the ideal SCCM solution must
- Easily receive, evaluate, and manage change requests.
- Easily assign work items or split into smaller work items as appropriate for the specifics of the team, or the target delivery -- i.e., a single work request might need to be broken down into smaller work items because of team capabilities, time constraints, or flexibility for staggered delivery of the original work item.
- Distinguish which work items are complete versus incomplete.
- Easily assign work items for testing and inclusion to a build.
- Provide visibility at various levels, including status of project, release, components, and individual changes. This includes management reports such as change record distribution across team members, average bug fix time, etc.
- Provide visibility into individual developers' workload and progress.
From a Business Analyst/Manager perspective, the ideal SCCM solution must
- Allow them to request/register new requirements.
- Run reports against people, costs, project status, etc.
- Keep their customers updated with the progress of new releases.
Achieving a one-vendor SCCM solution
All software development organisations want an SCCM solution that is scalable, flexible, adaptable, all-encompassing, and ideally offers an integrated suite of tools (covering requirements definition through to testing and deployment) from a single vendor that will neatly accommodate existing development polices and processes (at the lowest possible cost). Primarily, the driver is to avoid making and maintaining integrations, and instead implement a reliable and tested solution that will not break integrations as new version upgrades are installed.
There is a general perception which says "it would be best if we could purchase our complete development environment from a single vendor," but in reality this happens rarely. The reasons are obvious: Every product influencer has their favourite vendors and products; legacy tools become deeply rooted into organisations, making them hard to migrate from; and over time new tools enter the market using modern technologies with functionality that outperforms last month's "tool of choice." Regardless of how you look at it, the reality for most organisations is a mixture of software development products.
Vendor solutions in today's SCCM market
In the software development market today, there exist the following types of vendor:
- Single flagship product with potential integrations to other aspects of the SCCM suite
- Multiple point tools from one vendor which claim to integrate seamlessly
- One-product vendor solutions for end-to-end SCCM
- Single licensed, all-encompassing Application Lifecycle Management (ALM) solutions
- Hosted solutions on remote servers; i.e., software as a service (SaaS)
There are so many choices that any decision can be overwhelming and difficult, each one having its own advantages and disadvantages. If you're starting a development team today, options 4 and 5 above sound like the obvious choices. But the problem with both these options is that one solution does not fit all. Inevitably, many organisations will recognise that an alternative product fulfills part of their overall solution better, that they are paying for parts of the solution they will never use, or that the solution is not flexible enough for their preferred development techniques (XP, agile, iterative, etc.). The truth about "out-of-the-box" ALM solutions is that all customers are different, so the more functionality becomes built into an ALM system the harder it becomes to adapt and customise that solution for the needs and processes of each customer. The best ALM solutions are lightweight and allow the customer to adapt or integrate them with other products.
No one wants to use a different product or vendor for every software development project, but a utopia is difficult to find without trying a number of products from the vast selection of excellent point tools available. The challenge is around tighter, cleaner integration. The ideal world would be to have one common interface regardless of the underlying SCM products.
Challenges for SCCM policy enforcers
The past two years have seen a shift in attitude by all parties involved in software development. Historically, the consensus was to rigidly guard all source code and strictly govern how it was managed. More recent software development techniques suggest that the tighter the shackles, the more difficult it becomes to develop products on time, within budget, and to a high quality level.
As pressure to get products to market increases, developers inevitably spend more time developing and testing, and they following what they consider to be bureaucratic processes that involve complex tools and an integrated SCM structure. By contrast, the open nature of Subversion allows developers to download and start using it to effectively circumnavigate company policy without having to seek approval or authorisation. Traditional SCCM policy enforcers want to keep a clear process -- for good reason -- but often find themselves powerless to intervene as developers force the issue. In fact, they often embrace this change rather than fight it.
Tools such as Subversion address the minimum business and development requirements, with the most basic and essential features. Why not allow developers to use faster, more efficient tools if they're integrated with a company-approved solution that provides all the backup, security, and audit requirements needed to meet compliance guidelines?
Cost has always been a major factor in the uptake of commercially available SCCM products, and many companies are now moving to the simpler "use and learn" tools for many aspects of an SCCM suite, especially open-source SCM solutions such as Subversion or Mercurial.
IBM Rational has been at the front of the development tools market for years with their integrated suite of products. Traditionally, they have not created integrations beyond their own suite of tools; however, more recently they have acknowledged a wider market opportunity, especially with tools from the open-source arena.
Many organisations recognise they no longer require overly complex configuration management solutions, so long as they place more emphasis on the change management solution and place tools such as Rational ClearQuest at the heart of development.
Migrating your change management solution
Central to everyone involved in the software development lifecycle is the change management process. For organisations without a change management solution, the benefits of implementing one are clear and compelling.
For organisations that already have a change management product, nearly every aspect of a business is likely to have involvement with the software change management process. Furthermore, processes will have been built up and refined over many years. Moving from their existing change management solution to a new product is a significant undertaking, not to be tackled lightly. The question is not if you should change, but rather how to weigh the potential benefits against the cost of migration.
Companies considering migrating from their change management solution must contemplate the following possibilities and contingencies:
- Losing or breaking reports based on historic data or statistical analysis (often essential for companies with SLA contracts).
- The vast majority of commercial change management tools use databases in the backend; however, recreating a schema and behaviours can be challenging.
- Exposing information that should remain private/confidential.
- Migrating scripted, customised behaviours and existing product integrations.
- Existing integrations and customer portals.
- Educating all users in a short time frame while managing disruption to the business.
- Impact on remote users and customers.
Companies must ask themselves if switching their change management product for an alternative solution is a sensible decision, and not automatically migrate to a new change management tool simply because their version control product is changing. If nothing else, any company considering an SCCM solution migration should migrate in manageable stages -- i.e., migrate the version control first, then migrate the change management after the initial move has been proved successful and stable.
The key question is "How do you choose and implement an SCCM solution which meets such a wide ranging collection of requirements?" The answer is to create a set of processes that incorporate best practice and are easily portable, regardless of the technology and software products around them.
Focusing investment in the right area
Traditionally, the heart of any SCCM solution was configuration management (version control). At a minimum, any configuration management solution must:
- Version and manage files and directories while tracking and maintaining history.
- Provide audit capability.
- Manage baselines and releases.
- Allow parallel development and manage merging from branches.
- Provide user access (easily manage data security and user access).
- Be dependable and scalable.
- Integrate into the development environment (the IDE).
- Provide light application lifecycle management.
- Support multiple platforms and cross-platform implementations.
- Automate manual or error-prone tasks.
The majority of standalone configuration management solutions today provide this functionality as standard. It is generally accepted that investment in change management gives a better return than configuration management because a wider audience makes use of the information available from change management systems (requirements, design, defects, enhancements, build information, release notes, etc.). Controlling and managing the flow of information throughout the organisation, change management systems quickly become the focal point for software development.
Many organisations are looking for a simple approach to ALM, something which can manage small isolated activities of work and control their integration with other activities. The essential requirement for recording, retrieving, and reporting on development tasks are the capabilities offered by change management tools such as Rational ClearQuest, Jira, Trac, Bugzilla, etc.
The change management tool inevitably becomes the focal point for all information. Ensuring change sets are automatically recorded within the change management tool is invaluable:
- Should a record require rework, the original change set file list is available for inspection.
- Release notes (down to the level of specific files changes) can be easily extracted.
- The change management tool can be used to drive a fully automated continuous integration and merge process. Team leader or build masters select from the list of completed activities, those required in the next build. The change management tool reads the change set list and performs an automated merge on the recorded change set.
In early 2000, Karl Fogel and Jim Blandy initiated a new project to correct what were considered imperfections with CVS. Initially the community was small but grew quickly once the simple objectives of Subversion became public knowledge.
They were not attempting to write a heavyweight configuration management tool, but something which resembled CVS with improved performance. They also wanted to address the software development challenges of today.
- Cost. Under the Debian Free Software guidelines, 1 Subversion is free to download, modify, and redistribute.
- Reliability. It's no exaggeration to say one of the great advantages of open-source is that everything works reliably and is simple to download and use. Of course there are defects but they are quickly resolved with pre-release testing from the community being intensive. The most reliable statistics indicate that Subversion is growing at a rate of more than 200,000 downloads per month, with over 2.5 million Subversion users.
- Flexibility. The architecture of Subversion has been carefully and cleverly designed with regard to performance, usability, and scalability. Furthermore, it is built with Internet technology and distributed development in mind, unlike many of the older tools available.
- Ease of use. Those with CVS experience will understand Subversion quickly. The third party graphical interfaces that integrate with Subversion (Tortoise, Eclipse, NetBeans, etc.) effectively make Subversion disappear into the background. Subversion is also relatively simple to deploy and maintain.
- Openness. Information is visible and open to scrutiny. This helps partner companies and users plan new releases or evaluate how a recently submitted enhancement is progressing.
Subversion is suited for developers who want to work in a secure, agile way but at the same time include a minimal amount of control at the file version level. It is not just technically minded developers who need to register file changes for a project, but also less technically aware business users who need to manage non-code related artifacts. Installing a heavy configuration management tool can be off-putting for even technically minded people. Subversion, however, is intuitive to use and install.
Figure 1 illustrates developers changing files based upon activities; each activity is isolated and self-contained. The activities are integrated one at a time to an integration stream after which they are built and deployed to appropriate test environments. Managers are not interested in file versions, but only the finished artifacts or, more importantly, what has not been finished and might impact a deadline so that they can accurately report back to the key decision makers and customers.
Figure 1: File changes are based on activities, which are integrated one at a time to an integration stream, after which they are built and deployed to appropriate test environments.
Subversion has many advantages, but no solution meets all of the ideals. Here are a few of the disadvantages of Subversion:
- Cost to migrate data to a new SCM tool. It is not always possible to capture the full history trail from the original tool. It should be noted that this is not unique to Subversion.
- It is open-source. No single vendor exists to provide support and guarantee the product will continue to be enhanced and developed. Clearvision offer a full range of Subversion related services including support, training and additional tools.
- Subversion is still relatively young. It therefore lacks some of the key functionalities of more mature SCM solutions. Development of Subversion has a clear and open plan, so inevitably the community will implement the missing pieces of the jigsaw.
Combining vendor solutions with Subversion
The impact of Subversion on traditional vendors has been significant. And there's no question that Subversion will continue to be a successful SCM product, whether due to price points, architecture, or functionality.
The development IDEs such as Eclipse, JBuilder, NetBeans, IBM Rational Application Developer, IBM Jazz, and independent ALM solutions such as CodeBeamer from Intland are masking the need to interact with the underlying version control tool. In Q2 2008, Clearvision will release a lightweight Rational Unified Change Management (UCM) style layer for Subversion. Prior to this, third-party interfaces vendors such as IBM, Serena, Perforce, and Accurev provide user interfaces to improve usability of the underlying change and configuration management tool.
Companies who have historically used a combined solution of Rational ClearQuest and IBM Rational ClearCase®, and are now introducing Subversion to harness the individual strengths of both products and create a hybrid solution (ClearCase combined with Subversion), can continue to use ClearQuest as their total change management solution by using the ClearQuest Subversion integration from Clearvision. Anyone considering a change in tools must balance the impact of transitioning to a new product versus continued usage and extension of an existing tool.
Every tool has an "end of life." The question is whether each part of your existing solution remains cost-effective and functionally suitable for today's environment (also considering the investment in time and effort used to tailor/implement, roll out, train staff, and maintain the original solution).
Integrated change management: ClearQuest and Subversion
Rational ClearQuest is an industry-leading change management solution which provides flexible change tracking, process automation, reporting, and lifecycle traceability for better visibility and control of the software and systems development lifecycle. Integration with software configuration management provides the ability to link development activities with file changes, recording the version of the file(s) changed within ClearQuest, and the activity information within the configuration management tool.
Once a change management tool has visibility into which files have been changed for any activity, it is relatively simple to perform complex tasks such as delivering just-changed files into a main development line. Having a development process that enables an integrator to pick and choose which items of functionality should be integrated with or backed out from a release gives incredible flexibility. However, without the initial record of the change set it would be extremely difficult to achieve.
It is generally accepted that development should not take place unless a request for change has been raised. Change requests are generally considered to fall into one of three types: defects, enhancements, and new functionality.
Even if development is in a preliminary state and considered to be exploratory, work should be registered against a change record. Even in the worst case, when a change record starts off as vague or high-level, it is at least being tracked and documented.
Developers generally understand the benefits of a change management system if the process is simple and does not interfere with or slow their progress.
Integrating ClearQuest and Subversion
Traditionally, ClearQuest has only integrated fully with ClearCase. With the introduction of CQ2SVN 2 (a Clearvision product), ClearQuest now fully integrates with Subversion, allowing Subversion users to benefit from a fully functional and integrated change management solution.
Rational ClearQuest is recognised as an industry leader in the change management market. Subversion, while relatively new, suits many projects that are seeking a development environment with reduced overheads, and it is thus especially suited to the agile way of working.
For those organisations presently using Rational ClearCase and Subversion (whether integrated or as separate SCCM products), the CQ2SVN interface works seamlessly alongside the existing Rational ClearCase/Rational ClearQuest integration (whether ClearCase Base and/or UCM).
The Subversion to ClearQuest integration is simple in its approach:
- Subversion user commits a set of files.
- CQ2SVN traps the commit and queries the ClearQuest repository for a filtered list of records (those assigned to the developer).
- The list of records is presented to the user on the client machine.
- The user selects one or more records they wish to associate with the activity.
- CQ2SVN then associates the check-out to an activity by writing the change set of files against the ClearQuest record and simultaneously writing the ClearQuest record ID against the Subversion revision.
The following screen images provide an idea of the CQ2SVN user interface.
Figure 2: CQ2SVN provides an additional Subversion tab in Rational ClearQuest records.
Figure 3: Automatically displaying the developer's Subversion check-in comments within Rational ClearQuest
Figure 4: Window allowing the Subversion committer to select the appropriate activity
On March 25, 2008, the experts from Clearvision will present a one-hour webcast titled "Effectively managing change in your Subversion Environment," based on the concepts presented in the above article. This webcast will show how the ClearVision integration (CQ2SVN) between Subversion and IBM Rational ClearQuest can be used to enhance your development environment to effectively manage change. Watch as we demonstrate the integration's utilization of change requests to seamlessly manage changes made to development artifacts.
Date: Tuesday, March 25, 2008
Time: 10:00 AM PT / 12:00 PM CT / 1:00 PM ET
Duration: One hour
Clearvision was established in 1997 and is the leading independent change and configuration management consultancy. The first UK IBM Rational partner, we are partnered with the world leaders in configuration and change management, including IBM Rational, WANdisco, Intland, Polarion, CollabNet, and Atlassian. These relationships have been built over many years and allow us to sell the latest products while also giving us access to resources and technical assistance.
Originally founded in the UK, Clearvision now has offices throughout Europe and the US and regularly works with clients in offices throughout the world. The Clearvision customer base includes global top 100 companies such as CitiGroup, HSBC, Barclays, Ericsson, Sony, and many more.
In recent years Clearvision has helped customers combine the power and reliability of IBM Rational change and configuration management solutions with the flexibility and low-cost of open-source solutions.
Clearvision is a market leader in:
- CM Training (both Web Based and conventional classroom training)
- CM consulting, strategy, process definition, and product recommendation
- Product integrations: e.g. CQ2SVN and Jira2SVN
CQ2SVN is an IBM Ready-for-Rational certified product. See the IBM Rational Website for more details, at http://www-304.ibm.com/jct09002c/gsdod/solutiondetails.do?solution=33614&expand=true&lc=en
- Participate in the discussion forum.
- A new forum has been created specifically for Rational Edge articles, so now you can share your thoughts about this or other articles in the current issue or our archives. Read what your colleagues the world over have to say, generate your own discussion, or join discussions in progress. Begin by clicking HERE.
-
Global Rational User Group Community
Gerald Tombs has worked in software development for more than twenty-five years and has specialized in configuration and change management for the last fifteen. He joined Atria Software in 1993, working with ClearCase until 1997 when he left Rational Software to become the founder of Clearvision, where he is responsible for company direction and product definition. He is lead architect for tools such as CQ2SVN (ClearQuest integration with Subversion) and the Web Based Training platform. Clearvision is the largest independent change management consulting firm, with offices in Europe and the USA.
Comments (Undergoing maintenance)
Table of contents
- The history of software change and configuration management
- The rise of open-source SCM
- The SCCM challenge
- Migrating your change management solution
- The emergence of Subversion
- Combining vendor solutions with Subversion
- Integrated change management: ClearQuest and Subversion
- Register for the webcast
- About Clearvision
- Resources
- About the author
- Comments
Next steps from IBM
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
