As demands for productivity in the use of software have increased, the ability to follow processes efficiently has become increasingly important. The efficient performance of software processes enhances product quality and stakeholder satisfaction.
This article discusses the benefits and the performance, reliability, and scalability gains from changes made to IBM® Rational® ClearCase® and Rational® ClearQuest® Change Management (CM) Server in the V184.108.40.206 release. Information provided shows the advantages of migration or upgrade to 220.127.116.11 for Rational ClearCase Remote Client and ClearQuest Web from earlier releases.
Fixes and features in Rational ClearCase and ClearQuest 18.104.22.168
- Added ClearCase Remote Client/CM Server load balancing support with IBM HTTP server
In addition to the CM Server load balancing with IBM® WebSphere® Application Server Network Deployment Edge Component in Version 7.1, ClearCase Remote Client and CM Server load balancing can be accomplished with the IBM® HTTP Server that ships with the Rational ClearCase and ClearQuest software.
For more information about configuring ClearCase Remote Client and CM Server load balancing his with the HTTP server for ClearCase Remote Client, see IBM Technote 1377474.
- Implemented compression of downloaded cleartext
Loading a ClearCase Remote Client view or updating a ClearCase Remote Client view is one of the most commonly operated ClearCase Remote Client operations. Time spent for this type of operations includes computing what is to be loaded and transmitting the files. The time for transmitting is much longer than computing the contents. Compressing the downloaded cleartext resulted in an average of 30% increase in speed for ClearCase Remote Client view load or update over a WAN connection. One thing to note is that the size of the download is the major factor in the speedup. So the larger the download, the faster the speed.
- Forced a synchronous data transfer in CM Server
The use of asynchronous data transfer requires an excessive number of buffers to send data over a TCP/IP connection. This mainly occurs when files are being transferred, but it also occurs with normal responses from IBM® WebSphere® Application Server where the CM Server resides. In the 22.214.171.124 release, CM Server forced the synchronous data transfer by setting the WAS attribute:
This solved the excessive native memory use in the WebSphere Application-Change Manager Server, which was a problem observed with CM Server 126.96.36.199 and 188.8.131.52 releases.
- Optimized some heavily used CM Server code paths to improve performance, reliability and to fix memory leaks
CM Server architecture contains multiple layers, as seen in a typical Java™ 2 Platform, Enterprise Edition (J2EE) application, such as servlet, Enterprise JavaBeans™, Java™ Connector architecture (J2C) connection, and so on. The layered implementation, however, presents problems such as code complexity, data accumulation, and difficulty of debugging. In the 184.108.40.206 release, we removed one layer from the call stack, which resolved a JVM heap
OutOfMemoryissue discovered in long run or heavily loaded test scenarios and resulted in great improvement in performance, reliability, and transaction throughput.
- Fixed memory leak problem and deadlock issue in CM API code and in ClearQuest Web code
A ClearQuest Web 5-user as fast as possible test showed a deadlock situation in the CM API code. This issue was addressed by tightening the scope of some synchronized method calls.
The ClearQuest Web 130-user test showed memory leaks in CM API code and in ClearQuest Web code, which have been fixed.
- Fixed a problem of excessive CCRPC process creation during high concurrent load test scenario.
There was a flaw in the CM Server lifecycle management algorithm that could result in excessive ClearCase RPC (CCRPC) processes and
OutOfMemoryerrors in a CM Server during a multi-user, high-load ClearCase Remote Client test scenario. This algorithm was improved to take into consideration all known CCRPC processes when determining if the system has capacity to allocate a new one. If the system resource limit or the capacity MBean attributes are reached, CM Server issues a "Server is busy" error. Wait for a few minutes and retry the operation.
- Improved the ClearCase Remote Client checkin and AddToSourceControl (mkelem) performance by increasing the data transfer chuck size
For ClearCase Remote Client checkin and
mkelem) operations, there are multiple threads involved in streaming the data from the request socket to the back end. They are all activated once for each "chuck." By increasing the data transfer chuck size (from 1000 bytes to 32K), and thus reducing the context switching, performance was greatly improved.
- Delivery of CM Server Administration GUI
CM Server Admin GUI is a browser-based tool for monitoring and managing CM Server behavior and for reporting active ClearCase Remote Client users and ClearCase Web active users. This tool can be used to easily change CM Server MBean attributes without any scripting.
For more information about CM Server Administration GUI, see IBM Technote 1377925.
These changes, collectively, improved the CM Server performance, reliability, and scalability, and enhanced the usability. See Comparison of sample test data for the ClearCase Remote Client, Versions 220.127.116.11 and 18.104.22.168, and Sample test data for ClearQuest Web, comparison between 7.0.1 and 22.214.171.124, for detailed test reports of the 126.96.36.199 release.
New test cases added in the 188.8.131.52 release
Since Version 184.108.40.206, the Rational Performance Engineering Team and Rational System Verification Team have added additional test configurations and the following test cases:
- High transaction rate CM Server stress test
This test simulation tested users working as fast as machine speed will allow to test the CM Server.
- High user load concurrent load view tests
Load view is one of the most demanding ClearCase Remote Client operations. We have run up to 100 concurrent load view operations, loading directories of varying sizes from the server.
- IHS load balance tests on supported CM Server platforms
IHS load balance support was added in V220.127.116.11. We have run ClearCase Remote Client load balance tests on almost all supported OS platforms and validated this improved load balance solution.
- ClearQuest Web 75-, 150-, and 225-user scalability tests on the Windows platform
Rational Performance Engineering ran a scalability test to demonstrate the 18.104.22.168 release's improved scalability and performance by using a schema of medium complexity on the Microsoft® Windows® operating system (OS).
- Rational ClearQuest Web 130-user, 3-day test on the Linux platform.
Our System Verification Test team conducted a 130-user, 3-day, 15 transaction per hour test of Rational ClearQuest Web and CM Server against a populated application lifecycle management (ALM) schema database on the Linux® platform.
- ClearCase Remote Client 200-user, 3-day test run on the Linux platform
System Verification Test team conducted a 200-user, 3-day, 15 transactions per hour ClearCase Remote Client test against CM Server on RedHat Linux 5. Tests were executed by using the CM API for ClearCase.
- CM Server Administration GUI load test
Rational Performance Engineering and System Verification teams ran load tests to demonstrate that constant use of the CM Server Administration GUI has no adverse effect on CM Server performance.
Comparison of sample test data for the ClearCase Remote Client, Versions 22.214.171.124 and 126.96.36.199
Table 1. Hardware configuration for the tests
|Test driver||IBM x336 with 2 Intel Xeon 3.4 GHz CPUs, 4 GB RAM, |
OS: RHEL 4
|CM Server||IBM x366 with 4 Intel Xeon 3.67 GHz CPUs, 8 GB RAM,|
OS: RHEL 5.3
|VOB server||IBM x366 with 4 Intel Xeon 3.67 GHz CPUs, 8 GB RAM, |
OS: RHEL 5.3
The directory size in the LoadView operation was 22 MB. The size of the file in MakeFile, Checkin, and Checkout operations was 100 KB. A FreeBSD DummyNET WAN simulator was between the IBM® Rational® Performance Tester driver machine and the CM Server. We simulated 100 ms round-trip latency and 3 Mbps network bandwidth for these tests.
The graph in Figure 1 shows ClearCase Remote Client CM API performance improvements in 188.8.131.52 in single user benchmark tests.
Figure 1. Single user CM API benchmark
In multiuser tests, the combined impact of aforementioned performance improvements in 184.108.40.206 is much more visible than the single user benchmark tests. In the 25 user tests, there was 400% performance improvement for load view operation, and 10% to 50% performance improvements for other operations, 220.127.116.11 over 18.104.22.168.
Figure 2. The 25 user CM API benchmark
Sample test data for ClearQuest Web, comparison between 7.0.1 and 22.214.171.124
Table 2. Hardware configuration for the tests
|Test driver||Lenovo ThinkCentre with two dual 3 GHz Intel® Pentium® 4 CPUs, 2.5 GB memory|
Operating system: Microsoft Windows XP Professional, Version 2002 Service Pack 2
|Rational ClearQuest Web and CM Server||IBM® IntelliStation® Z Pro with two dual 2.40 GHz Intel® Xeon® CPUs, 4 GB memory, Physical Address Extension enabled|
Operating system: Microsoft Windows 2003 Server, Enterprise Edition with Service Pack 2
|Database server||IBM® xSeries® 345 with two dual 3.06 GHz Intel Xeon CPUs, 8 GB memory, Physical Address Extension enabled|
Operating system: Microsoft Windows 2003 Server, Enterprise Edition with Service Pack 2
Database server: Microsoft SQL Server 2005 Service Pack 3
The graph in Figure 3 shows ClearQuest Web single-user performance improvements between 126.96.36.199 and 188.8.131.52:
Figure 3. Rational ClearQuest Web 184.108.40.206 compared to ClearQuest Web 7.0.1 single-user benchmark
The graph in Figure 4 shows ClearQuest Web scalability for the 220.127.116.11 release.
Figure 4. ClearQuest Web 18.104.22.168 scalability
Fixes and features planned for the next patch releases
This section lists the performance-, reliability-, and scalability-related work that we are working on now. No feature or fix is guaranteed.
- Better cleartext upload compression to further speed up ClearCase Remote Client view load and update
- ClearCase Remote Client client-side code optimization
- Improve ClearCase Remote Client data transfer by increasing the size of file being uploaded. Preliminary test showed significant reduction in time for uploading large files and good increase in network utilization.
- Efficient data fetching for ClearQuest record fields that contain large number of references.
- Fast ClearCase Remote Client discordance detection.
- Continue to simplify CM Server call stack .
- Increase reliability by guarding against memory corruption in CM Server CQ Web backend
- Fast record form loading for ClearQuest Web.
- Performance improvement to ClearCase Remote Client UCM activity and bound CQ record in CQ-enabled UCM environment by efficient look up and caching.
- Performance improvement to ClearCase Remote Client pending change view. Additional options in Pending Changes view configuration to scope by components (for UCM) or directory structure (for base CC).
- Additional scalability improvements.
In addition, the next release of Rational ClearCase and ClearQuest CM Server will include FixPack 25. The Rational Performance Engineering and Rational System Verification teams have tested WebSphere Application Server FixPack 25. Various tests have shown great improvement in performance, reliability, and transaction throughput with FixPack 25. An IBM Technote on how to upgrade your CM Server WebSphere Application Server installation to FixPack 25 has been published. See Resources for a list and links.
IBM continues to work diligently to improve Rational CM Server, ClearCase Remote Client, and ClearQuest Web performance. As we have stated in this report, there have been dramatic improvements made to CM Server since the 7.1 release. These improvements make ClearCase Remote Client and ClearQuest Web perform better and make the server more reliable and scalable.
Data and facts provided by:
David Qiang Li, Grant Covell, Paul Weiss
IBM Rational Performance Engineering
Mark Duquette, Denice Ortege, Joey Zhou, Lily Shi
IBM Rational System Verification Team
Development work by:
Kevin Esler, Jon Freyberger, Steve Rehrauer, Rob Title, Yuhong Yin, Doug Mckenzie, Masa Koinuma, Terry Sun, Shaila Golikeri, Daniel Wang, Leon Liang, Jack Ma, Knut Radloff, Herb Miller, Matt Lennon, Adina Adler, Rob Stanzel, Mike Prentice, Mike Mole, Liz Bonesteel, Jim Wu, Howie Bernstein, Eleonora Ludin, Hans Heilman
IBM Rational CM Server Team, ClearQuest Web Team, and ClearCase Remote Client Team
Here is a list of tech notes related to CM Server for quick reference:
- CM Server Configuration Guidelines
- ClearCase Remote Client Load Balancing with IBM HTTP Server (1377474)
- ClearQuest Web Load Balancing with IBM HTTP Server (1377478)
- CM Server Administration GUI (1377925)
- Preserve the CM Server Configuration Before Customizing the MBean Attributes or Upgrading or Reinstalling ClearCase and ClearQuest (1384746)
- Recommended WebSphere Application Server 6.1 FixPack version
- How to update the IBM WebSphere Application Server components in IBM Rational ClearCase and IBM Rational ClearQuest version 7.1 (1390803)
- Enable CM Server Administrative Security (1386762)
- Learn more about ClearCase:
- Explore the Software Configuration Management Information Center to learn how features such as local and remote access, a proven use model, a wide range of supported environments, transparent access to files, and parallel development support give your team members instant, controlled access to the project assets that they need to create, update, build, reuse, and maintain your software.
- Learn about other applications in the IBM Rational Software Delivery Platform, including collaboration tools for parallel development and geographically dispersed teams, plus specialized software for architecture management, asset management, change and release management, integrated requirements management, process and portfolio management, and quality management. You can find product manuals, installation guides, and other documentation in the IBM Rational Online Documentation Center.
- Visit the Rational software area on developerWorks for technical resources and best practices for Rational Software Delivery Platform products.
- Explore Rational computer-based, Web-based, and instructor-led online courses. Hone your skills and learn more about Rational tools with these courses, which range from introductory to advanced. The courses on this catalog are available for purchase through computer-based training or Web-based training. Additionally, some "Getting Started" courses are available free of charge.
- Subscribe to the Rational Edge newsletter for articles on the concepts behind effective software development.
- Subscribe to the IBM developerWorks newsletter, a weekly update on the best of developerWorks tutorials, articles, downloads, community activities, webcasts and events.
- Check out Rational ClearCase forum to post questions, find answers, and share solutions about configuration management and Rational ClearCase.
Get products and technologies
- Download trial versions of IBM Rational software.
- Download these IBM product evaluation versions and get your hands on application development tools and middleware products from DB2®, Lotus®, Tivoli®, and WebSphere®.