Rational RequisitePro Client for Web 7.1.1.1: Performance Test Comparison Report

Content

Rational RequisitePro Client for Web 7.1.1.1

Performance Test Comparison Report

1. Introduction
   1.1 Document Terminology and Acronyms
2. Summary
3. Test Environment and Conditions
   3.1 Performance Testing Focus
   3.2 Project Used in Test
   3.3 Configurations Tested
   3.4 Test Topology
   3.5 Machine Specifications
4. User Simulation
   4.1 User Load
   4.2 Work Load
5. Detailed Performance Test Results
   5.1 RequisiteWeb 7.1.1.1 testing results with 50, 75 and 100 active users
   5.1.1 System Performance Execution Results
   5.1.2 Analysis and Conclusion
   5.2 Testing result comparison with RequisiteWeb 7.1.1.0
   5.2.1 System Performance Execution Results
   5.2.2 Analysis and Conclusion

1. Introduction

This report provides performance benchmark test results of the Rational RequisitePro 7.1.1.1 release compared to the previous release Rational RequisitePro 7.1.1.

This report covers product performance collected in a specific environment under specific test workloads. Product performance in other environments or under other conditions may be different.

1.1. Document Terminology and Acronyms

2. Summary

In this RequisitePro client for Web (RequisiteWeb) version 7.1.1.1 performance benchmark test, the performance test team tested RequisiteWeb 7.1.1.1 against a customer database. All the tests are run on a stable workload with user simulation. A varying amount of test runs were run against each condition, based on the requirements, to produce meaningful characterization of the system under load. The hardware used for the application server and database server remained constant for all tests. The test environment also included hardware for the Rational Performance Tester workbench and agent machines; it also remained constant for all tests.

In this document, the performance data collected from this test was compared from the following perspectives:

• RequisiteWeb 7.1.1.1 performance comparison under 50, 75 and 100 active user loads
• RequisiteWeb 7.1.1.1 performance comparison with RequisiteWeb 7.1.1

Observations:

• For RequisiteWeb 7.1.1.1, the test results seem to be equal with 50 and 75 active users. When more than 75 users are active, the response times increase. The page element success rate remains acceptable (higher than 98%) throughout all the user load tests. Note: Using RequisiteWeb in a Load Balanced environment will support a higher number of users.
• Compared with RequisiteWeb 7.1.1, RequisiteWeb 7.1.1.1 has improved its overall performance very significantly.

3. Test Environment and Conditions

3.1. Performance Testing Focus

Performance testing focused on:

• RequisiteWeb performance with 50, 75 and up to 100 active users, against a customer database. For the database characteristics, please refer to Table 1.
• The traceability relationship/edition functions and performance bench marking.

The following table shows the details of project used.

3.3. Configurations Tested

3.4. Test Topology

Figure 1: Test Topology

3.5. Machine Specifications

4. User Simulation

4.1. User Load

The test collected RequisiteWeb server response times under multi-user loads. This table indicates the relative distribution of simulated users across the various user roles. The user counts used were 50, 75, and 100.

4.2. Work Load

This table describes the distribution of specific operation sequences for each user role (full allocation of 100% for each role). The percentage next to each corresponding action represents the portion that sequence has out of 100 for that user role.

5. Detailed Performance Test Results

5.1. RequisiteWeb 7.1.1.1 testing results with 50, 75 and 100 active users

For observing the performance trend when active users increase in the RequisiteWeb 7.1.1.1 test, test cases were executed with 50, 75 and 100 users under the same conditions. The response time and element page success rate of each operation for different roles were considered in this part of test.

5.1.1 System Performance Execution Results

In the following testing results, the figures and tables within this section all contain three sets of testing results. The first one is against 50 active users; the second one is against 75 active users; the third is against 100 active users.

The following figure shows a basic comparison for 50, 75, and 100 active users against response time in RequisiteWeb 7.1.1.1 performance test.

Figure 2: 10 Average Page Response Time Comparisons for 50, 75 and 100 Active Users

The following table shows the page element success rate with different numbers of active users.

The following three figures show the trend of response time when the number of active users increases separately against Author, Editor and Viewer roles.

Figure 3: Response Time Trend for Increasing Users [Author Part]

Figure 4: Response Time Trend for Increasing Users [Editor Part]

Figure 5: Response Time Trend for Increasing Users [Viewer Part]

5.1.2 Analysis and Conclusion

From the above figures and table, the following points were observed:

(1). As a whole, the response time is not obviously changed when the number of users increases from 50 to 75 active users. It could be said that it is equal on performance under 50 and 75 active users.

(2). When the count of active users increases from 75 to 100, the response time increases quickly.

(3). The page element success rate shows a little bit of degradation when active users increases up to 100, but all rates were higher than 98%, which is acceptable in the test.

In conclusion for the RequisiteWeb 7.1.1.1 performance test, the test results seem to be equal under 50 and 75 active users. When more than 75 users are introduced, the response time increases. The page element success rate is acceptable (higher than 98%) up to 100 active users.

5.2. Testing result comparison with RequisiteWeb 7.1.1.0

The test results for RequisiteWeb 7.1.1.1 were compared to RequisiteWeb 7.1.1.0 in the same test environment.

5.2.1. System Performance Execution Results

The following figures and tables within this section all contain two sets of testing results. The former is based on RequisiteWeb 7.1.1.0, and the latter is based on RequisiteWeb 7.1.1.1.

The following figure shows a basic comparison for response times between RequisiteWeb 7.1.1.0 and 7.1.1.1.

Figure 6: 10 Average Page Response Time Comparisons [RequisiteWeb 7.1.1.0 VS. 7.1.1.1]

The following table shows the results for each operation with 50 users, and compares results for RequisiteWeb 7.1.1.0 and 7.1.1.1.

5.2.2. Analysis and Conclusion

Based on the figures and table above, RequisiteWeb 7.1.1.1 has improved its overall performance very significantly, compared with RequisiteWeb 7.1.1.0.

It's also worth noting that each test case was run more than 5 times and the test results were stable, that is, the results were reproducible.