 | Level: Intermediate Razeyah Stephen, Manager, IBM
Joseph Malek (jmalek@us.ibm.com), Software Developer, IBM
Leon Desy (ldesy@us.ibm.com), Software Developer, IBM
18 Sep 2007 Learn about the IBM Lotus Notes/ Domino V8 enhancements to the NotesBench and Server.Load workloads. Three new workloads, N8Mail, N7Mail, and N8MailInit, demonstrate the typical tasks of a Lotus Notes user interacting with a Lotus Domino server.
One of the ways to achieve a close-to-real-world view of a Lotus Notes user is to closely mimic the API calls by the Lotus Notes V8 and Lotus Notes V7 clients. The N8Mail and N7Mail workloads used to reproduce those calls are completely new workloads with heavier transaction rates, which means that the data is different from your previous R6Mail results and that the results cannot be directly compared to each other. A future article will provide a detailed performance comparison of Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.
This article is intended for system administrators and designers interested in conducting performance measurements in their Lotus Notes/Domino V8 environments. This article assumes that you are an experienced Lotus Domino administrator.
NotesBench and Server.Load workloads
The NotesBench workloads are available only to members of the NotesBench Consortium. Server.Load, GUI version of the NotesBench workloads, is widely available to all customers. Server.Load is the Lotus Domino feature-based load generation tool that you can use to measure and characterize various Lotus Domino server capacity and response metrics. The workloads are available in both Server.Load and NotesBench for Lotus Notes and Domino V8. The workloads (also called tests or scripts) simulate the behavior of Lotus Domino client-to-server operations. The workloads allow you to use a common tool to apply a consistent, repeatable load against the Lotus Domino server so that you can assess the effects of various operating system, hardware, and configuration changes. For an overview of Server.Load, see the developerWorks Lotus article, "Introduction to Domino performance tuning." Server.Load is an installation option in the Lotus Domino Administrator client. Server.Load documentation is included in the Lotus Domino Administrator help.
| |
The Lotus Domino Performance Team continues to add features and enhancements to the Server.Load utility that ships with the Lotus Domino Administrator client. With each major release of Lotus Notes and Lotus Domino, you've told the team how your use has changed and, therefore, how our workloads should change. In addition to this advice, we have paid close attention to the changes in the Lotus Notes V8 client to represent workload user actions accurately. This accuracy stems from the team making sure the workload's transactions with the Lotus Domino server matches that of the Lotus Notes V8 client. Also, with this release, we have incorporated into the workload setup components of the Lotus Domino environment that were not involved in previous workload versions. The new components consist of all default server tasks, mail journaling, transaction logging, Lotus Domino Domain Monitoring (DDM) probes, mail rules, and larger mail files. All of this enables the N8Mail workload to provide a closer-to-real world view of a Lotus Notes user interacting with a Lotus Domino server in a more typical environment. Throughout this process, we also created the N7Mail workload, which in essence performs the same user workload actions as N8Mail. Behind the scenes, though, the individual commands accurately reflect the Lotus Notes V7 client transactions to the Lotus Domino V7 server. Because of this approach, all documentation pertaining to N8Mail is accurate for N7Mail as well. Last, we bring you the N8MailInit workload. This workload populates mail databases for use with N8Mail and N7Mail under the new profiled user model described later in this article.
The users shown in tables 1-4 are only instances of the benchmark script running, and they do not necessarily correlate to the number of actual users deployed on a server. The goal of the improvements in the N8Mail and N7Mail workloads is to bring the user model closer to a typical production user. Because representing every customer usage scenario is difficult, we emphasize that you should contact the IBM TechLine and your hardware vendors for the best capacity planning and deployment advice for your configuration and environment.
|
Workloads to use with different versions of Lotus Domino
N8Mail workload can be used to benchmark Lotus Domino V8.0.
N7Mail workload can be used to benchmark Lotus Domino V6.0.x, Lotus Domino V6.5.x, Lotus Domino V7.0.x, and Lotus Domino V8.0.
R6Mail workload can be used to benchmark Lotus Domino V5.0.x, Lotus Domino V6.0.x, Lotus Domino V6.5.x, Lotus Domino V7.0.x, and Lotus Domino V8.0.
|
|
Defining a workload usage pattern
When we start the development cycle of a new major release of Lotus Domino, we review the current version of the workloads. Through discussions with Lotus Notes and Domino developers and customers, review of production data, assessment of industry trends, and input from members of the NotesBench Consortium and various Lotus Domino performance groups, we define new workloads that simulate how we expect users' usage patterns to behave approximately 24 months in the future.
What do the new workloads provide?
After all the research (and while keeping your comments and feedback in mind), we developed the new workloads for Lotus Notes and Domino V8. These workloads do the following:
- Ensure accurate simulation of Lotus Notes V8 and Lotus Notes V7 transactions to the Lotus Domino server from the workloads.
- Account for a more realistic Lotus Domino server test environment than was done previously.
- Bring the profiled user model closer to an enterprise user by increasing the variety of features and usage patterns. Therefore, by using these workloads, you can get more realistic information (CPU, disk, memory, and network) for deployment planning than with prior workloads.
Workload specifications
N8Mail simulates full Lotus Notes V8 client users. The new features of the Lotus Notes and Domino V8 workloads include those shown in table 1.
Table 1. More realistic mail database profile for N8Mail and N7Mail
| N8Mail/N7Mail | R6Mail |
|---|
| Mail database size | Approximately 250 MB | 20 MB |
|---|
| Average document size | 100 KB | 10 KB |
|---|
| Initial document count | 3,000 | 100 |
|---|
Messages generated
The frequency of creating/sending memos has increased. Fifty total documents are generated in an average six-hour run, one per iteration, and an additional one every other iteration. One iteration is approximately 15 minutes. Memo recipients now vary between one and three recipients. There are now fewer invitations and appointments, one each every six hours. Overall, you find 50 memos, invitations, and RSVPs to about 78 recipients, per user for a six-hour run. A new feature is the generation of replies to received mail, one every other iteration. See table 2 for more details about memo profiles.
Table 2. Memo profile for R6Mail, N8Mail, and N7Mail
| Message body size in KB | Attachment size in KB (if any) | Percent |
|---|
| 500 | N/A | 10 |
|---|
| 10,000 | N/A | 30 |
|---|
| 50,000 | N/A | 40 |
|---|
| 50,000 | 50,000 | 10 |
|---|
| 150,000 | N/A | 9.5 |
|---|
| 1,000 | 10,000,000 | 0.5 |
|---|
This profile is the same in N8Mail and N7Mail as it was in R6Mail, so you can assume this distribution to be valid. To get the big picture of what has changed
in the new workloads, review table 3, which shows data from the perspective of recipients of mail and the message distribution therein.
Table 3. Message distribution in N8Mail and N7Mail
Message size distribution
(using the profile option) | Percent of messages to total number of recipients | Attachment size (if any) |
|---|
| 0 < size <= 1K | 32.0% | N/A |
|---|
| 1K < size <= 10K | 3.6% | N/A |
|---|
| 10K < size <= 100K | 57% | 50 KB |
|---|
| 100K < size <1M | 6.8% | N/A |
|---|
| 1M < size <= 10M | 0.4% | 10 MB |
|---|
NOTE: The percentages do not match the percentages defined in the message profile discussed previously because the profile is used only for memos. It is not used for replies, invitations, or RSVPs, which are accounted for in this table.
N8Mail and N7Mail new user actions include the following:
- Move random notes from the inbox to the NotesBench folder
- Read the newest inbox items
- Reply to a read message
- Save all sent messages in the Sent folder
- Conduct new mail polling every 15 minutes (that is, every iteration)
As you can see in table 4, there are now 35 more transactions overall (267 - 232) in N8Mail/N7Mail than there were in
R6Mail for a difference of 1.45 more transactions per iteration per user.
Table 4. User actions: N8Mail, N7Mail, and R6Mail (six-hour period, 24 iterations)
| Workload actions | N8Mail and N7Mail workload action count | R6Mail workload action count |
|---|
| Refresh inbox | 24 | 24 |
|---|
| Read message | 120 (reads 5 per iteration) | 120 (reads 5 per iteration) |
|---|
| Reply to all | 12 | N/A |
|---|
| Send message to one recipient | 24 | N/A |
|---|
| Send message to three recipients | 12 | 4 with NthIteration set to 6 |
|---|
| Create appointment | 1 | 4 with NthIteration set to 6 |
|---|
| Send invitation | 1 | 4 with NthIteration set to 6 |
|---|
| Send RSVP | 1 | 4 with NthIteration set to 6 |
|---|
| Move to folder NotesBench | 24 | N/A |
|---|
| New mail poll | 24 | N/A |
|---|
| Delete two documents | 24 | 24 |
|---|
| Stamp documents (Release 6) | N/A | 24 |
|---|
| Add documents to the inbox (Release 6) | N/A | 24 |
|---|
| Total transactions (per user in a six-hour period, 24 iterations) | 267 | 232 |
|---|
NOTE: One iteration is approximately 15 minutes. Also, because N8Mail and N7Mail have identical sets of user actions, they are shown in one column, and the workload action counts are identical.
- Send 3,000 mail notes (from distinct users) to populate the inbox
- Create necessary profile documents with appropriate fields
- Create a NotesBench folder
- Generate five replies from what is newest in the inbox
- Put ten invitations in the database
- Optionally add the blocksender rule
- Add ten users not in the range of valid workload users to the rule
- Optionally add the color profile
- Run Updall -R <maildb> to update the maildb
Server configuration
|
Test database setup for mail journaling
NOTE: For mail journaling to work correctly, distinct replication IDs are needed for the mail databases. After you create the mail databases,
run this script across all to ensure this distinction.
The following three lines can be copied to a text file and run using Server.Load as a customized script. For client notes.ini settings, MailServer refers to the server on which the mail databases reside; nb_dbdir is the directory relative to the data directory where the mail databases can be found.
changeto [MailServer]!![nb_dbdir]mail[#].nsf
setreplid
dbclose
|
|
In addition to offering improved user actions, the N8Mail and N7Mail workloads use server features not used for R6Mail. Using these features brings the benchmark environment closer to your deployment environment. This increases the Lotus Domino server CPU utilization.
Server features include the following:
- Transaction logging. Disk space is approximately 4 GB, Circular, Standard, or favor run time.
- Mail journaling. Disk space is approximately 4 GB, and all messages are journaled.
- DDM probes. All operating system and messages with default values (events4.nsf) are enabled.
- Mail filtering. Each mail file has 10 users in the block mail list. The 10 users are not in the active user list in the workload run. The goal is measuring the overhead of having filtering active, not the overhead of doing the filtering.
- Use default server tasks. ServerTask=Update,Replica,Router,AMgr,AdminP,CalConn,Sched,RnRMgr,LDAP (from the server's Notes.ini file)
See the Release Notes for Lotus Notes and Domino V8 for more information about the N8Mail, N7Mail, and N8MailInit workloads.
Conclusion
N8Mail promises to be highly accurate and enables our heavier Lotus Notes and Domino V8 workloads to help our user model match your expectations more closely. Of course, these are still only benchmark tools. We believe these workloads, along with other data you obtain from sources such as your hardware vendors, can help better equip you to more accurately plan your current and future upgrades, purchases, and deployment decisions. A follow-on article uses the new workloads to compare Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.
Resources Learn
Discuss
About the authors | | | Razeyah Stephen is manager of the IBM Lotus Domino and IBM Lotus Sametime performance team. She joined Iris Associates in October 1998. |
| | | Joe Malek is a developer for the IBM Lotus Domino performance team. His current focus involves developing performance tools for the Lotus Domino performance team working on various platforms such as Microsoft Windows, Linux, IBM AIX, and Sun Solaris. Joe joined Iris Associates in October 2000. |
| | | Leon Desy is a developer for the IBM Domino performance team. His current focus involves developing performance tools for the Domino performance team working on various platforms such as Microsoft Windows, Linux, IBM AIX, and Sun Solaris. Prior to joining the team in August 2006, Leon worked in L3 support for Lotus Notes/Domino for eight years. He joined Iris Associates in August 1998. |
Rate this page
| |
