One of the ways to achieve a close-to-real-world view of a Lotus Notes user is to closely mimic the API calls by the Lotus Notes V8 and Lotus Notes V7 clients. The N8Mail and N7Mail workloads used to reproduce those calls are completely new workloads with heavier transaction rates, which means that the data is different from your previous R6Mail results and that the results cannot be directly compared to each other. A future article will provide a detailed performance comparison of Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.
This article is intended for system administrators and designers interested in conducting performance measurements in their Lotus Notes/Domino V8 environments. This article assumes that you are an experienced Lotus Domino administrator.
NotesBench and Server.Load workloads
The NotesBench workloads are available only to members of the NotesBench Consortium. Server.Load, GUI version of the NotesBench workloads, is widely available to all customers. Server.Load is the Lotus Domino feature-based load generation tool that you can use to measure and characterize various Lotus Domino server capacity and response metrics. The workloads are available in both Server.Load and NotesBench for Lotus Notes and Domino V8. The workloads (also called tests or scripts) simulate the behavior of Lotus Domino client-to-server operations. The workloads allow you to use a common tool to apply a consistent, repeatable load against the Lotus Domino server so that you can assess the effects of various operating system, hardware, and configuration changes. For an overview of Server.Load, see the developerWorks Lotus article, "Introduction to Domino performance tuning." Server.Load is an installation option in the Lotus Domino Administrator client. Server.Load documentation is included in the Lotus Domino Administrator help.
The Lotus Domino Performance Team continues to add features and enhancements to the Server.Load utility that ships with the Lotus Domino Administrator client. With each major release of Lotus Notes and Lotus Domino, you've told the team how your use has changed and, therefore, how our workloads should change. In addition to this advice, we have paid close attention to the changes in the Lotus Notes V8 client to represent workload user actions accurately. This accuracy stems from the team making sure the workload's transactions with the Lotus Domino server matches that of the Lotus Notes V8 client. Also, with this release, we have incorporated into the workload setup components of the Lotus Domino environment that were not involved in previous workload versions. The new components consist of all default server tasks, mail journaling, transaction logging, Lotus Domino Domain Monitoring (DDM) probes, mail rules, and larger mail files. All of this enables the N8Mail workload to provide a closer-to-real world view of a Lotus Notes user interacting with a Lotus Domino server in a more typical environment. Throughout this process, we also created the N7Mail workload, which in essence performs the same user workload actions as N8Mail. Behind the scenes, though, the individual commands accurately reflect the Lotus Notes V7 client transactions to the Lotus Domino V7 server. Because of this approach, all documentation pertaining to N8Mail is accurate for N7Mail as well. Last, we bring you the N8MailInit workload. This workload populates mail databases for use with N8Mail and N7Mail under the new profiled user model described later in this article.
The users shown in tables 1-4 are only instances of the benchmark script running, and they do not necessarily correlate to the number of actual users deployed on a server. The goal of the improvements in the N8Mail and N7Mail workloads is to bring the user model closer to a typical production user. Because representing every customer usage scenario is difficult, we emphasize that you should contact the IBM TechLine and your hardware vendors for the best capacity planning and deployment advice for your configuration and environment.
Defining a workload usage pattern
When we start the development cycle of a new major release of Lotus Domino, we review the current version of the workloads. Through discussions with Lotus Notes and Domino developers and customers, review of production data, assessment of industry trends, and input from members of the NotesBench Consortium and various Lotus Domino performance groups, we define new workloads that simulate how we expect users' usage patterns to behave approximately 24 months in the future.
What do the new workloads provide?
After all the research (and while keeping your comments and feedback in mind), we developed the new workloads for Lotus Notes and Domino V8. These workloads do the following:
- Ensure accurate simulation of Lotus Notes V8 and Lotus Notes V7 transactions to the Lotus Domino server from the workloads.
- Account for a more realistic Lotus Domino server test environment than was done previously.
- Bring the profiled user model closer to an enterprise user by increasing the variety of features and usage patterns. Therefore, by using these workloads, you can get more realistic information (CPU, disk, memory, and network) for deployment planning than with prior workloads.
N8Mail simulates full Lotus Notes V8 client users. The new features of the Lotus Notes and Domino V8 workloads include those shown in table 1.
Table 1. More realistic mail database profile for N8Mail and N7Mail
|Mail database size||Approximately 250 MB||20 MB|
|Average document size||100 KB||10 KB|
|Initial document count||3,000||100|
The frequency of creating/sending memos has increased. Fifty total documents are generated in an average six-hour run, one per iteration, and an additional one every other iteration. One iteration is approximately 15 minutes. Memo recipients now vary between one and three recipients. There are now fewer invitations and appointments, one each every six hours. Overall, you find 50 memos, invitations, and RSVPs to about 78 recipients, per user for a six-hour run. A new feature is the generation of replies to received mail, one every other iteration. See table 2 for more details about memo profiles.
Table 2. Memo profile for R6Mail, N8Mail, and N7Mail
|Message body size in KB||Attachment size in KB (if any)||Percent|
This profile is the same in N8Mail and N7Mail as it was in R6Mail, so you can assume this distribution to be valid. To get the big picture of what has changed in the new workloads, review table 3, which shows data from the perspective of recipients of mail and the message distribution therein.
Table 3. Message distribution in N8Mail and N7Mail
|Message size distribution (using the profile option)||Percent of messages to total number of recipients||Attachment size (if any)|
|0 < size <= 1K||32.0%||N/A|
|1K < size <= 10K||3.6%||N/A|
|10K < size <= 100K||57%||50 KB|
|100K < size <1M||6.8%||N/A|
|1M < size <= 10M||0.4%||10 MB|
NOTE: The percentages do not match the percentages defined in the message profile discussed previously because the profile is used only for memos. It is not used for replies, invitations, or RSVPs, which are accounted for in this table.
N8Mail and N7Mail new user actions include the following:
- Move random notes from the inbox to the NotesBench folder
- Read the newest inbox items
- Reply to a read message
- Save all sent messages in the Sent folder
- Conduct new mail polling every 15 minutes (that is, every iteration)
As you can see in table 4, there are now 35 more transactions overall (267 - 232) in N8Mail/N7Mail than there were in R6Mail for a difference of 1.45 more transactions per iteration per user.
Table 4. User actions: N8Mail, N7Mail, and R6Mail (six-hour period, 24 iterations)
|Workload actions||N8Mail and N7Mail workload action count||R6Mail workload action count|
|Read message||120 (reads 5 per iteration)||120 (reads 5 per iteration)|
|Reply to all||12||N/A|
|Send message to one recipient||24||N/A|
|Send message to three recipients||12||4 with NthIteration set to 6|
|Create appointment||1||4 with NthIteration set to 6|
|Send invitation||1||4 with NthIteration set to 6|
|Send RSVP||1||4 with NthIteration set to 6|
|Move to folder NotesBench||24||N/A|
|New mail poll||24||N/A|
|Delete two documents||24||24|
|Stamp documents (Release 6)||N/A||24|
|Add documents to the inbox (Release 6)||N/A||24|
|Total transactions (per user in a six-hour period, 24 iterations)||267||232|
NOTE: One iteration is approximately 15 minutes. Also, because N8Mail and N7Mail have identical sets of user actions, they are shown in one column, and the workload action counts are identical.
- Send 3,000 mail notes (from distinct users) to populate the inbox
- Create necessary profile documents with appropriate fields
- Create a NotesBench folder
- Generate five replies from what is newest in the inbox
- Put ten invitations in the database
- Optionally add the blocksender rule
- Add ten users not in the range of valid workload users to the rule
- Optionally add the color profile
- Run Updall -R <maildb> to update the maildb
In addition to offering improved user actions, the N8Mail and N7Mail workloads use server features not used for R6Mail. Using these features brings the benchmark environment closer to your deployment environment. This increases the Lotus Domino server CPU utilization.
Server features include the following:
- Transaction logging. Disk space is approximately 4 GB, Circular, Standard, or favor run time.
- Mail journaling. Disk space is approximately 4 GB, and all messages are journaled.
- DDM probes. All operating system and messages with default values (events4.nsf) are enabled.
- Mail filtering. Each mail file has 10 users in the block mail list. The 10 users are not in the active user list in the workload run. The goal is measuring the overhead of having filtering active, not the overhead of doing the filtering.
- Use default server tasks. ServerTask=Update,Replica,Router,AMgr,AdminP,CalConn,Sched,RnRMgr,LDAP (from the server's Notes.ini file)
See the Release Notes for Lotus Notes and Domino V8 for more information about the N8Mail, N7Mail, and N8MailInit workloads.
N8Mail promises to be highly accurate and enables our heavier Lotus Notes and Domino V8 workloads to help our user model match your expectations more closely. Of course, these are still only benchmark tools. We believe these workloads, along with other data you obtain from sources such as your hardware vendors, can help better equip you to more accurately plan your current and future upgrades, purchases, and deployment decisions. A follow-on article uses the new workloads to compare Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.
- Get started with IBM Lotus Notes and Domino V8 technical content.
- Read the developerWorks article, "Extending the IBM Lotus Notes V8 mail with Eclipse."
- Read the developerWorks article, "Integrating IBM Lotus Notes data into the Lotus Notes V8 sidebar and toolbar."
- Read the developerWorks article, "Extending the IBM Lotus Notes V8 sidebar and toolbar."
- Read the developerWorks article, "Leveraging user context in the IBM Lotus Notes V8 sidebar and toolbar."
- Read the developerWorks article, "Using the Message Recall feature in IBM Lotus Notes and Domino V8."
- Read the developerWorks article, "The new IBM Lotus Notes 8 Out of Office functionality."
- Read the developerWorks article, "Engineering WS-I compliant Web services for IBM Lotus Domino V8."
- Read the developerWorks tip, "Enabling calendar ghosting in IBM Lotus Notes V8."
- Read the developerWorks tip, "Finding available time in IBM Lotus Notes V8 using the Scheduler UI."
- Refer to the developerWorks Lotus Composite Applications page.
- Read the "Lotus Notes and Domino 8 Reviewer's Guide."
- Read about the Eclipse project resources on developerWorks.
- Read more about IBM Lotus Notes and Domino V8.