IBM Lotus Notes V8 workloads: Taking performance to a new level

Learn about the IBM Lotus Notes/ Domino V8 enhancements to the NotesBench and Server.Load workloads. Three new workloads, N8Mail, N7Mail, and N8MailInit, demonstrate the typical tasks of a Lotus Notes user interacting with a Lotus Domino server.

Razeyah Stephen, Manager, IBM

Razeyah Stephen is manager of the IBM Lotus Domino and IBM Lotus Sametime performance team. She joined Iris Associates in October 1998.



Joseph Malek (jmalek@us.ibm.com), Software Developer, IBM

Joe Malek is a developer for the IBM Lotus Domino performance team. His current focus involves developing performance tools for the Lotus Domino performance team working on various platforms such as Microsoft Windows, Linux, IBM AIX, and Sun Solaris. Joe joined Iris Associates in October 2000.



Leon Desy (ldesy@us.ibm.com), Software Developer, IBM

Leon Desy is a developer for the IBM Domino performance team. His current focus involves developing performance tools for the Domino performance team working on various platforms such as Microsoft Windows, Linux, IBM AIX, and Sun Solaris. Prior to joining the team in August 2006, Leon worked in L3 support for Lotus Notes/Domino for eight years. He joined Iris Associates in August 1998.



18 September 2007

Also available in Chinese

One of the ways to achieve a close-to-real-world view of a Lotus Notes user is to closely mimic the API calls by the Lotus Notes V8 and Lotus Notes V7 clients. The N8Mail and N7Mail workloads used to reproduce those calls are completely new workloads with heavier transaction rates, which means that the data is different from your previous R6Mail results and that the results cannot be directly compared to each other. A future article will provide a detailed performance comparison of Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.

This article is intended for system administrators and designers interested in conducting performance measurements in their Lotus Notes/Domino V8 environments. This article assumes that you are an experienced Lotus Domino administrator.

NotesBench and Server.Load workloads

The NotesBench workloads are available only to members of the NotesBench Consortium. Server.Load, GUI version of the NotesBench workloads, is widely available to all customers. Server.Load is the Lotus Domino feature-based load generation tool that you can use to measure and characterize various Lotus Domino server capacity and response metrics. The workloads are available in both Server.Load and NotesBench for Lotus Notes and Domino V8. The workloads (also called tests or scripts) simulate the behavior of Lotus Domino client-to-server operations. The workloads allow you to use a common tool to apply a consistent, repeatable load against the Lotus Domino server so that you can assess the effects of various operating system, hardware, and configuration changes. For an overview of Server.Load, see the developerWorks Lotus article, "Introduction to Domino performance tuning." Server.Load is an installation option in the Lotus Domino Administrator client. Server.Load documentation is included in the Lotus Domino Administrator help.

The Lotus Domino Performance Team continues to add features and enhancements to the Server.Load utility that ships with the Lotus Domino Administrator client. With each major release of Lotus Notes and Lotus Domino, you've told the team how your use has changed and, therefore, how our workloads should change. In addition to this advice, we have paid close attention to the changes in the Lotus Notes V8 client to represent workload user actions accurately. This accuracy stems from the team making sure the workload's transactions with the Lotus Domino server matches that of the Lotus Notes V8 client. Also, with this release, we have incorporated into the workload setup components of the Lotus Domino environment that were not involved in previous workload versions. The new components consist of all default server tasks, mail journaling, transaction logging, Lotus Domino Domain Monitoring (DDM) probes, mail rules, and larger mail files. All of this enables the N8Mail workload to provide a closer-to-real world view of a Lotus Notes user interacting with a Lotus Domino server in a more typical environment. Throughout this process, we also created the N7Mail workload, which in essence performs the same user workload actions as N8Mail. Behind the scenes, though, the individual commands accurately reflect the Lotus Notes V7 client transactions to the Lotus Domino V7 server. Because of this approach, all documentation pertaining to N8Mail is accurate for N7Mail as well. Last, we bring you the N8MailInit workload. This workload populates mail databases for use with N8Mail and N7Mail under the new profiled user model described later in this article.

The users shown in tables 1-4 are only instances of the benchmark script running, and they do not necessarily correlate to the number of actual users deployed on a server. The goal of the improvements in the N8Mail and N7Mail workloads is to bring the user model closer to a typical production user. Because representing every customer usage scenario is difficult, we emphasize that you should contact the IBM TechLine and your hardware vendors for the best capacity planning and deployment advice for your configuration and environment.

Workloads to use with different versions of Lotus Domino

N8Mail workload can be used to benchmark Lotus Domino V8.0.

N7Mail workload can be used to benchmark Lotus Domino V6.0.x, Lotus Domino V6.5.x, Lotus Domino V7.0.x, and Lotus Domino V8.0.

R6Mail workload can be used to benchmark Lotus Domino V5.0.x, Lotus Domino V6.0.x, Lotus Domino V6.5.x, Lotus Domino V7.0.x, and Lotus Domino V8.0.

Defining a workload usage pattern

When we start the development cycle of a new major release of Lotus Domino, we review the current version of the workloads. Through discussions with Lotus Notes and Domino developers and customers, review of production data, assessment of industry trends, and input from members of the NotesBench Consortium and various Lotus Domino performance groups, we define new workloads that simulate how we expect users' usage patterns to behave approximately 24 months in the future.

What do the new workloads provide?

After all the research (and while keeping your comments and feedback in mind), we developed the new workloads for Lotus Notes and Domino V8. These workloads do the following:

  • Ensure accurate simulation of Lotus Notes V8 and Lotus Notes V7 transactions to the Lotus Domino server from the workloads.
  • Account for a more realistic Lotus Domino server test environment than was done previously.
  • Bring the profiled user model closer to an enterprise user by increasing the variety of features and usage patterns. Therefore, by using these workloads, you can get more realistic information (CPU, disk, memory, and network) for deployment planning than with prior workloads.

Workload specifications

N8Mail simulates full Lotus Notes V8 client users. The new features of the Lotus Notes and Domino V8 workloads include those shown in table 1.

Table 1. More realistic mail database profile for N8Mail and N7Mail
N8Mail/N7MailR6Mail
Mail database sizeApproximately 250 MB20 MB
Average document size100 KB 10 KB
Initial document count 3,000100

Messages generated

The frequency of creating/sending memos has increased. Fifty total documents are generated in an average six-hour run, one per iteration, and an additional one every other iteration. One iteration is approximately 15 minutes. Memo recipients now vary between one and three recipients. There are now fewer invitations and appointments, one each every six hours. Overall, you find 50 memos, invitations, and RSVPs to about 78 recipients, per user for a six-hour run. A new feature is the generation of replies to received mail, one every other iteration. See table 2 for more details about memo profiles.

Table 2. Memo profile for R6Mail, N8Mail, and N7Mail
Message body size in KB Attachment size in KB (if any)Percent
500N/A10
10,000N/A30
50,000N/A40
50,00050,00010
150,000N/A9.5
1,000 10,000,0000.5

This profile is the same in N8Mail and N7Mail as it was in R6Mail, so you can assume this distribution to be valid. To get the big picture of what has changed in the new workloads, review table 3, which shows data from the perspective of recipients of mail and the message distribution therein.

Table 3. Message distribution in N8Mail and N7Mail
Message size distribution (using the profile option)Percent of messages to total number of recipientsAttachment size (if any)
0 < size <= 1K 32.0% N/A
1K < size <= 10K3.6%N/A
10K < size <= 100K 57% 50 KB
100K < size <1M 6.8%N/A
1M < size <= 10M0.4% 10 MB

NOTE: The percentages do not match the percentages defined in the message profile discussed previously because the profile is used only for memos. It is not used for replies, invitations, or RSVPs, which are accounted for in this table.

N8Mail and N7Mail new user actions include the following:

  • Move random notes from the inbox to the NotesBench folder
  • Read the newest inbox items
  • Reply to a read message
  • Save all sent messages in the Sent folder
  • Conduct new mail polling every 15 minutes (that is, every iteration)

As you can see in table 4, there are now 35 more transactions overall (267 - 232) in N8Mail/N7Mail than there were in R6Mail for a difference of 1.45 more transactions per iteration per user.

Table 4. User actions: N8Mail, N7Mail, and R6Mail (six-hour period, 24 iterations)
Workload actions N8Mail and N7Mail workload action countR6Mail workload action count
Refresh inbox2424
Read message 120 (reads 5 per iteration)120 (reads 5 per iteration)
Reply to all12N/A
Send message to one recipient24N/A
Send message to three recipients124 with NthIteration set to 6
Create appointment14 with NthIteration set to 6
Send invitation14 with NthIteration set to 6
Send RSVP14 with NthIteration set to 6
Move to folder NotesBench24N/A
New mail poll24N/A
Delete two documents2424
Stamp documents (Release 6)N/A24
Add documents to the inbox (Release 6) N/A 24
Total transactions (per user in a six-hour period, 24 iterations) 267 232

NOTE: One iteration is approximately 15 minutes. Also, because N8Mail and N7Mail have identical sets of user actions, they are shown in one column, and the workload action counts are identical.

  • Send 3,000 mail notes (from distinct users) to populate the inbox
  • Create necessary profile documents with appropriate fields
  • Create a NotesBench folder
  • Generate five replies from what is newest in the inbox
  • Put ten invitations in the database
  • Optionally add the blocksender rule
  • Add ten users not in the range of valid workload users to the rule
  • Optionally add the color profile
  • Run Updall -R <maildb> to update the maildb

Server configuration

Test database setup for mail journaling

NOTE: For mail journaling to work correctly, distinct replication IDs are needed for the mail databases. After you create the mail databases, run this script across all to ensure this distinction.

The following three lines can be copied to a text file and run using Server.Load as a customized script. For client notes.ini settings, MailServer refers to the server on which the mail databases reside; nb_dbdir is the directory relative to the data directory where the mail databases can be found.

changeto [MailServer]!![nb_dbdir]mail[#].nsf
setreplid
dbclose

In addition to offering improved user actions, the N8Mail and N7Mail workloads use server features not used for R6Mail. Using these features brings the benchmark environment closer to your deployment environment. This increases the Lotus Domino server CPU utilization.

Server features include the following:

  • Transaction logging. Disk space is approximately 4 GB, Circular, Standard, or favor run time.
  • Mail journaling. Disk space is approximately 4 GB, and all messages are journaled.
  • DDM probes. All operating system and messages with default values (events4.nsf) are enabled.
  • Mail filtering. Each mail file has 10 users in the block mail list. The 10 users are not in the active user list in the workload run. The goal is measuring the overhead of having filtering active, not the overhead of doing the filtering.
  • Use default server tasks. ServerTask=Update,Replica,Router,AMgr,AdminP,CalConn,Sched,RnRMgr,LDAP (from the server's Notes.ini file)

See the Release Notes for Lotus Notes and Domino V8 for more information about the N8Mail, N7Mail, and N8MailInit workloads.


Conclusion

N8Mail promises to be highly accurate and enables our heavier Lotus Notes and Domino V8 workloads to help our user model match your expectations more closely. Of course, these are still only benchmark tools. We believe these workloads, along with other data you obtain from sources such as your hardware vendors, can help better equip you to more accurately plan your current and future upgrades, purchases, and deployment decisions. A follow-on article uses the new workloads to compare Lotus Domino V8.0 and Lotus Domino V7.0 for the Notes client.

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into IBM collaboration and social software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Lotus
ArticleID=255856
ArticleTitle=IBM Lotus Notes V8 workloads: Taking performance to a new level
publish-date=09182007