IBM Lotus Domino V8 server with the IBM Lotus Notes V8 client: Performance

Maintaining IBM Lotus Domino V8 mail server scalability with the IBM Lotus Notes V8 client is critical, and this article shows that Lotus Domino V8 not only equals, but outperforms the previous release. Let us show you how Lotus Domino V8 with the Lotus Notes V8 client performs across a variety of server platforms.

IBM Lotus Notes/Domino Performance Team, Software Performance Analysts, IBM

Members of the IBM Lotus Notes/Domino Performance Team who contributed to this article include Eric Bjorklund, Rich Buck, Wu W Huang, Angelo Lynn, James Powers, Andrew Nolet, and Nirmala Venkatraman.



18 January 2008 (First published 09 October 2007)

Also available in Chinese Russian

With IBM Lotus Notes/Domino V8, we have an all new, redesigned Lotus Notes client with increased functionality and a far superior user interface. It is critical that Lotus Domino mail server scale with this new client. In this article, we show that Lotus Domino V8 outperforms the previous release. We also show how Lotus Domino V8 with the Lotus Notes V8 client performs across the following server platforms:

  • IBM AIX
  • Linux
  • Sun Solaris
  • IBM System i
  • Microsoft Windows Server 2003
  • Linux on System z
  • IBM z/OS

We show tests with thousands of simulated Lotus Notes users running through common mail and calendar scenarios. Because the new client has changed some of the requests made of the Lotus Domino server, we created a new workload (N8Mail) that models transactions observed, while tracing the Lotus Notes V8 client as a real Lotus Notes V8 user performing the chosen tasks. Then we did the same thing for the Lotus Notes V7 client, and we created the N7Mail workload, which models the same user actions, but with the set of requests that a Lotus Notes V7 user would generate.

We also updated our testing methodology to more accurately reflect the environment in which our customers would deploy. In addition to the new workloads, we now have users start with 3,000 documents of various sizes in their mail files. We also enabled transaction logging, mail journaling, Domino Domain Monitoring (DDM) probes, and mail rules. Default server tasks ran during the tests. The net result was a much heavier workload than what we had with our previous R6Mail workload, so you should not compare the two workloads. In addition, we no longer show the maximum number of users possible, but rather focus on a typical deployment of about 4,000 users. For more details on the new workloads, see "IBM Lotus Notes V8 workloads: Taking performance to a new level."

The test suites that we ran for this article are designed to help you understand Lotus Domino V8's performance compared to that of Lotus Domino V7 and to help with your upgrade planning:

  • Consider the first test as the baseline. It is a Lotus Domino V7 server running with the Notes V7 mail database (using the mail7.ntf template) and exercised with the N7Mail workload, which simulates a Lotus Notes V7 client mail user.
  • The second test is a mixture, which represents a customer in a migration path. This test consists of a Lotus Domino V8 server running with mail7.ntf-based mail databases (at Lotus Domino V8 ODS 48 level, typical of databases created on a Lotus Domino V8 server), exercised with the N7Mail workload to simulate users who have not migrated to the new Lotus Domino V8 client yet.
  • The third test, a pure Lotus Domino V8 test, consists of a Lotus Domino V8 server running with Lotus Notes V8 mail databases (using the mail8.ntf template) and exercised with the N8Mail workload, which simulates a new Lotus Notes V8 client mail user.

For quick reference, table 1 lists the labels used on the charts in this article.

Table 1. Labels for performance test results
Test suiteWorkloadTemplateODSServer
Lotus Domino V7N7MailMail7.ntf43Lotus Domino V7.0
Lotus Domino V8/V7 templateN7MailMail7.ntf48Lotus Domino V8.0
Lotus Domino V8N8MailMail8.ntf48Lotus Domino V8.0

The Lotus Domino server Notes.ini values in the following sections represent the culmination of many tests and extensive statistical analysis. The use of these values is not recommended without analyzing the performance of a production Lotus Domino server. Be aware that the Notes.ini settings shown here for one platform may not work well on a different platform. For an explanation of the INI parameters and how to determine the optimum value range, refer to the appendix at the end of this article. Creation of databases based on the Lotus Domino V8 templates requires the Notes.ini setting Create_R8_Databases=1. This setting was used for all Lotus Domino V8 tests in this report.

NOTE: The results in this article were from benchmarks executed in a controlled environment. While some effort was made during the creation of the benchmark to include typical user operations, it is likely that real users make different use of Lotus Domino than the narrow range of functions tested by the benchmark. Therefore, you should use these numbers primarily to understand the relative performance of the Lotus Domino releases. These test results do not represent recommendations for real-world deployment. For assistance with capacity planning, we recommend that you consult your hardware vendor and the IBM TechLine, which handles capacity planning for new hardware purchases.

Although we show results on a variety of hardware platforms, these configurations are not of uniform capacity. It is our intent to focus on the performance of Lotus Domino, and this data should not be used to compare platforms against one another.

Now, let's see the results on the different server platforms.

IBM AIX results

Table 2 shows the details of the AIX configuration, and listing 1 details the Notes.ini settings for AIX.

Table 2. Summary of AIX configuration
Lotus Domino server configurationDetails
ModelP570 PowerPC_POWER5
CPUs for test/speed Two physical/two logical CPUs (SMT) at 1.5 GHz
Installed memory16 GB RAM
Active physical drives IBM FastT Fiber Channel storage, seven trays each with fourteen 15,000 RPM using RAID 0
Active logical volumes Three arrays for Lotus Domino binaries/data, one array for transaction logging
OSIBM AIX 5.3 ML5 64-bit
Listing 1. Notes.ini settings for AIX
ServerTasks=Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,LDAP
NLCACHE_SIZE=159383552
Server_Transinfo_range=22
NSF_Buffer_Pool_Size_MB=250
NSF_DBUCACHE_MAX_ENTRIES=6000
RouterDbCacheSize=6100
Server_Pool_Tasks=100
Server_Max_Concurrent_Trans=100
Server_Show_Performance=1
NSF_DbCache_MaxEntries=6000
SCHEDULE_NO_VALIDATE=1

The pSeries test machine used was a p570+ configured into multiple LPARs. (A logical partition, or LPAR, is the division of a computer's processors, memory, and hardware resources into multiple environments so that each environment can be operated independently with its own operating system and applications.) The LPAR used for these tests was configured as a single DPAR (Domino partition running an instance of the Lotus Domino server). We configured the LPAR to have two (capped) physical CPUs, and because we were running AIX 5.3, we used the SMT (Symetrical Multi Threading) feature that additionally enabled two logical CPUs.

The IBM FastT Fiber Attached Storage has been configured using RAID 0 and JFS2. Figure 1 shows the percent busy CPU for the AIX platform.

Figure 1. Percent CPU busy for AIX
Percent CPU busy for AIX

We see that the percent busy CPU tracks very closely for the three test cases from 1,000 to 4,000 virtual users.

Figure 2. Total disk I/O operations per second for AIX
Total disk I/O operations per second for AIX

Figure 2 shows the total disk I/O operations per second that were reported at steady state conditions. A simulated load was applied in increments of 1,000 simulated users by running:

  • The N7Mail workload with ODS 43 mail files created using mail7.ntf-based databases on a Lotus Domino V7.0 server
  • The N7Mail workload with ODS 48 mail files created using mail7.ntf-based databases on a Lotus Domino V8.0 server
  • The N8Mail workload with ODS 48 mail files created using mail8.ntf-based databases on a Lotus Domino V8.0 server

To measure the disk I/O operations per second at each simulated user interval, we used the operating system command, iostat –DR. This command was issued with a script every 20 minutes (approximately), and the resulting data then was averaged over the one-hour steady state time period. The commands (iostat –DR and/or iostat –d) should be analyzed to look for hot disks, file systems, and disk queue depths that are candidates to be tuned to provide better storage throughput.

NOTE: Disable read ahead/read caching for Lotus Domino data. Because Lotus Domino is not a relational database and does not write sequentially (transaction logging not withstanding), read-ahead performance enhancements available in SANs (Storage Area Networks) produce no increase in performance. Dedicate as much RAM as possible to the Write cache. This cache helps prevent slow Write requests.

Table 3. Resource usage at 4,000 users for AIX*
ResourceLotus Domino V7Lotus Domino V8Percent change
Mail templateMail7.ntfMail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy6159-3.3%
Total disk reads/sec734375-49%
Total disk writes/sec896619-31%
Shared memory used (MB)1,2491,49820%
Process memory used (MB)28.639.839%
Network bytes/sec2,426,5902,903,36220%

* NOTE: See the Addendum at the end of the article for corrected network bandwidth.

New for Lotus Domino V8, the default value for the NSF buffer pool is 512 MB on a machine with adequate memory installed (typically equal to or greater than 4 GB RAM), or the NSF Buffer_Pool_Size_MB value is explicitly set to a value less than or greater than the default value of 512. ConstrainedSHM should no longer be needed, nor do we recommend its use unless absolutely required.

In Lotus Domino V7.0.2 and IBM AIX 5LTM version 5.2 and later, when using the 64-bit kernel, we have extended the number of 256 MB memory segments available for shared memory from 11 to 13 (two additional segments) ONLY by enabling AIX_Very_Large_MM=1. Dynamic segment allocation allows the additional segments to be used by user heap or shared memory. Lotus Domino begins using the first two available segments for user heap memory (0x3 and 0x4). Lotus Domino begins using the last segment (0xF) for shared memory. As the memory needs of Lotus Domino increase, segments are allocated for use as heap or shared segments with heap growing upward from 0x3/0x4 and shared growing downward from 0xF. To see how many memory segments are being used by Lotus Domino and other applications, you can use either of the following:

ipcs –bm | grep <notes_id>
svmon -P


Linux results

This section covers the NRPC results of testing on an Intel system running 64-bit Linux version SuSE SLES 10 x86/64. Even though the operating system is 64-bit, the Lotus Domino versions used in the testing were all 32-bit. There are advantages to running a 32-bit application such as Lotus Domino on 64-bit Linux. One of these advantages is memory. On 32-bit Linux, a 32-bit application can get a maximum of 3 GB of memory to use, whereas on 64-bit Linux, a 32-bit application can get up to 4 GB of memory. Lotus Domino is very good at utilizing this memory increase for additional server tasks or applications.

Table 4 shows the details of the Linux configuration, and listing 2 details the Notes.ini settings for Linux.

Table 4. Summary of Linux configuration
Lotus Domino server configuration Details
PlatformIntel Xeon MP dual-core with hyperthread
CPUs for test/speedFour at 3400 MHz
Installed memory8 GB
Active physical drives99
Active logical volumes 14 RAID 0 arrays
OSSLES 10 x86-64
Listing 2. Notes.ini settings for Linux
Show_Server_Performance=1
platform_statistics_enabled=1
SCHEDULE_NO_VALIDATE=1
MEM_AddressableMemSizeMB=3500
ConstrainedSHMSizeMB=3000
MEM_EnablePreAlloc=1
EVENT_POOL_SIZE=41943040
NSF_DBcache_maxentries=6000
NSF_buffer_pool_size_MB=512
Server_Max_Concurrent_trans=200
server_pool_tasks=100
RouterDbCacheSize=6100
Servertasks=Replica,Router,Update,AMgr,Adminp,Sched,CalConn,RnRMgr,LDAP

The test server used for this set of tests uses four Intel Xeon MP dual-core CPUs with hyperthreading enabled. It has 8 GB of RAM and three DS4000 disk subsystems configured as 14 RAID 0 arrays. For the tests, the simulated mail user databases are spread evenly over the 14 arrays. A single drive on one of the DS4000s is used for the Lotus Domino transaction log files. The tests were run with the new N7Mail and N8Mail workloads.

Some Notes.ini parameters that are of special interest to this Linux environment are the following:

  • MEM_AddressableMemSizeMB=3500
  • ConstrainedSHMSizeMB=3000
  • MEM_EnablePreAlloc=1

These server Notes.ini parameters deal with the allocation of memory for the Lotus Domino server. As mentioned, a system running 64-bit Linux can allocate 4 GB of memory to a 32-bit application. The Notes.ini parameter MEM_AddressableMemSizeMB=3500 tells Lotus Domino that it has 3.5 GB of memory to use. This may sound like a contradiction because we stated that a 32-bit application can get 4 GB of memory. The operating system actually needs some of this 4 GB of memory for mapping and other memory pools for all the Lotus Domino tasks that it runs. Usually, 500 MB is ample for this.

The next Notes.ini parameter, ConstrainedSHMSizeMB=3000, tells Lotus Domino how much shared memory it can use. Because most of the memory Lotus Domino requires is shared memory, we give it 3 GB for these tests.

The third Notes.ini parameter, MEM_EnablePreAlloc=1, tells Lotus Domino to pre-allocate the shared memory defined in the Notes.ini parameter. This helps prevent Lotus Domino from crashing with an insufficient shared memory error, but it is a balancing act because locking down more shared memory than needed is not wise. Doing so can cause a shortage of local memory for applications to use and cause insufficient memory errors and application failure.

Figure 3 charts the percent CPU busy rate at different simulated user loads running the N7Mail and N8Mail workloads in the configurations described earlier. As you can see from this data, Lotus Domino V8 in both of its tests delivers a percent CPU savings over Lotus Domino V7.

Figure 3. Percent CPU busy for Linux
Percent CPU busy for Linux

Figure 4 shows the total read/write operations per second at different simulated user loads running the N7Mail and N8Mail workloads. It shows that there is a reduction in I/O operations using Lotus Domino V8 with both the mail7.ntf-based databases and the mail8.ntf-based databases. It also shows that the best results over Lotus Domino V7 are with Lotus Domino V8 using the mail8.ntf-based databases.

Figure 4. Total disk I/O operations per second for Linux
Total disk I/O operations per second for Linux

Table 5 shows the resource usage at steady state for 4,000 simulated users. The comparison is between Lotus Domino V7 using mail7.ntf -based databases running the N7Mail workload, which simulates Notes 7 mail users, and Lotus Domino V8 using mail8.ntf-based databases running the N8Mail workload, which simulates new Lotus Notes V8 mail users.

The CPU percent busy reflects the information that is displayed in figure 3. It shows that with Lotus Domino V8, we can attain a CPU savings over Lotus Domino V7.

One of the important features in Lotus Domino V8 is a reduction in I/O activity. Figure 4 gives you a pictorial representation of this I/O savings, whereas table 5 shows you where the savings are and an example of what you can expect. The field for shared memory used in the table shows about 3 GBs of memory for both tests. This is due to the pre-allocation of shared memory that was done for each of the tests and explained previously. The heavier network bytes/second seen with the Lotus Domino V8 test reflects the heavier workload activity that the N8Mail workload does to exercise the extra functionality of the Lotus Notes V8 client.

Table 5. Resource usage at 4,000 users for Linux
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail template Mail7.ntf Mail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy11.069.09-17.8%
Total disk reads/sec927465 -49.8%
Total disk writes/sec900744-17.3%
Shared memory used (MB) 2,9532,9530%
Network bytes/sec2,763,3973,309.59816%

The data presented from these tests should help you plan for upgrades to Lotus Domino V8 on a Linux platform. It should give you some insight into the percent CPU savings and I/O savings that are possible with Lotus Domino V8.


Sun Solaris results

Table 6 shows the configuration used with Sun Solaris on an eight CPU Sun server. Solaris allows applications to use nearly the entire 4 GB address space, and for this environment, Lotus Domino was configured to take advantage of the large amount of memory available. Listing 3 details the Notes.ini settings for Solaris.

Table 6. Summary of Solaris configuration
Lotus Domino server configuration Details
ModelSun 6800
CPUs for test/speedEight CPUs at 1050 MHz
Installed memory32 GB
Active physical drives63
Active logical volumes Seven T3 arrays configured as RAID 0
OSSolaris 10
Listing 3. Notes.ini settings for Solaris
server_pool_tasks=100
server_max_concurrent_trans=100
NSF_Buffer_Pool_Size_MB=1024
NLCACHE_SIZE=159383552
EVENT_CORRELATION_POOL_SIZE=20971520
RouterDbCacheSize=5100
nsf_dbcache_maxentries=5100
log_mailrouting=10
SCHEDULE_NO_VALIDATE=1
MEM_EnablePreAlloc=1
DEBUG_ENABLE_SYS_V_SHM=1
ConstrainedSHMSizeMB=3300
ServerTasks=Update,Replica,Router,AMgr,AdminP,CalConn,Sched,RnRMgr,LDAP

The server used for Solaris testing is an eight CPU Lotus domain with six arrays to hold the Lotus Domino data and executable files. In addition, two file systems on a seventh array were used to hold the transaction log and the journal files. Solaris large page support was enabled, and 3300 mes of shared memory was pre-allocated to Lotus Domino.

Figure 5 shows the CPU saving on Lotus Domino V8 for both migrating Lotus Notes V7 users and Lotus Notes V8 users. Lotus Domino V7 and Lotus Domino V8/V7 template measurements used the N7Mail workload, which mimics the behavior of Lotus Notes V7. The Lotus Domino V8 measurement used the N8Mail workload, which models the redesigned Lotus Notes V8 client. The CPU utilization numbers are as reported by Domino platform statistics.

Figure 5. Percent CPU busy for Solaris
Figure 5. Percent CPU busy for Solaris

We also see reductions in the total number of disk I/O operations with Lotus Notes/Domino V8 as shown in figure 6, though in this case. there is an increase for Lotus Notes V7 users in the migration process. The I/O statistics here are calculated from the Solaris iostat command and are the sum of the reads and writes for the six data volumes plus the transaction log and journal file systems.

Figure 6. Total disk I/O operations per second for Solaris
Total disk I/O operations per second for Solaris

Table 7 reports the resource usage at 4,000 users.

Table 7. Resource usage at 4,000 users for Solaris
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail templateMail7.ntf Mail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy39.935.1-12.0%
Total disk reads/sec158114-27.8%
Total disk writes/sec894735-17.8%
Memory used (MB)3,1853,1940.3%
Network bytes/sec2,190,7602,601,33118.7%

At the 4,000-user level, Lotus Domino V8 outperforms Lotus Domino V7 in CPU and disk I/O, but there is an increase in the network utilization. Network and memory utilization come from the Solaris netstat and prstat commands. Because we pre-allocate shared memory for Lotus Domino, the difference between the two measurements is minimal.


IBM System i results

Table 8 shows the details of the hardware and software used for testing Lotus Domino on IBM System i, and listing 4 details the Notes.ini settings.

Table 8. Summary of System i configuration
Lotus Domino server configuration Details
ModelSystem i 520
CPUs for test/speedTwo CPUs at 1.9 GHz
Installed memory32 GB
Disk drives98
OSi5/OS V5R4M0
Listing 4. Notes.ini settings for System i
RouterDbCacheSize=6100
NSF_Buffer_Pool_Size_MB=300
NSF_DBCache_MaxEntries=6100
Server_Pool_Tasks=100
Server_Max_Concurrent_Trans=-1
Schedule_No_Validate=1

The test environment used a System i Model 520 with two processors, 32 GB of memory, and 98 disk drives with RAID-5 protection. The system was configured with one Lotus Domino partition. Lotus Domino V7 and Lotus Domino V8 were both tested with the N7Mail workload using the Lotus Domino V7 mail template. Lotus Domino V8 was also tested with the N8Mail workload using the Lotus Domino V8 mail template. Each test was run with 1,000, 2,000, 3,000, and 4,000 simulated users, and performance information was collected at each data point. See figure 7.

Figure 7. Percent CPU busy for System i
Percent CPU busy for System i

Lotus Domino V8 reduced the CPU utilization of the system at each data point with both the N7Mail and N8Mail workloads. This indicates that both Lotus Notes V7 and Lotus Notes V8 clients benefit from the CPU utilization improvements in Lotus Domino V8 as shown in figure 8.

Figure 8. Total disk I/O operations per second for System i
Total disk I/O operations per second for System i

Lotus Domino V8 also reduced the total disk I/O operations of the system at each data point with both the N7Mail and N8Mail workloads. This indicates that both Lotus Notes V7 and Lotus Notes V8 clients benefit from the disk I/O improvements in Lotus Domino V8. Table 9 reports the resource usage at 4,000 users.

Table 9. Resource usage at 4,000 users for System i
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail templateMail7.ntf Mail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy42.737.3-13%
Total disk reads/sec28741043%
Total disk writes/sec829578-30%
Base pool pages/sec26237142%
Network bytes/sec2,404,3522,979,840 24%

Lotus Domino V8 reduced the CPU utilization of the system at 4,000 users. The page faulting rate and number of disk reads were higher with Lotus Domino V8, but the number of disk writes was lower. Because most of the disk I/O operations with Lotus Domino V7 were writes, the total number of disk I/O operations was lower with Lotus Domino V8. There was an increase in network utilization with Lotus Domino V8.

Lotus Domino V8 reduced both CPU utilization and total disk I/O operations on a typical System i configuration for both Lotus Notes V7 and Lotus Notes V8 clients.


Microsoft Windows Server 2003 results

Table 10 shows the hardware and software configuration for the Microsoft Windows tests, and listing 5 details the Notes.ini settings.

Table 10. Summary of Microsoft Windows Server 2003 configuration
Lotus Domino server configuration Details
ModelIBM x3850-[8863MC1]
CPUs for test/speedEight CPUs at 3.6 GHz
Installed memory8 GB
Active physical drives42 disks
Active logical volumesSix arrays of RAID 0
OSMicrosoft Windows Server 2003 Enterprise x64
Listing 5. Notes.ini settings for Microsoft Windows Server 2003
platform_statistics_enabled=1
server_pool_tasks=80
server_max_concurrent_trans=100
Show_Server_Performance=1
RouterDbCacheSize=6100
NSF_DBcache_maxentries=6000
server_pool_tasks=80
SCHEDULE_NO_VALIDATE=1
CONSOLE_LOG_ENABLED=1
NSF_Buffer_Pool_Size_MB=250

Lotus Domino V8 was set up as a single partition server on an eServer xSeries 3850 running Microsoft Windows Server 2003 Enterprise x64 edition with eight processors and 8 GB of memory recognized by the Microsoft Windows operating system. The Lotus Domino executable files were installed on one IBM FAStT 600 (200 GB, RAID 0). The mail databases were spread across three IBM FAStT 600 arrays of 800 GB each, also RAID 0. Transaction logging was set up on a separate three-disk array of 200 GB, RAID 0. Mail journaling was set up on a separate disk array. The Windows Page File was placed on a separate single disk array. Network access occurred through a single 1 GB Ethernet adapter running in full-duplex mode.

As was also shown on other platforms, Lotus Domino V8 running on Microsoft Windows Server 2003 Enterprise x64 offers improvement in CPU utilization and I/O performance as shown in figure 9. We scaled results to 4,000 users to represent a customer environment.

Figure 9. Percent CPU busy for Microsoft Windows Server 2003
Percent CPU busy for Microsoft Windows Server 2003

As shown in figure 10, Lotus Domino V8 shows significant reduction in I/O utilization over Lotus Domino V7 at several virtual user load levels. A decrease in total I/O utilization is a significant improvement for Lotus Domino V8. A simulated load was applied in increments of 1,000 simulated users by running the following:

  • The N7Mail workload with ODS 43 mail files created using the mail7.ntf template on a Lotus Domino V7 server
  • The N7Mail workload with ODS 48 mail files created using the mail7.ntf template on a Lotus Domino V8 server
  • The N8Mail workload with ODS 48 mail files created using the mail8.ntf template on a Lotus Domino V8 server
Figure 10. Total disk I/O operations per second for Microsoft Windows Server 2003
Total disk I/O operations per second for Microsoft Windows Server 2003

The statistics in table 11 were obtained by Lotus Domino platform stats along with Microsoft Windows Performance Monitor. You can use the following Performance Monitor counters to track the performance of disk I/O, memory, and CPU percent, which accurately reflect the usage of the Lotus Domino server on the Microsoft Windows operating system:

  • Logical disk - total disk read/sec
  • Logical disk - total disk write/sec
  • Process - private bytes
  • Process - virtual bytes
  • Processor - percent processor time
Table 11. Resource usage at 4,000 users for Microsoft Windows Server 2003*
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail templateMail7.ntf Mail8.ntfN/A
WorkloadN7Mail N8MailN/A
CPU percent busy19.218.1-5.7%
Total disk reads/sec690292-57.6%
Total disk writes/sec1512617-59.1%
Shared memory used (MB) 1,5031,70315.1%
Process memory used (MB)131113-13.5%
Network bytes/sec 2,632,1803,071,60216.7%

* NOTE: See the Addendum at the end of the article for corrected network bandwidth.

This test, and previous tests, confirms that Lotus Domino V8 improves on the performance and total cost of ownership (TCO) when compared to Lotus Domino V7. Upgrading to Lotus Domino V8 and the Notes V8 mail template showed a reduction of I/O usage in our tests with a decrease of more than 50% total disks reads and writes.


Linux on System z results

We saw great performance results for Lotus Domino V8 on Linux on System z with both I/O and CPU improvements. Table 12 summarizes the configuration for Linux on System z, and listing 6 details the Notes.ini settings.

All performance test results documented here come from one logical partition (LPAR) on a series z9 model 2094-S54. This LPAR has six dedicated processors, but we enabled two processors for this test because we wanted the CPU utilization higher for comparison purposes. The LPAR was configured as a single DPAR (Domino partition running an instance of the Lotus Domino server). This machine booted SLES 10 with some updates. The performance test LPAR was configured with 12 GB memory. We used a single Gigabit Ethernet Open System Architecture (OSA) card. Our LAN is isolated to avoid other network traffic interference from unrelated activities. All disks were allocated from an Enterprise Storage Server (DS8300) array with each disk configured as a 3390 model 9. Client mail databases were distributed evenly over 57 LVM file systems, each allocated across four volumes in a single LVM, providing 28 GB of usable space per file system. Separate volumes were allocated for the Lotus Domino execution, Notes data, Domino Directory, mailbox 1 to 4, and transaction logging. Another four volumes were used in an LVM for mail journaling. EXT3 file system was used on Linux for System z.

Table 12. Summary of Linux on System z configuration
Lotus Domino server configuration Details
Modelz9 2094–S54
CPUs for test/speedTwo dedicated processors, LPAR
Installed memory12 GB
DASD typeDS8300
Active logical volumes57x4 LVM mail databases, 1x4 LVM mail journaling, and seven volumes for notesdata, names, mailbox, and translog
OSSUSE SLES 10 with some updates
Listing 6. Notes.ini settings for Linux on System z
Platform_statistics_enabled=1
Server_pool_tasks=100
RouterDbCacheSize=6100
NSF_DBcache_maxentries=6000
NSF_Buffer_Pool_Size_MB=386
NLCACHE_SIZE=104857600
TRANSLOG_Performance=1
TRANSLOG_Path=/translog1
TRANSLOG_Status=1
SCHEDULE_NO_VALIDATE=1

Figure 11 shows the CPU percent from Lotus Domino V7 with the Notes V7 mail template run with the N7Mail workload, Lotus Domino V8 with the Notes V7 mail template run with the N7Mail workload, and Lotus Domino V8 with the Notes V8 mail template run with the N8Mail workload. The CPU percent showed the average of the one-hour steady state of each 1,000-user increment. Figure 11 shows a range of CPU improvement from 26% to 34% running Lotus Domino V7 with the Notes V7 mail template and Lotus Domino V8 with the Notes V8 mail template. Figure 11 also shows the similar CPU improvement from 23% to 31% for Lotus Domino V8.0 with the Notes V7 mail template compared with Lotus Domino V7 with the same template.

Figure 11. Percent CPU busy for Linux on System z
Percent CPU busy for Linux on System z

Figure 12 shows the total I/O operations improvement from Lotus Domino V7 to Lotus Domino V8. This chart shows a range of I/O improvement from 6% to 19% with the Notes V7 mail template in Lotus Domino V7 and Lotus Domino V8, and a range of 25% to 34% I/O improvement from Lotus Domino V7 with the Notes V7 mail template and Lotus Domino V8 with the Notes V8 mail template.

Figure 12. Total disk I/O operations per second for Linux on System z
Total disk I/O operations per second for Linux on System z

Figure 12 shows that I/O read improved 26% and I/O write improved 36% for Lotus Domino V8 with the Notes V8 mail template. Total share memory increased 11%, and total process memory increased 24%. Also, total network bytes received and transmitted increased 22%.

Table 13. Resource usage at 4,000 users for Linux on System z
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail templateMail7.ntf Mail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy69.751.3-26%
Total disk reads/sec919585-36%
Total disk writes/sec951876-7.8%
Shared memory used (MB) 9601,07011.4%
Process memory used (MB)15319124.8%
Network bytes/sec2,100,2212,580,28422.8%

Lab measurements indicated that CPU improved as much as 34% for the new Lotus Notes mail workload and that I/O improved as much as 34% for the N7Mail workload for Lotus Domino V7 over the N8Mail workload for Lotus Domino V8 on Linux on System z. With the new features for Lotus Domino V8 and the mail8.ntf template, lab results showed some memory degradation, such as shared memory increased up to 11%, process memory degraded up to 23%, and network total bytes per second increased up to 22%. The CPU and the I/O reductions, though, translate into a great win for Lotus Domino V8.


IBM z/OS results

As another great achievement for Lotus Domino V8 on IBM z/OS, lab measurements indicated fairly large CPU and I/O reductions from Lotus Domino V8 compared with Lotus Domino V7.

All performance test results documented here come from one logical partition (LPAR) on a series z9 model 2094-S54. The LPAR was configured as a single DPAR. This LPAR had six dedicated processors, but we enabled two processors for this test because we wanted the CPU utilization to be higher for comparison. This machine ran z/OS version 1.7. The performance test LPAR was configured with 12 GB memory. We used a single Gigabit Ethernet Open System Architecture (OSA) card. Our LAN was isolated to avoid other network traffic interference from unrelated activities. All disks were allocated from an Enterprise Storage Server (DS8300) array with each disk configured as 3390 model 9. Client mail databases were distributed evenly over 57 z/FS file systems, each allocated across four span volumes providing 28 GB of usable space per file system. Separate z/FS file systems were allocated on a single volume (disk) for the Lotus Domino execution, Domino/Notes data, Domino Directory, mailbox 1 to 4, and transaction logging, and another four span volumes z/FS were allocated for mail journaling.

Table 14 summarizes the z/OS configuration, and listing 7 details the Notes.ini settings.

Table 14. Summary of z/OS configuration
Lotus Domino server configuration Details
Modelz9 2094–S54
CPUs for test/speedTwo dedicated processors, LPAR
Installed memory12 GB
DASD typeDS8300
Active logical volumes57x4 z/FS mail databasess, 1x4 zFS mail journaling, and seven zFS volumes for Notes data, names, four mailboxes, and translog
OSz/OS Version 1.7
Listing 7. Notes.ini settings for z/OS
Platform_statistics_enabled=1
Server_pool_tasks=100
RouterDbCacheSize=6100
NSF_DBcache_maxentries=6000
NSF_Buffer_Pool_Size_MB=196
NLCACHE_SIZE=83886080
TRANSLOG_Performance=1
TRANSLOG_Path=/translog1
TRANSLOG_Status=1 
SCHEDULE_NO_VALIDATE=1

Figure 13 shows the CPU for Lotus Domino V7 with the mail7.ntf template run with the N7Mail workload in the green line, Lotus Domino V8 with the mail7.ntf template run with the N7Mail workload in the blue line, and Lotus Domino V8 with the mail8.ntf template run with the N8Mail workload in the red line. The CPU percent showed the average of the one-hour steady state of each 1,000-user increment. This chart shows a range of CPU improvements from 9% to 16% for Lotus Domino V8 with the mail7.ntf template. It also shows 18% to 22% CPU improvement for Lotus Domino V8 with the mail8.ntf template compared to Lotus Domino V7 with the mail7.ntf template.

Figure 13. Percent CPU busy for z/OS
Percent CPU busy for z/OS

Figure 14 shows the total I/O operations improvement for Lotus Domino V8 compared to Lotus Domino V7. Figure 14 also shows a range of I/O improvement from 20% to 22% for Lotus Domino V8 with the mail7.ntf template, and a range of 34% to 40% I/O improvement for Lotus Domino V8 with the mail8.ntf template.

Figure 14. Total disk I/O rate per second for z/OS
Total disk I/O rate per second for z/OS

Table 15 shows I/O rate improved 34% and I/O intensity (where I/O intensity equals I/O rate times I/O response time) improved 52% for Lotus Domino V8 with the mail8.ntf template. Total shared handles increased 15.9%, and shared memory increased 13% for Lotus Domino V8. Total shared handles improved 30%, and total process memory improved by 25% for Lotus Domino V8. Note that the last two metrics have very small values in megabytes compared with the total shared handles and the shared memory metrics. Also, total network bytes received and transmitted increased 25%.

Table 15. Resource usage at 4,000 users for z/OS
Resource Lotus Domino V7Lotus Domino V8Percent change
Mail template Mail7.ntf Mail8.ntfN/A
WorkloadN7MailN8MailN/A
CPU percent busy62.751.0-18.6%
Total disk rate17,92311,730-34.5%
Total disk intensity49,05423,113-52.0%
Shared handles used (MB)39846115.9%
Process handles used (MB)4.32.9-30.0%
Shared memory used (MB)79290213.8%
Process memory used (MB)4332-25.8%
Network bytes received and sent (data from SMF108 NWBR and NWBS)2,128,9352,675,56225.7%

Our lab data showed CPU reductions up to 22% and I/O reductions up to 40% for Lotus Domino V8 on z/OS. With the new features for Lotus Domino V8 and the mail8.ntf template, some memory increased, such as shared memory increased up to 13% and shared handles increased up to 16%. Also, total network bytes increased up to 26%. Overall, the CPU and the I/O reductions are the great win for Lotus Domino V8 on z/OS.


Addendum

The network bandwidth increase, for Lotus Domino V8 when compared with Lotus Domino V7, reported in the developerWorks Lotus article, "Lotus Domino V8 server performance with the Notes Client," occurred for two reasons.

  • A Lotus NotesBench N8 Client workload simulation bug. The N8Mail workload in NotesBench 8.0 Gold version had two issues. Issue #1 was that the workload did not set the correct Since Time for the Modified Note Info request calls during View Refresh transaction in the workload. Issue #2 was that the workload simulated a full Note Open instead of summary only Note Open call during the delete transaction in the workload.

    These two NotesBench bugs in the N8Mail workload caused the server to return more data to the NotesBench clients than the Lotus Notes 8 client would retrieve for the above two transactions in the N8Mail workload.
  • The Lotus Notes V8 client retrieval of more documents in a view for a given screen size, than the :Lotus Notes V7 client retrieval. This was a design consideration in the Lotus Notes V8 client to cache more documents for a better user experience during page scrolling . The :Lotus Notes client uses the NIFReadEntries API call to fetch the entries in a Lotus Notes View . The Lotus Notes V8 client retrieves 50 entries in the Inbox , while the Lotus Notes V7 client retrieves only 42 entries in the Inbox.

For Lotus Domino V8.0.1, the NotesBench Notes V8 client workload simulation bugs were fixed. The number of readentries in the Lotus Notes V8 client does was not changed.

Tables 16 and 17 illustrate the effect of the NotesBench workload change to the Network Bandwidth calculation. Note that the remaining increase is a result of the new to Lotus Notes V8 readentries increase. Other metrics such as CPU and disk I/O did not change with the workload change.

Table 16. Network bandwidth resource usage at 4000 users for AIX
Network bandwidth resource Lotus Domino V7Lotus Domino V8% Change
Mail template Mail7.ntfMail8.ntfN/A
Workload N7MailN8Mail N/A
Network Kbytes/sec (with old N8Mail workload)2,4272,90320%
Network Kbytes/sec (with new N8Mail workload)2,3862,5507%
Table 17. Network bandwidth resource usage at 4000 users for Microsoft Windows
Network bandwidth resource Lotus Domino V7Lotus Domino V8% Change
Mail template Mail7.ntfMail8.ntfN/A
Workload N7MailN8Mail N/A
Network Kbytes/sec (with old N8Mail workload)2,6323,07217%
Network Kbytes/sec (with new N8Mail workload)2,540 2,7689%

Conclusion

As the measurements in this report indicate, moving to Lotus Domino V8 and the new Lotus Notes V8 client should be a very positive experience. We see CPU reductions across the board as well as I/O reductions on most platforms when compared to Lotus Notes/Domino V7 performance. Even with the more functional Lotus Notes V8 client, we see CPU and I/O savings on the Lotus Domino V8 server. Therefore, the total cost of ownership for the Lotus Domino server infrastructure does not need to be increased to support the new Lotus Notes V8 client. With the new client interactions, we do see some increase in network utilization. The heavier network utilization seen with the Lotus Domino V8 test reflects the heavier workload activity that the N8Mail workload does to exercise the extra functionality of the Lotus Notes V8 client. Also, on some platforms, memory utilization has increased, so installations that are sensitive to those parameters should plan accordingly.

Appendix 1. Notes.ini settings
Notes.ini parameterDescriptions
ConstrainedSHMSizeMBSet it to restrict shared memory to the size in megabytes specified.
Create_R8_DatabasesApplies to Lotus Domino V8 servers. Set it to 1 to create new ODS48 databases on Lotus Domino V8 servers.
DEBUG_ENABLE_SYS_V_SHMSet it to 1 to let Lotus Domino use System V for shared memory instead of mmap files on UNIX platforms.
DEBUG_NSF_SHOW_ALLSTATSNew in Lotus Domino V8. Set it to 1 to enable Domino I/O stat collection for database metadata, views, and objects.
EVENT_CORRELATION_POOL_SIZESpecifies the size of event correlation pool used by event task and DDM.
EVENT_POOL_SIZESet this variable to specify the amount of memory used for unprocessed events, unprocessed notifications, and event suppression. The default setting is 5242880 (5 MB).
log_mailroutingSpecifies the level of logging of router events performed by the router process.
MEM_AddressableMemSizeMBSet to override our default setting for the amount of memory Domino will see as "addressable" memory (where VALUE is in megabytes, and the default varies by platform).
MEM_EnablePreAllocSet it to 1 to enable pre-allocation of shared memory.
NLCACHE_SIZESpecifies the size of Namelookup Cache in bytes. The default value is 16 MB.
NSF_Buffer_Pool_Size_MBSpecifies the maximum size (in bytes) of the NSF buffer pool, a section of memory dedicated to buffering I/O transfers between Lotus Domino and disk storage.
NSF_Dbcache_MaxentriesSpecifies the number of databases that a server can hold in its database cache at one time.
RouterDbCacheSizeSpecifies the size of Router Database cache used to cache mail databases in the router process.
SCHEDULE_NO_VALIDATEDetermines whether or not SchedMgr validates its busytime database entry on a daily basis as follows:
0 (default) - Enables validation
1 - Disables validation
Server_MAX_CONCURRENT_TRANSSets the limit for the number of concurrently scheduled transactions on a server.
Server_Pool_TasksTotal number of physical threads in the Domino server (DbServer) threadpool.
Server_Show_PerformanceApplies to servers. Specifies whether or not server performance events are displayed on the console. Set this variable to 1 to display server performance events on console.
ServerTasksSpecifies the tasks that begin automatically at server startup and continue until the server is shut down.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into IBM collaboration and social software on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Lotus
ArticleID=260105
ArticleTitle=IBM Lotus Domino V8 server with the IBM Lotus Notes V8 client: Performance
publish-date=01182008