IBM Lotus Domino 8.0.1 for 64-bit server performance
With IBM Lotus Domino 8.0.1, Lotus Domino has added a 64-bit native support option for Microsoft® Windows® 2003 and IBM AIX® platforms. With 64-bit support, Lotus Domino can address more memory on the server than is possible in previous versions. Maintaining Lotus Domino mail server performance with the new 64-bit platform was important.
In this article, we show how the 64-bit Lotus Domino 8.0.1 server performs against 32-bit Lotus Domino 8.0.1 using the N8Mail workload, which simulates a Lotus Notes® 8 client mail user. For more details on the N8Mail workloads, refer to the developerWorks® article, "IBM Lotus Notes 8 workloads: Taking performance to a new level."
We updated our testing methodology to more accurately reflect the environment in which our customers deploy. We now have users start with 3,000 documents of various sizes in their mail files. We also enabled transaction logging, mail journaling, Domino domain monitoring (DDM) probes, and mail rules. Default server tasks ran during the tests. The net result was a much heavier workload than in the previous R6Mail workload, so do not compare the two workloads. In addition, we no longer show the maximum number of users possible, but rather focus on a typical deployment of about 4,000 users.
The test suites that we ran for this article are designed to help you understand 64-bit Lotus Domino performance compared to that of 32-bit Lotus Domino on the same hardware configuration. The first test could be considered the baseline. It is a 32-bit Lotus Domino 8.0.1 server running with mail8.ntf-based mail databases, at Lotus Domino 8 ODS 48 level and exercised with the N8Mail workload. The second test is a 64-bit Lotus Domino 8.0.1 server running with mail8.ntf-based mail databases, at Lotus Domino 8 ODS 48 level and exercised with the N8Mail workload.
For quick reference, the labels on the following charts are listed in table 1.
Table 1. Labels for performance testing
|Lotus Domino 8.0.1, 32-bit||N8Mail||Mail8.ntf||48||Lotus Domino 8.0.1|
|Lotus Domino 8.0.1, 64-bit||N8Mail||Mail8.ntf||48||Lotus Domino 8.0.1|
The Lotus Domino server Notes.ini values in the following sections represent the culmination of many tests and extensive statistical analysis. The use of these values is not recommended without analyzing the performance of a production Lotus Domino server. Also, be aware that the Notes.ini settings shown here for one platform might not work well on a different platform. For an explanation of what each of the ini parameters is used for and how to determine the optimum value range, refer to the Appendix.
NOTE: The results in this article come from benchmarks executed in a controlled environment. Although some effort was made during the creation of the benchmark to include typical user operations, it is likely that real users make different use of Lotus Domino than the narrow range of function that is tested by the benchmark. Therefore, these numbers should be used primarily to understand the relative performance of the Lotus Domino releases, and they do not represent recommendations for real-world deployment. For assistance with capacity planning, consult your hardware vendor.
Also, although we show results on a variety of hardware platforms, these configurations are not of uniform capacity. It is our intent to focus on the performance of Lotus Domino itself, and this data should not be used to compare hardware platforms against each other.
Microsoft Windows 2003 server results
The following tables show the hardware and software configurations for our Microsoft Windows tests.
Table 2. Hardware configuration for Microsoft Windows tests
|Model||Intel x64-based PC|
|Processors for test/speed||Eight processors @ 3.33GHz|
|Installed memory||8 GB|
|Active physical drives||35 disks|
|Active logical volumes||Five arrays using RAID 0|
|Operating system||Microsoft Windows 2003 Server Enterprise x64 SP2|
Table 3. Software configuration for Microsoft Windows tests
Lotus Domino server was set up as a single-partition server on an Intel x64-based system running Microsoft Windows 2003 64-bit Enterprise Server with eight processors, and with 8 GB of memory recognized by the Microsoft Windows operating system. The Lotus Domino data files, mail journal, and mailboxes were installed on one IBM FAStT 600 array of 475 GB and that used RAID 0. The mail databases were spread across three IBM FAStT 600 arrays of 475 GB and that used RAID 0. Transaction logging files were in a separate hard disk drive. The system page file, configured for 8000 MB, was located on a small three-disk RAID volume in the IBM FAStT 600 array. Network access occurred through a single 1 GB Ethernet adapter running in full duplex mode. Figures 1-3 show the resource utilization charts for the two measurements at various user loads.
Figure 1. Processor utilization
Figure 1 shows almost the same processor utilization for both 32-bit and 64-bit Lotus Domino at all the user loads from 1,000 to 5,000 simulated users.
Figure 2. Disk I/O per second
Figure 2 shows about the same I/O per second for both 32-bit and 64-bit Lotus Domino at all the user loads from 1,000 to 5,000 simulated users.
Figure 3. nServer virtual memory (MB)
Both 32-bit and 64-bit Lotus Domino servers explicitly set the LargeSystemCache in Microsoft Windows 2003 server to 2.4 GB, using the cacheset.exe tool provided with Lotus Domino. In figure 3, 64-bit Lotus Domino server process virtual memory was over 2 GB even at a load of 3,000 virtual users. The 32-bit Lotus Domino server reached close to 2 GB virtual memory at 5,000 users. At each load point, roughly 400 MB more virtual bytes are consumed by the 64-bit Lotus Domino server as compared to the 32-bit Lotus Domino server. The increase in virtual memory of the nserver.exe process is primarily attributed to the process private memory increase, not to the memory that is shared across all the Lotus Domino processes. We also observed that there was not a substantial difference in available physical memory between Lotus Domino 32-bit and Lotus Domino 64-bit servers at the same number of users.
Table 4. Resource usage at 4,000 users
|Server||Lotus Domino 8.0.1, 32-bit||Lotus Domino 8.0.1, 64-bit||% change|
|Mail database ODS level||48||48||NA|
|Processor % busy||18.1||17.5||-3.3%|
|Total disk reads/second||216||206||-4.6%|
|Total disk writes/second||586||587||0%|
|nServer virtual memory used (MB)||1670||2128||+27.4%|
|Physical memory available (MB)||2850||2764||-3%|
Table 4 compares several key resource utilization metrics on the Lotus Domino server at a virtual user load of 4,000 for each of the tests. This table clearly shows that there is no substantial change in processor utilization, disk I/O metrics, and network bytes by upgrading from 32-bit Lotus Domino.
AIX server results
Tables 5 and 6 show the hardware and software configurations for our Microsoft Windows tests for AIX server.
Table 5. Hardware configuration for AIX server tests
|Processors for test/speed||Two physical/two logical processors (SMT) 1.5GHz|
|Installed memory||16 GB RAM|
|Active physical drives||IBM FAStT Fiber Channel storage, seven trays each with 14 15,000 RPM disks, configured as four logical volumes using RAID 0|
|Active logical volumes||Three arrays for Lotus Domino binaries/data, one array for transaction logging|
|Operating system||AIX 5.3 ML5 64-bit|
Table 6. Software configuration for AIX server tests
The AIX test server was a p570+ configured into multiple LPARS. (A logical partition, or LPAR, is the division of a computer's processors, memory, and hardware resources into multiple environments so that each environment can be operated independently with its own operating system and applications.) The LPAR we used for these tests was configured as a single DPAR (Domino partition running an instance of the Lotus Domino server). We configured the LPAR to have two (capped) physical processors, and because we ran AIX 5.3, we were able to utilize the simultaneous multi threading (SMT) feature that additionally enabled two logical processors.
The IBM FAStT Fiber Attached Storage was configured using RAID 0 and JFS2.
The Lotus Domino data files, mail journal, and mailboxes were installed on one IBM FAStT 600 array of 475 GB and that used RAID 0. The mail databases were spread across three IBM FAStT 600 arrays of 475 GB each and that used RAID 0. Transaction logging files were in a separate logical volume using RAID 0. Network access occurred through a single 1 GB Ethernet adapter running in full duplex mode.
Figures 4 and 5 show the resource utilization for the two tests at various user loads.
Figure 4. Processor utilization
Figure 4 shows the processor utilization for Lotus Domino 32-bit and Lotus Domino 64-bit at the same number of virtual users. The processor utilization was approximately the same for both 32-bit and 64-bit Lotus Domino at all user loads from 1,000 to 5,000 simulated users.
Figure 5. Disk I/O per second
Figure 5 shows the I/O per second for Lotus Domino 32-bit and Lotus Domino 64-bit at the same number of simulated users. The disk I/O per second rate was slightly improved (a lower number is better) with Lotus Domino 64-bit as compared to Domino 32-bit due to the increase in virtual memory addressing with 64-bit applications.
Table 7. Resource usage at 4,000 users
|Server||Lotus Domino 8.0.1, 32- bit||Lotus Domino 8.0.1, 64-bit||% change|
|Mail database ODS level||48||48||NA|
|Processor % busy||58||59||1.7%|
|Total disk reads/second||525||381||-21%|
|Total disk writes/second||680||626||-8%|
|Server virtual memory used (MB)||1.599||2.528||58%|
Table 7 compares several key resource utilization metrics on the Lotus Domino 8.0.1/AIX server at a virtual user load of 4,000 for each of the tests. It shows that with Lotus Domino 64-bit, we can access additional virtual memory as compared to running Lotus Domino 32-bit on the same server configuration . Also, Lotus Domino 64-bit server does slightly fewer disk reads and writes per second as compared to Lotus Domino 32-bit server.
The measurements in this article suggest that upgrading from the Lotus Domino 8.0.1 32-bit application to a Lotus Domino 8.0.1 64-bit application is a positive experience. We see no change to processor and I/O utilization in Lotus Domino 64-bit as compared to Lotus Domino 32-bit for mail applications. The increase in virtual memory footprint for Lotus Domino 64-bit rather than Lotus Domino 32-bit can be explained by the need to support 64-bit pointers and addresses instead of 32-bit pointers and addresses. This requirement causes an increase in several Lotus Domino internal data structures.
Table 8. Notes.ini settings
|Notes.ini parameter name||Explanation|
|ConstrainedSHMSizeMB||Set it to restrict shared memory to the size in megabytes specified.|
|Create_R8_Databases||Applies to Lotus Domino 8 servers. Set it to 1 to create new ODS 48 databases on Lotus Domino 8 servers.|
|DEBUG_ENABLE_SYS_V_SHM=1||Set it to 1 to let Lotus Domino use System V for shared memory instead of mmap files on UNIX® platforms.|
|DEBUG_NSF_SHOW_ALLSTATS||New in Lotus Domino 8. Set it to 1 to enable Lotus Domino I/O statistics collection for database metadata, views, and objects.|
|EVENT_CORRELATION_POOL_SIZE||Specifies the size of the event correlation pool used by the event task and DDM.|
|EVENT_POOL_SIZE||Set this variable to specify the amount of memory used for unprocessed events, unprocessed notifications, and event suppression. The default setting is 5242880 (5 MB).|
|log_mailrouting||Specifies the level of logging of router events performed by the router process.|
|MEM_AddressableMemSizeMB||Set it to override the default setting for the amount of memory Lotus Domino has available as "addressable" memory (where VALUE is in megabytes and where the default varies by platform).|
|MEM_EnablePreAlloc=1||Set it to 1 to enable pre-allocation of shared memory.|
|NLCACHE_SIZE||Specifies the size of Namelookup Cache in bytes. The default value is 16 MB.|
|NSF_Buffer_Pool_Size_MB||Specifies the maximum size (in bytes) of the NSF buffer pool, a section of memory dedicated to buffering I/O transfers between Lotus Domino and disk storage.|
|NSF_Dbcache_Maxentries||Specifies the number of databases that a server can hold in its database cache at one time.|
|RouterDbCacheSize||Specifies the size of the Router Database cache used to cache mail databases in the router process.|
|SCHEDULE_NO_VALIDATE||Enables or disables whether SchedMgr validates its busytime
database entry on a daily basis, as follows: |
0 (default) - Enables validation
1 - Disables validation
|Server_MAX_CONCURRENT_TRANS||Sets the limit for the number of concurrently scheduled transactions on a server.|
|Server_Pool_Tasks||Sets the total number of physical threads in the Lotus Domino server (DbServer) thread pool.|
|Server_Show_Performance||Applies to servers. Specifies whether server performance events are displayed on the console. Set this variable to 1 to display server performance events on the console.|
|ServerTasks||Specifies the tasks that begin automatically at server startup and continue until the server is shut down.|