With IBM Lotus Domino 8.0.1, Lotus Domino has added a 64-bit native support
option for Microsoft® Windows® 2003 and IBM AIX® platforms. With 64-bit
support, Lotus Domino can address more memory on the server than is
possible in previous versions. Maintaining Lotus Domino mail server
performance with the new 64-bit platform was important.
In this article, we show how the 64-bit Lotus Domino 8.0.1 server performs
against 32-bit Lotus Domino 8.0.1 using the N8Mail workload, which
simulates a Lotus Notes® 8 client mail user. For more details on the
N8Mail workloads, refer to the developerWorks® article, “IBM Lotus Notes 8 workloads: Taking performance to a new level.”
We updated our testing methodology to more accurately reflect the
environment in which our customers deploy. We now have users start with
3,000 documents of various sizes in their mail files. We also enabled
transaction logging, mail journaling, Domino domain monitoring (DDM)
probes, and mail rules. Default server tasks ran during the tests. The net
result was a much heavier workload than in the previous R6Mail workload,
so do not compare the two workloads. In addition, we no longer show the
maximum number of users possible, but rather focus on a typical deployment
of about 4,000 users.
The test suites that we ran for this article are designed to help you
understand 64-bit Lotus Domino performance compared to that of 32-bit
Lotus Domino on the same hardware configuration. The first test could be
considered the baseline. It is a 32-bit Lotus Domino 8.0.1 server running
with mail8.ntf-based mail databases, at Lotus Domino 8 ODS 48 level and
exercised with the N8Mail workload. The second test is a 64-bit Lotus
Domino 8.0.1 server running with mail8.ntf-based mail databases, at Lotus
Domino 8 ODS 48 level and exercised with the N8Mail workload.
For quick reference, the labels on the following charts are listed in table
Table 1. Labels for performance
|Lotus Domino 8.0.1, 32-bit||N8Mail||Mail8.ntf||48||Lotus Domino 8.0.1|
|Lotus Domino 8.0.1, 64-bit||N8Mail||Mail8.ntf||48||Lotus Domino 8.0.1|
The Lotus Domino server Notes.ini values in the following sections
represent the culmination of many tests and extensive statistical
analysis. The use of these values is not recommended without analyzing the
performance of a production Lotus Domino server. Also, be aware that the
Notes.ini settings shown here for one platform might not work well on a
different platform. For an explanation of what each of the ini parameters
is used for and how to determine the optimum value range, refer to the
NOTE: The results in this article come from benchmarks executed in a
controlled environment. Although some effort was made during the creation
of the benchmark to include typical user operations, it is likely that
real users make different use of Lotus Domino than the narrow range of
function that is tested by the benchmark. Therefore, these numbers should
be used primarily to understand the relative performance of the Lotus
Domino releases, and they do not represent recommendations for real-world
deployment. For assistance with capacity planning, consult your hardware
Also, although we show results on a variety of hardware platforms, these
configurations are not of uniform capacity. It is our intent to focus on
the performance of Lotus Domino itself, and this data should not be used
to compare hardware platforms against each other.
Microsoft Windows 2003
The following tables show the hardware and software configurations for our
Microsoft Windows tests.
Table 2. Hardware configuration for
Microsoft Windows tests
|Model||Intel x64-based PC|
|Processors for test/speed||Eight processors @ 3.33GHz|
|Installed memory||8 GB|
|Active physical drives||35 disks|
|Active logical volumes||Five arrays using RAID 0|
|Operating system||Microsoft Windows 2003 Server Enterprise x64 SP2|
Table 3. Software configuration for
Microsoft Windows tests
Lotus Domino server was set up as a single-partition server on an Intel
x64-based system running Microsoft Windows 2003 64-bit Enterprise Server
with eight processors, and with 8 GB of memory recognized by the Microsoft
Windows operating system. The Lotus Domino data files, mail journal, and
mailboxes were installed on one IBM FAStT 600 array of 475 GB and that
used RAID 0. The mail databases were spread across three IBM FAStT 600
arrays of 475 GB and that used RAID 0. Transaction logging files were in a
separate hard disk drive. The system page file, configured for 8000 MB,
was located on a small three-disk RAID volume in the IBM FAStT 600 array.
Network access occurred through a single 1 GB Ethernet adapter running in
full duplex mode. Figures 1-3 show the resource utilization charts for the
two measurements at various user loads.
Figure 1. Processor
Figure 1 shows almost the same processor utilization for both 32-bit and
64-bit Lotus Domino at all the user loads from 1,000 to 5,000 simulated
Figure 2. Disk I/O per second
Figure 2 shows about the same I/O per second for both 32-bit and 64-bit
Lotus Domino at all the user loads from 1,000 to 5,000 simulated
Figure 3. nServer virtual memory
Both 32-bit and 64-bit Lotus Domino servers explicitly set the
LargeSystemCache in Microsoft Windows 2003 server to 2.4 GB, using the
cacheset.exe tool provided with Lotus Domino. In figure 3, 64-bit Lotus
Domino server process virtual memory was over 2 GB even at a load of 3,000
virtual users. The 32-bit Lotus Domino server reached close to 2 GB
virtual memory at 5,000 users. At each load point, roughly 400 MB more
virtual bytes are consumed by the 64-bit Lotus Domino server as compared
to the 32-bit Lotus Domino server. The increase in virtual memory of the
nserver.exe process is primarily attributed to the process private memory
increase, not to the memory that is shared across all the Lotus Domino
processes. We also observed that there was not a substantial difference in
available physical memory between Lotus Domino 32-bit and Lotus Domino
64-bit servers at the same number of users.
Table 4. Resource usage at 4,000
|Server||Lotus Domino 8.0.1, 32-bit||Lotus Domino 8.0.1, 64-bit||% change|
|Mail database ODS level||48||48||NA|
|Processor % busy||18.1||17.5||-3.3%|
|Total disk reads/second||216||206||-4.6%|
|Total disk writes/second||586||587||0%|
|nServer virtual memory used (MB)||1670||2128||+27.4%|
|Physical memory available (MB)||2850||2764||-3%|
Table 4 compares several key resource utilization metrics on the Lotus
Domino server at a virtual user load of 4,000 for each of the tests. This
table clearly shows that there is no substantial change in processor
utilization, disk I/O metrics, and network bytes by upgrading from 32-bit
Tables 5 and 6 show the hardware and software configurations for our
Microsoft Windows tests for AIX server.
Table 5. Hardware configuration for AIX
|Processors for test/speed||Two physical/two logical processors (SMT) 1.5GHz|
|Installed memory||16 GB RAM|
|Active physical drives||IBM FAStT Fiber Channel storage, seven trays each with 14 15,000
RPM disks, configured as four logical volumes using RAID 0
|Active logical volumes||Three arrays for Lotus Domino binaries/data, one array for
|Operating system||AIX 5.3 ML5 64-bit|
Table 6. Software configuration for AIX
The AIX test server was a p570+ configured into multiple LPARS. (A logical
partition, or LPAR, is the division of a computer’s processors, memory,
and hardware resources into multiple environments so that each environment
can be operated independently with its own operating system and
applications.) The LPAR we used for these tests was configured as a single
DPAR (Domino partition running an instance of the Lotus Domino server). We
configured the LPAR to have two (capped) physical processors, and because
we ran AIX 5.3, we were able to utilize the simultaneous multi threading
(SMT) feature that additionally enabled two logical processors.
The IBM FAStT Fiber Attached Storage was configured using RAID 0 and JFS2.
The Lotus Domino data files, mail journal, and mailboxes were installed on
one IBM FAStT 600 array of 475 GB and that used RAID 0. The mail databases
were spread across three IBM FAStT 600 arrays of 475 GB each and that used
RAID 0. Transaction logging files were in a separate logical volume using
RAID 0. Network access occurred through a single 1 GB Ethernet adapter
running in full duplex mode.
Figures 4 and 5 show the resource utilization for the two tests at various
Figure 4. Processor
Figure 4 shows the processor utilization for Lotus Domino 32-bit and Lotus
Domino 64-bit at the same number of virtual users. The processor
utilization was approximately the same for both 32-bit and 64-bit Lotus
Domino at all user loads from 1,000 to 5,000 simulated users.
Figure 5. Disk I/O per second
Figure 5 shows the I/O per second for Lotus Domino 32-bit and Lotus Domino
64-bit at the same number of simulated users. The disk I/O per second rate
was slightly improved (a lower number is better) with Lotus Domino 64-bit
as compared to Domino 32-bit due to the increase in virtual memory
addressing with 64-bit applications.
Table 7. Resource usage at 4,000
|Server||Lotus Domino 8.0.1, 32- bit||Lotus Domino 8.0.1, 64-bit||% change|
|Mail database ODS level||48||48||NA|
|Processor % busy||58||59||1.7%|
|Total disk reads/second||525||381||-21%|
|Total disk writes/second||680||626||-8%|
|Server virtual memory used (MB)||1.599||2.528||58%|
Table 7 compares several key resource utilization metrics on the Lotus
Domino 8.0.1/AIX server at a virtual user load of 4,000 for each of the
tests. It shows that with Lotus Domino 64-bit, we can access additional
virtual memory as compared to running Lotus Domino 32-bit on the same
server configuration . Also, Lotus Domino 64-bit server does slightly
fewer disk reads and writes per second as compared to Lotus Domino 32-bit
The measurements in this article suggest that upgrading from the Lotus
Domino 8.0.1 32-bit application to a Lotus Domino 8.0.1 64-bit application
is a positive experience. We see no change to processor and I/O
utilization in Lotus Domino 64-bit as compared to Lotus Domino 32-bit for
mail applications. The increase in virtual memory footprint for Lotus
Domino 64-bit rather than Lotus Domino 32-bit can be explained by the need
to support 64-bit pointers and addresses instead of 32-bit pointers and
addresses. This requirement causes an increase in several Lotus Domino
internal data structures.
Table 8. Notes.ini settings
|Notes.ini parameter name||Explanation|
|ConstrainedSHMSizeMB||Set it to restrict shared memory to the size in megabytes
|Create_R8_Databases||Applies to Lotus Domino 8 servers. Set it to 1 to create new ODS
48 databases on Lotus Domino 8 servers.
|DEBUG_ENABLE_SYS_V_SHM=1||Set it to 1 to let Lotus Domino use System V for shared memory
instead of mmap files on UNIX® platforms.
|DEBUG_NSF_SHOW_ALLSTATS||New in Lotus Domino 8. Set it to 1 to enable Lotus Domino I/O
statistics collection for database metadata, views, and objects.
|EVENT_CORRELATION_POOL_SIZE||Specifies the size of the event correlation pool used by the event
task and DDM.
|EVENT_POOL_SIZE||Set this variable to specify the amount of memory used for
unprocessed events, unprocessed notifications, and event
suppression. The default setting is 5242880 (5 MB).
|log_mailrouting||Specifies the level of logging of router events performed by the
|MEM_AddressableMemSizeMB||Set it to override the default setting for the amount of memory
Lotus Domino has available as “addressable” memory (where VALUE is
in megabytes and where the default varies by platform).
|MEM_EnablePreAlloc=1||Set it to 1 to enable pre-allocation of shared memory.|
|NLCACHE_SIZE||Specifies the size of Namelookup Cache in bytes. The default value
is 16 MB.
|NSF_Buffer_Pool_Size_MB||Specifies the maximum size (in bytes) of the NSF buffer pool, a
section of memory dedicated to buffering I/O transfers between
Lotus Domino and disk storage.
|NSF_Dbcache_Maxentries||Specifies the number of databases that a server can hold in its
database cache at one time.
|RouterDbCacheSize||Specifies the size of the Router Database cache used to cache mail
databases in the router process.
|SCHEDULE_NO_VALIDATE||Enables or disables whether SchedMgr validates its busytime
database entry on a daily basis, as follows:
0 (default) –
1 – Disables validation
|Server_MAX_CONCURRENT_TRANS||Sets the limit for the number of concurrently scheduled
transactions on a server.
|Server_Pool_Tasks||Sets the total number of physical threads in the Lotus Domino
server (DbServer) thread pool.
|Server_Show_Performance||Applies to servers. Specifies whether server performance events
are displayed on the console. Set this variable to 1 to display
server performance events on the console.
|ServerTasks||Specifies the tasks that begin automatically at server startup and
continue until the server is shut down.