Messaging and Directory Server software used around the world in the Government, Military, Aviation and Commercial sectors.

isode.com / whitepapers /

M-Store X.400 Benchmarks

Summary: This paper provides performance benchmarks for Isode's M-Store X.400 product.	Publish Date 31st May 2007 Last Modify Date n/a
Share this whitepaper:

Usage Models

X.400 P7 can be used in many ways, and it would be impractical and confusing to test all combinations. This section summarizes the common usage models that are being modelled here.

1. Full message fetch. All clients we are aware of operate by fetching complete messages, and not by selective component access.
2. List based on message status. Although P7 permits many ways to select messages, most clients select messages based on message status and this is the focus of testing here.
3. Empty mailbox. Many deployments work by the client fetching messages, and then deleting them soon afterward, so that the mailbox is kept small or empty.
4. Large mailbox as archive. A common approach is for clients to fetch new messages, and then leave in the in box (or out box). Messages will be deleted by the administrator, typically after an archive period. The client will fetch new messages. Messages in the archive will primarily be used for recovery purposes.
5. Periodic fetch, where a client will connect from time to time, list new messages and fetch them all.
6. Auto-alert with immediate fetch, where a client remains bound with auto-alert set. When an auto-alert is received, the message is immediately fetched.

The tests made are intended to cover all of these elements. More testing is done on "Large Mailbox", as this is more demanding than "Empty Mailbox".

Test Configuration

Testing was done on the following hardware:

• Dual 2.2 GHz Opteron.
• 4 GByte Memory
• 135 GByte SCSI RAID (0+1 configuration) with write-back cache
• Red Hat Linux

The core tests were done on a message store with the following configuration:

• 100 Mailboxes
• 11,000 messages in each mailbox, with 1,000 of status new and 10,000 of status processes.
• Total 1,100,000 messages stored.
• Messages 10 kBytes each.

This message store configuration is intended as a realistic test of the large mailbox model. The M-Store X.400 Server and the associated M-Vault X.500 index server were run on the same server. For tests involving the M-Switch X.400 MTA, this was also operating on the same server.

Operation Tests

The first set of tests looks at each of the core P3 and P7 operations, and provides measurements for each, using the setup described above.

Message Delivery from MTA

Number of Mailboxes delivered to	Delivery Rate (messages/sec)
1	83
10	83
100	83

Message delivery rate is independent of the number of mailboxes delivered to.

P7 List

Test	Rate (messages/sec)
List all (uncached)	2248
List unread (un-cached)	2762
List all (cached)	3459

The list test measured the rate at which message sequence numbers are returned by the list command. Listing all messages would be important when recovering messages from an archive. M-Store X.400 builds a cache that will improve performance of the list operation. The last test shows the effect of this cache.

P7 Summarize

Test	Rate (messages/sec)
Summarize all	5396
Summarize unread	5525

The rate for summarize measures the number of messages that are analyzed in order to produce the summarize result.

P7 Fetch

Test	Rate (messages/sec)
Fetch Processed Messages (1 client)	485
Fetch Processed Messages (10 clients)	1188
Fetch Unread Messages (1 client)	155
Fetch Unread Messages (10 clients)	294

Message fetching tests were performed with one client and 10 clients. It can be seen that one client can use about 50% of the maximum store capacity. Fetching a processed message interacts only with the core M-Store X.400 process. When an unread message is fetched, the index server needs to be updated to reflect change in message status. This update reduces overall fetch performance.

Delete

Test	Rate (messages/sec)
Delete Messages	631

Message Submission

Test	Rate (messages/sec)
Message Submission (1 client)	500
Message Submission (10 clients)	405
Message Submission, with store on submission (1 client)	188
Message Submission, with store on submission (10 clients)	166

Message submission is done using a P7 client to M-Store X.400, which in turn submits to M-Switch X.400 using P3. Tests were done with multiple clients, and it can be seen that additional clients slightly reduce overall submission rate. Tests were also performed using store on submission, where M-Store X.400 files a message copy in the Out Box that can be accessed with P7.

Throughput Tests

The second set of tests looks at operation combinations that reflect the primary models of message handling (periodic fetch and auto-alert) for empty and large mailboxes. For large mailboxes, the configuration described above was used. The operation sequence for the two models are:

• Periodic Fetch: Deliver; List Unread; Fetch; Delete.
• Auto-Alert: Deliver; Auto-Alert; Fetch; Delete

Model	Mailbox	Rate (messages/sec)
Auto Alert	Large	37
Periodic Fetch	Large	14.8
Auto Alert	Empty	45.5
Periodic Fetch	Empty	43.3

Performance for empty mailbox is better, as would be expected, and similar for both models. For large mailboxes, the auto-alert model gives significantly better performance, and is comparable to empty mailbox performance.

Large Messages

All of the above tests were done with 10kbyte messages. M-Store X.400 is designed so that performance is not significantly affected by message size. One fetch test was repeated using 200 kByte messages.

Model	Mailbox	Rate (messages/sec)
Fetch New Messages (1 client)	10 kByte	155
Fetch New Messages (1 client)	200 kByte	153

This indicates that performance is reasonably independent of message size.

Sizing and Scaling

The tests have been performed on a reasonably large system, and the numbers will easily meet the requirements of many deployments. This section considers first how the system will scale on the increase of some parameters:

• Number of mailboxes. There is very little dependency on the number of mailboxes, and it is anticipated that the server will scale well to support thousands of mailboxes, without significant performance impact.
• Mailbox size. The large mailbox model tested mailboxes of 10,000 messages. Moderate increase of this size is not expected to have significant impact. Some tests were done for a mailbox of 1,000,000 messages. This is usable, but would not be generally recommended for the current product version.
• Total number of messages. There are scaling implications for the total number of messages, particularly for the index server. It is anticipated that use for up to 10,000,000 messages in total would not give significant degradation in performance.

Where a very high performance system is needed, a number of options are available:

1. Use a higher specification server, and in particular higher performance disks.
2. Split load across multiple servers.
3. Use of multiple file systems.
4. Operate M-Store X.400 and the index server (M-Vault X.500) on separate servers.