Technical Blog Post
Abstract
Email Listener: Reducing IMAP Message Fetch Latency Due to Large Attachments
Body
Introduction
In this blog entry we will look at reducing high email listener email fetch latency which may result when there is be a need to frequently create service requests from IMAP messages that have large documents attached.
To keep things simple, the constants will be a Tpae 7.5.0.X environment and a simple email server program both running on the same system. The initial conditions are the MXServer shut down and a batch of 6 emails, each with a 15 MB PDF attachment, sitting on the email server. The MXServer will be started, the listener will process the emails and the times as measured by the deltas between 6 sequential acknowledgement emails will be recorded. The time noted will be from the Date field of the raw email which denotes the exact time the email arrived at the mail server. This will give us 5 time intervals for each trial to average and then compare against the different settings.
Scenarios
We will examine 4 scenarios:
1) Using the Default
Without any adjustments the JavaMail API partially fetches email data in 16KB (16384 B) chunks.
2) Increasing Fetch Size to 1MB
Add or set the property/value mail.imap.fetchsize=1048576 at the application server JVM or in the System Properties application under Global Properties (a server restart may be required for either option).
3) Increasing Fetch Size to 4MB
Add or set the property/value mail.imap.fetchsize=4194304 at the application server JVM or in the System Properties application under Global Properties(a server restart may be required for either option).
4) Disabling Partial Fetch
Add or set the property/value mail.imap.partialfetch=false at the application server JVM or in the System Properties application under Global Properties (a server restart may be required for either option).
Data
| time | delta | |
| Fetch chunk size 16KB (default) | ||
| 1 | 10:19:28 | |
| 2 | 10:22:22 | 00:02:54 |
| 3 | 10:25:15 | 00:02:53 |
| 4 | 10:28:08 | 00:02:53 |
| 5 | 10:31:00 | 00:02:52 |
| 6 | 10:33:56 | 00:02:56 |
| average | 00:02:54 | |
| Fetch chunk size 1MB | ||
| 1 | 10:55:15 | |
| 2 | 10:55:28 | 00:00:13 |
| 3 | 10:55:41 | 00:00:13 |
| 4 | 10:55:53 | 00:00:12 |
| 5 | 10:56:05 | 00:00:12 |
| 6 | 10:56:17 | 00:00:12 |
| average | 00:00:12 | |
| Fetch chunk size 4MB | ||
| 1 | 11:21:55 | |
| 2 | 11:22:06 | 00:00:11 |
| 3 | 11:22:17 | 00:00:11 |
| 4 | 11:22:27 | 00:00:10 |
| 5 | 11:22:38 | 00:00:11 |
| 6 | 11:22:48 | 00:00:10 |
| average | 00:00:11 | |
| Partial fetch disabled | ||
| 1 | 12:06:46 | |
| 2 | 12:06:56 | 00:00:10 |
| 3 | 12:07:06 | 00:00:10 |
| 4 | 12:07:16 | 00:00:10 |
| 5 | 12:07:26 | 00:00:10 |
| 6 | 12:07:36 | 00:00:10 |
| average | 00:00:10 |
Conclusion
Under these conditions the biggest improvement (by a factor of 14.5) is gained by tweaking the fetch chunk to 1MB. From this reference point, there was a marginal improvement increasing to a 4MB chunk or disabling the function completely.
If you are having a high degree of attachment processing latency using the IMAP email protocol, then these settings should be a consideration. As with any system adjustment, this should be thoroughly tested before using in a production environment.
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSLKT6","label":"IBM Maximo Asset Management"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB59","label":"Sustainability Software"}}]
UID
ibm11132767