IMAP (Internet Message Access Protocol) and its popular cousin, POP3 (the Post Office Protocol), are both very widely implemented protocols that allow access to e-mail. Although POP is ubiquitous, IMAP is the better choice for handling mail in many cases. For example, POP3's single mailbox model can lead to thousands of large messages in a single location, making access and control of the mailbox a tedious affair. IMAP, however, is designed specifically for leaving the messages on the server, classified in various mailboxes. Additionally, IMAP can check for new mail over an existing IMAP connection, while POP3 has to open a new connection every time. (For links to technical details on both protocols, check the Resources section of this article. The POP3 RFC has specific information on connections in the "Scaling and Operational Considerations" section.)

In this article I'll focus on IMAP and present my ifrom utility for managing your e-mail (see Resources to download ifrom.pl). Personally, I use ifrom for listing, printing, and moving messages on an IMAP server. I also run it nightly for backing up my IMAP mail, because I'm paranoid about it. (And remember, just because you're paranoid doesn't mean they're not after you.)

Listing messages

The simplest invocation of ifrom is just so, without any options. It will access your INBOX mailbox on the IMAP server and list your messages in a numbered list. The -host option tells ifrom which IMAP server to contact.

Note that ifrom uses the AppConfig module, which provides for default values, among other nice features.

Authentication in ifrom is done with plain LOGIN (meaning you specify your password and user name in plain text to the IMAP server). Follow the link to the Authmechanism capability of the Mail::IMAPClient package in Resources. You can use CRAM-MD5 authentication with the -crammd5 switch; no other mechanisms are supported by Mail::IMAPClient, so I did not bother to set up a generic -authentication switch. Note that you can implement your own authentication; see the Mail::IMAPClient documentation.

if ($config->CRAMMD5())
{
 my $authmech = "CRAM-MD5";
 if ($imap->has_capability($authmech))
 {
  print "Switching to $authmech authentication\n";
  $imap->Authmechanism($authmech);
 }
}

You can specify your name and password from the command line with the -user and -password switches, or through an authinfo (AKA netrc) file. The authinfo file stores a list of authentication options, such as:

machine imap.yourserver.here login joe password JoeSecret
machine imap.yourserver.there password FredSecret port 244

The authinfo file is used by other programs, and the format is variable depending on what those programs need; ifrom parses it looking for the machine, login, password, and port keywords. If any of those keywords are not specified, the defaults are used. The authinfo file overrides the -user and -password command-line switches. Make sure your authinfo file is only readable by you!

The machine name is your IMAP server's name. The machine name in the authinfo file has to match exactly the name given to ifrom with the -host switch.

When ifrom connects to the server, it sets the Peek variable to 1, so that messages we examine are not marked read.

Next, ifrom opens the mailbox specified by the -mailbox switch. This is INBOX by default: the standard IMAP main mailbox.

After all this trouble, ifrom prints out messages using this format:

printf "%5d %-35.35s %s\n", $count, $address,
   ((defined $data->{Subject}->[0]) ? $data->{Subject}->[0] : '');

The sender's address in $address is extracted from the full user name whenever possible, using a simple regular expression match.

If you give ifrom the -dump switch, it will also print the message's contents after the header. This is very useful if you don't have time to launch your e-mail application to see the contents of a last-minute message. It's okay to interrupt ifrom while it's printing a large message; the message will not be deleted or otherwise affected, since we opened the server with the Peek option set to 1.

Note that the -dump switch uses the body_string() function, whereas the -backup option we'll see later uses the message_string() function.

Back to top

Moving messages

Occasionally, you want to move messages from one mailbox to another. If there are many messages and your mail client loses the connection to the IMAP server while moving them (as happens often over a slow network link), this capability is for you.

Just use the -mailbox and -to options. It's that simple. Run ifrom like this:

ifrom -mailbox newmail -to archive

You can also tell ifrom to stop moving messages after a certain number with the -n switch.

Here is the actual code that does the moving. IMAP has built-in article moving, so the client simply has to invoke the move() function.

 foreach my $message (@msg_list)
 {
  $count++;
  if ($config->TO)
  {
   die "Could not move message $message: $!" unless $imap->move($config->TO, $message);
   print "Moved message $message to " . $config->TO, "\n";
   $imap->expunge() if $config->EXPUNGE_OFTEN;
   last if $count >= $config->N;
  }
 }

 if ($config->TO)
 {
  $imap->expunge();
 }

Moving messages in IMAP does not respect the Peek setting. Even if it is set to 1, Peek is only relevant to looking at messages. Moving the messages will always delete their originals after the copies have been made in the destination mailbox, as implemented by Mail::IMAPClient.

Back to top

Expunging and deleting mailboxes

When moving messages above, you must have noticed the mysterious expunge() function. It simply tells the IMAP server to flush the mailbox, removing all messages marked as deleted. Normally, when listing messages, we don't mark them as deleted because we have Peek set to 1. The move() function, however, will mark them as deleted as mentioned above. In order to really delete those messages, the expunge() function must be called.

This wouldn't be necessary if IMAP really moved messages, but it doesn't, so Mail::IMAPClient implements move() as a copy() followed by a deletion.

Why is this useful? We all delete mail accidentally. This IMAP feature allows you to get deleted messages back before the final expunge is done.

The -expunge_often flag is for moving messages when you are on an unreliable link and could get disconnected any time. It ensures that after every message is moved, expunge() will be called (otherwise, the message will remain in the mailbox). It is better, however, to use plain expunge() after all the messages are moved, in combination with the -n flag. That way, expunge() will be called after every 10, 15, or however many messages you specify, are moved.

I also gave ifrom the -delete_mailbox_really option. When given that option, ifrom will delete whatever mailbox is named by the -mailbox switch, so don't use it on the default INBOX mailbox! The Perl code involved couldn't be simpler:

if ($config->DELETE_MAILBOX_REALLY)
{
 $imap->delete($config->MAILBOX)
  or warn "Could not delete mailbox " . $config->MAILBOX . "\n";
}

Back to top

Backing up IMAP mail

I like backups; they have saved my work many, many times. When it comes to e-mail, backups may seem less important, but in fact many of our daily interactions fly by over e-mail. Often, we find out much later that we shouldn't have erased that memo about wearing shorts on Fridays, or the reminder about the company-wide meeting a month from now. If you aren't careful, the overworked assistant you ask to track down a lost memo will tell your boss you've been hoarding cocoa packets in your desk. Don't let this happen to you.

To back up your mail using ifrom, just give it the -backup flag. The -savedir switch is also important -- set it to wherever you want the saved messages to go. Everything else -- authentication, host, port, and so on -- works just like the regular ifrom. The -mailbox parameter doesn't work, because ifrom backs up all the mailboxes. I could have had a special -backup_mailbox flag to override "all mailboxes," but frankly I never needed it.

Messages in IMAP mailboxes have unique numbers. We take advantage of that fact in ifrom, saving each message to the file savedir/mailbox/messageNumber and skipping the message if the file already exists.

  foreach my $message (@msg_list)
  {
   my $filename = "$dir/$f/$message";
   next if -e $filename;
   print "saving message $f/$message to $filename\n" if $config->VERBOSE;
   my $data_fh = new IO::File $filename, "w";
   my $data = $imap->message_string($message);
   warn "Empty message data for $f/$message" unless defined $data && length $data;
   $data_fh->print($data);
  }

We use the message_string() function because it retrieves the whole message. The body_string() function used by the -dump switch earlier skips the messages headers, which are usually not interesting when you're looking at e-mail quickly.

Back to top

Conclusion

The ifrom utility is one of those little scripts that just kept growing and gaining useful features. I wouldn't be surprised if other people have done their own ifrom scripts. The basic message-listing feature has been incredibly useful to me, since checking my IMAP mail is sometimes impossible if I'm doing system maintenance in the machine room, or I'm on a slow connection.

I have already used the ifrom backups several times, when I've lost important mail by accident. It's a load off my mind to know that even if the ISP that runs my IMAP server should lose my mailboxes (it's happened to me before), I will lose at most one day's worth of mail.

I hope this article has interested you in writing Perl to interface with IMAP, and that you find ifrom useful. If you use ifrom, I'd love to hear how you use it.

Back to top

Download

Information about download methods

Resources

• Read Ted's other Perl articles in the "Cultured Perl" series on developerWorks.
  
  
• Download ifrom.pl, the ifrom script.
  
  
• The Mail::IMAPClient manual pages are your basic resource for using the package's features, including the Authmechanism capability.
  
  
• Net::IMAP is an alternative to Mail::IMAPClient, but it has not been updated since 1999. Mail::IMAPClient development seems to be more active. I have not used Net::IMAP because Mail::IMAPClient fits my needs perfectly.
  
  
• The IMAP4rev1 RFC 2060, the document where today's version of the IMAP protocol was defined, is required reading.
  
  
• Also read the POP3 RFC 1939, the document where today's version of the POP3 protocol was defined.
  
  
• The CPAN module archive has all the Perl modules you could ever want.
  
  
• Go to Perl.com for more Perl information and related resources.
  
  
• For a Java-based mail server that supports IMAP and other protocols, read "Working with James, Part 1: An introduction to Apache's James enterprise e-mail server" (developerWorks, June 2003).
  
  

About the author

Rate this article

Comments

Back to top

Table of contents

• Listing messages
• Moving messages
• Expunging and deleting mailboxes
• Backing up IMAP mail
• Conclusion
• Download
• Resources
• About the author
• Comments