Using Notes/Domino SMTP with a DMZ, Part 1

Email is everywhere -- and so are people who want to abuse it to gain access to your corporate environment. Help keep them out by setting up a DMZ between the public Internet and your company's users and resources. In this first of a two-part series, we explain how SMTP mail works. We take a look at its potential vulnerabilities and how spammers and hackers try to take advantage of them, and then discuss how a DMZ can help spoil their nefarious plans.


David Bell, Senior IT Architect, IBM, Software Group

David Bell is a Senior IT Architect with the IBM Software Group, specializing in services for the Lotus Software brand of products. In that capacity, he is responsible for anticipating and resolving technical issues that arise in the course of consulting engagements and for designing, building, and implementing messaging infrastructures that meet customer needs. He works with customers to facilitate the formulation and execution of information strategies that are innovative and well aligned with their business goals. David has been working with Lotus Notes and Domino since the 1990s, designing and implementing architectures for organizations of all sizes.

Timothy Speed, Consulting IT Architect, IBM, Software Group

Timothy Speed is an infrastructure and security architect for IBM Software Services for Lotus (ISSL). Tim has been involved in Internet and messaging security since 1992. He also participated with the Domino infrastructure at the Nagano Olympics and assisted with the Lotus Notes systems for the Sydney Olympics. His certifications include MCSE©, VCA (VeriSign Certified Administrator), Lotus Domino CLP Principal Administrator, and Lotus Domino CLP Principal Developer. Tim has also co-authored four books: The Internet Security Guidebook,ISBN: 0122374711, February, 2001; The Personal Internet Security Guidebook,ISBN: 0126565619, October, 2001; Enterprise Directory and Security Implementation Guide: Designing and Implementing Directories in Your Organization,ISBN:0121604527; and Internet Security: A Jumpstart for Systems Administrators and IT Managers, ISBN 1555582982.

09 November 2004

Also available in Japanese

One of the authors of this article (Tim) has been playing racquetball for about 30 years. During that time, he has been a member of several racquetball clubs -- it's a great way to acquire new playing partners and friends. In earlier days, when Tim felt like getting a little exercise, he walked up to someone who looked like he played at about the same level and engaged in the following typical exchange:

  1. Tim: Hello, want to play a game?
  2. Them: Sure, let's play.
  3. Game is played.
  4. Thank each other for a well played game, then exchange phone numbers for possible future games.

Today the process is pretty much the same with the exception of the last step. Nowadays (thanks to the near-ubiquity of on-line messaging), Tim and his new friend almost invariably exchange email addresses. No more dealing with answering machines; now Tim can just send potential partners an email. They may read it at the office or on a Palm/Blackberry device and send a reply in a few minutes -- and off they go to play.

This simple example illustrates the countless ways email is increasingly becoming the communication medium of choice; it is simple and quick and allows you to send a request and leave a message in one step. And thanks to SMTP, different email systems from different vendors can exchange messages relatively seamlessly over the Internet. Of course, the more open and accessible your email is, the more you need to be concerned with protecting your corporate systems and data from unauthorized access. One way to do this is by routing Internet email traffic through an intermediary network called a DMZ (demilitarized zone).

In this two-part article series, we explain how you can set up your Lotus Notes/Domino environment to route SMTP/Internet mail through a DMZ. We start by taking a look at how Lotus Notes mail and SMTP work internally. We'll also examine how a DMZ functions. In Part 2, we'll take a close look at setting up and using Domino with a DMZ. This article assumes that you're an experienced administrator with some familiarity with Lotus Notes and Domino (although even readers who are relatively new to Lotus products can follow along).

A brief history of Lotus Notes mail

In late 1989, Lotus released the first version of Lotus Notes. It had a messaging system included as part of its architecture (and of course, still does). This advanced messaging system was built around a set of internal routers that read a user directory and then determined the delivery path of each message.

As smart as Lotus Notes was, it initially did not have the capability to send or receive SMTP messages. So in release 3, a "gateway service" became available as an add-on. This gateway provided basic connectivity to other SMTP-based systems on the Internet. It processed messages to and from the Internet via a foreign domain document and a special database. After processing, the gateway converted the message to Notes document format, and then delivered it to the addressee. However, complete message fidelity was not guaranteed between Lotus Notes and SMTP mail. The messages were rendered as plain text within the Notes body field. Multipurpose Internet Mail Extensions (MIME) multi-part objects (both text and binary) were converted to attachments within the message. Due to limitations in the gateway, nested multi-part objects were "flattened" and converted as a separate attachment. Overall, the gateway was functional, but not entirely optimal when it came to message translation.

Improved Internet mail support came with Lotus Notes release 4 and the SMTP MTA, another add-on product. This provided much closer integration than the release 3 gateway provided. The Lotus Notes release 4 SMTP MTA provided a series of inbound and outbound work queues to process each message. Message fidelity was still not perfect, but it was much improved over release 3 as was MIME support. Lotus Domino 4.5 provided further advancements. This included SMTP/MIME via the SMTP MTA. There were also several enhancements for Web and Post Office Protocol (POP) support. But the SMTP service included with release 4.5 was still effectively an add-on component and was still based on queues (inbound and outbound).

Lotus Domino 5 fully resolved these issues and brought Lotus Notes into the world of native Internet standards. Release 5 gave full MIME fidelity and eliminated the old queue system for SMTP mail routing. New features for SMTP management and configuration were also included in the directory-based configuration with a number of release 4 Notes.ini settings replaced by settings in the configuration documents. This improved architecture provided a solid base for Lotus Notes/Domino 6 to build on, and today Lotus Notes/Domino provides world-class SMTP mail architecture.

Figure 1. Lotus Notes/Domino 6 SMTP architecture
Lotus Notes/Domino 6 SMTP architecture

Notes/Domino 6 SMTP mail routing services are built into the product; there are no more inbound and outbound work queues. The process is very simple:

  1. A message (Notes or SMTP-based) is created on the client’s local network.
  2. The user sends the message via the Domino 6 server.
  3. Lotus Domino executes a TCP/IP DNS (Domain Name System) resolution and finds the target server.
  4. The message is transferred to the target recipient’s server, and then delivered to the recipient.

As you can see, it's a relatively straightforward process. (For more information on the history of Notes, please refer to the developerWorks: Lotus article, "The History of Notes and Domino.")

SMTP mail routing

SMTP mail routing to the target server is done through a DNS entry known as a mail exchange (MX) record, which is defined in RFC1035. MX records are used to specify the inbound email servers responsible for any particular SMTP domain name. Each MX record points to the name of an email server and holds a preference number for that server. If a domain name is handled, then multiple email servers use a separate MX record for each email server, and each preference number then determines the order in which other email servers should use these servers. Preference values are calculated with lower numbers having the higher priority. When sending an email to, the email server must first look up any MX records for to see which email servers handle incoming email for this domain. The value returned is the names of the email servers that have MX records assigned to that domain. The format of the MX record in DNS is:

name IN MX preference host

Here is an example of mail for a domain being directed to specific SMTP server machines: MX 10 MX 20

In the previous example, the MX records are configured to accept mail for and to attempt to direct it to the host ( because the MX record directing mail to has the lower reference value. If is unreachable at the time of delivery, the mail is directed to the next lowest preference mail server. In this example, is the one.

Figure 2. MX records in action
MX records in action

This is an important concept. If you understand Lotus Notes Named Networks (also known as Domino Named Networks), then there are conditions in which Notes mail will route alphabetically. This is not the case with SMTP mail routing; MX records are routing mechanisms as specified in a DNS.

SMTP vulnerabilities

If there is a potential issue with SMTP mail routing, it's that it's very simple. So simple in fact that evil doers have taken advantage of the simplicity of the SMTP protocol to do things such as:

  • Spam
    In the off-chance you have been living on Mars, spam is the common name for unsolicited "junk" email sent to large groups of people to promote some type of product or service. Spam wastes your time, increases the load on all SMTP servers, and eats up disk space on your messaging servers.
  • SMTP relaying
    The original design of SMTP included a concept known as SMTP relay. SMTP mail can be sent from a user hosted on ServerA to ServerB, which will relay the message to ServerC. Then ServerC can deliver the message to the recipient or transfer it to another server. Overall, this is an excellent architecture. However, spammers can use relays to bounce messages to other servers and have those server route the mail to the targeted user.
  • Address spoofing
    "Spoofing" occurs when an unauthorized user sends email under someone else's name. It is very easy to spoof an SMTP address, and it can be difficult to stop. Malicious users can set the from and sendto fields in a message to anything they want. (For more information on spoofing, see the developerWorks: Lotus article, "Lessons in secure messaging using Domino 6.")

Lotus Notes/Domino offers a number of features designed to help prevent spammers from taking advantage of these potential vulnerabilities. These features are described in the two-part article series, "Controlling spam: Advanced SMTP settings in Lotus Domino, Part 1" and "Controlling spam: Advanced SMTP settings in Lotus Domino, Part 2." In this article, we discuss another way that you can help control SMTP traffic in your messaging environment. This involves setting up a trusted network with a DMZ.

Anatomy of a spam attack

A typical spammer technique is to generate spam with fake from addresses and to use a known list of sendto address and/or just guess at an address. This all starts with the "local" part of the email address and the domain suffix. The domain suffix is the portion of the Internet address that follows the @ symbol. The local part is the actual name that you use to make the user unique. For example, in the address, Bubba is the local part and is the domain suffix.

Spammers can easily harvest email addresses directly from the Internet. To see how, try this: Go to and type either inurl:"@<your company email domain suffix>" or intext:” @<your company email domain suffix>" in the search field. For example, if your email address is bubba@the, type inurl:"". You will then see a listing of sites that contain email addresses with your company's domain name in them.

This is a very simple form of address harvesting; many more sophisticated ones exist. The point is that there are millions of SMTP addresses just waiting to be found on the Internet. Spammers also often use a program that opens a list (or even a directory) of words and that mixes and matches those words with numbers to create potential email addresses (for instance, Mike, Smith, MikeSmith, SmithMike, and so on). The program then mixes the computed local part with a domain suffix, yielding addresses such as,,, and so on.

Now that spammers have obtained potential addresses, they can start looking for an open relay. This is often relatively easy: Create a program to find a server listening on TCP port 25 (SMTP). Once found, a simple test determines whether or not the relay is enabled. If so, the spammer can use this server to route mail by creating a series of messages using the addresses found/created, entering a fake name into the from field, and sending the messages out to the Internet via the relay. A target site may accept the message if the address matches the sendto name; if the message is accepted, it is then delivered to the computed user's mailbox. If the message is not accepted, then it can be bounced back to the from address, meaning some poor innocent user or company may be inundated with non-delivery reports (NDR).

To test for open SMTP relays, use Telnet. This terminal emulation program allows you to enter commands on your PC and have them executed on the server as though they were entered directly at the server console. Telnet lets you enter SMTP commands that check whether or not you can send messages to target addresses. For instance, you can open Telnet and issue the following SMTP commands:

HELO <fake.domain>
MAIL FROM: <bogus@fake.domain>
RCPT TO: <arealaddress@arealsuffix.domain>
<text message followed by a period>

If the message is delivered to the target address <arealaddress@arealsuffix.domain>, then the relay is enabled. If not, you will see the following error message:

554 Relay rejected for policy reasons

This is very important – in the Notes 4 SMTP MTA, the message is accepted into the inbound work queue. Then a Notes.ini variable is checked; if the relay is disabled, the message is rejected and sent to the outbound work queue. Next, the SMTP MTA attempts to send an NDR back to the user listed in the from field. In some cases, this architecture can result in large numbers of dead messages being stuck in the MTA. Release 5 and Lotus Notes/Domino 6 are much smarter and will not process the message if the relay is disabled.

Threats to the SMTP hosting server

The hosting server is the device that runs the SMTP service. This can run on virtually any operating system; many enterprise companies use servers based on Windows, Linux, UNIX, AIX, AS400, and others. Each of these has its own set of vulnerabilities. You need to work with your security team to "harden" each of these operating systems.

There are also network vulnerabilities. These include denial of service attacks, TCP/IP attacks, and address spoofing (which we mentioned earlier). The book, "Enterprise Directory and Security Implementation Guide: Designing and Implementing Directories in Your Organization," offers a detailed explanation of the issues and the various types of attacks.

In summary, the server can be a weak point in your SMTP architecture, and you need to protect it. The worst case scenario is that you will lose control of your SMTP server to a spammer or hacker, who then uses that machine to attack your trusted network. A hacker could also obtain access directly to your network and execute a denial of service attack on your trusted network.

The trusted network and the DMZ

A trusted network is the network that a company uses to conduct internal business. In many cases, the trusted network is by default defined in the organization as secure. The trusted network typically supports the back-end systems, internal-only Web pages, data processing, messaging, internal directories, and internal instant messaging.

In many companies, the trusted network allows interaction between internal systems directly without encryption. (We don't recommend this; it's just an observation.) Also, various protocols can exist within the trusted network without any type of filtering or even virus scanning. The problem with this is that many assumptions are being made by these companies. In reality, a trusted network is not always a secure network. In fact, in many cases the trusted network cannot be trusted! The reason is that an internal network is comprised of many different networks. These include new acquisitions, old acquisitions, international access points, and even access points to the outside world.

A common practice is to define the trusted network as the network that internal employees use when at the office or via a secure controlled remote access method (dial-up, IPSec, SSLVPN, RAS, and others). An access point is then established to the outside world via a mechanism called the DMZ (short for demilitarized zone, a military term for a barrier through which no one is supposed to cross without permission). A DMZ is an isolated network placed as a buffer between a company's trusted network and the outside. The primary goal of the DMZ is to limit access to the trusted network.

The DMZ is very important to an SMTP service. The features that a DMZ can provide include:

  • Passive eavesdropping/packet sniffing prevention
  • Application-layer attack prevention
  • Filtering and managing denial of service attacks
  • IP address spoofing prevention
  • Port scans
  • Limiting access to the trusted network via a single protocol
  • Spam management
  • Control of email messages based on size and even content
  • Virus scanning (inbound and outbound)
  • Limiting messages based on Internet domain suffix
  • Network and port translation
  • TCP/IP termination

Figure 3 shows a DMZ in action:

Figure 3. DMZ with SMTP hosted services
DMZ with SMTP hosted services

In the preceding diagram:

  1. A message is ready for routing at a remote server on the Internet.
  2. The remote server looks up the domain suffix of the recipient via the Internet DNS, and then resolves the MX record.
  3. After the address is resolved, the remote server establishes a TCP/IP connection to the target server on port 25 and communicates with the target server using SMTP commands.
  4. After all of the SMTP connection rules have passed and the SMTP service allows the connection, the message is transferred to the SMTP service.
  5. The SMTP service accepts the message on the default port 25. Depending on the service and equipment, it may scan for a virus and/or execute other tests.
  6. If the tests are passed, the SMTP service device transfers the message to the Domino SMTP servers.
  7. The message is transferred from the SMTP service device on port 2222. (We picked this number randomly; you can use virtually any port number higher than well known port ranges.)
  8. The Domino SMTP servers check the connection from the SMTP service device and check the incoming message against any rules it has configured.
  9. If all of the Notes/Domino 6 server rules pass, the message addresses are validated in the Domino Directory and routed via Notes NRPC to the trusted network on a (ideally encrypted) Notes port 1352 connection.
  10. The Domino servers receive the message, and then transfer it as needed to other servers in the Domino domain.

The path out is the same as the one in; the only difference is that, in this example, the SMTP service device performs the MX record lookup.

Point-of-Presence sites

A Point-of-Presence site is a location at which some form of connectivity to the Internet is provided. Smaller organizations typically only have one Point-of-Presence site. Large enterprises may have several, especially those that operate on a global scale. The following diagram illustrates this concept.

Figure 4. Point-of-Presence site
Point-of-Presence site

In the preceding illustration, Location A and Location B are two Point-of-Presence sites connecting the Internet to a company's internal WAN environment. In this case, the locations may be on different continents, or they may be in the same office building. Regardless of the physical distance between them, they have independent Internet connections via independent ISPs.

Having multiple official Point-of-Presence sites allows an organization to provide resilience and load balancing in their mail routing architecture. This affords better usage of resources and protection should one of the Point-of-Presence sites lose connectivity for any reason, such as a major network outage, a problem at the ISP, or a natural disaster. The goal is to provide multiple inbound and outbound paths between the Internet and the internal Domino mail service so that mail can always be routed in either direction. Multiple "unofficial" Point-of-Presence sites within your organization can raise a possible security issue. Each Point-of-Presence site must be maintained and monitored to ensure it does not provide unauthorized entry into your corporate environment. Too many sites (especially ones that are not carefully controlled) may allow a way for hackers to get in.


In this article, we examined the basics of SMTP mail and its potential vulnerabilities. We also introduced you to the concept of the DMZ and how you can use it to address some of these vulnerabilities. In the conclusion of this article series, we'll expand on this theme, looking at ways you can incorporate a DMZ into your Domino SMTP environment.





developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into IBM collaboration and social software on developerWorks

Zone=Lotus, Security
ArticleTitle=Using Notes/Domino SMTP with a DMZ, Part 1