Encapsulating Security Payload
The Encapsulating Security Payload (ESP) protocol provides data confidentiality, and also optionally provides data origin authentication, data integrity checking, and replay protection.
The difference between ESP and the Authentication Header (AH) protocol is that ESP provides encryption, while both protocols provide authentication, integrity checking, and replay protection. With ESP, both communicating systems use a shared key for encrypting and decrypting the data they exchange.
If you decide to use both encryption and authentication, then the responding system first authenticates the packet and then, if the first step succeeds, the system proceeds with decryption. This type of configuration reduces processing overhead, as well as reduces your vulnerability to denial-of-service attacks.
Two ways of using ESP
You can apply ESP in two ways: transport mode or tunnel mode. In transport mode, the ESP header follows the IP header of the original IP datagram. If the datagram already has an IPSec header, then the ESP header goes before it. The ESP trailer and the optional authentication data follow the payload.
Transport mode does not authenticate or encrypt the IP header, which might expose your addressing information to potential attackers while the datagram is in transit. Transport mode requires less processing overhead than tunnel mode, but does not provide as much security. In most cases, hosts use ESP in transport mode.
Tunnel mode creates a new IP header and uses it as the outermost IP header of the datagram, followed by the ESP header and then the original datagram (both the IP header and the original payload). The ESP trailer and the optional authentication data are appended to the payload. When you use both encryption and authentication, ESP completely protects the original datagram because it is now the payload data for the new ESP packet. ESP, however, does not protect the new IP header. Gateways must use ESP in tunnel mode.
What algorithms does ESP use to protect my information?
ESP uses a symmetric key that both communicating parties use
to encrypt and decrypt the data they exchange. The sender and the
receiver must agree on the key before secure communication takes place
between them. VPN uses Data Encryption Standard (DES), triple-DES
(3DES), RC5, RC4, Advanced Encryption Standard (AES), or AES-CBC for
encryption.
If you choose the AES algorithm for encryption then you might want to enable Extended Sequence Number (ESN). ESN allows you to transmit large volumes of data at a high speed. The VPN connection uses a 64-bit sequence numbers instead of 32-bit numbers over IPSec. Using 64-bit sequence numbers allows more time before re-keying, which prevents sequence number exhaustion and minimizes the use of system resources.
The Internet Engineering Task Force (IETF) formally defines the algorithms in the following Request for Comments (RFC):
- DES in Request for Comment (RFC) 1829, The ESP DES-CBC Transform
- 3DES in RFC 1851, The ESP Triple DES Transform
- AES-CBC in RFC 3602, The AES-CBC Cipher Algorithm
and Its Use with IPsec
You can view these and other RFCs on the Internet at the following Web address: http://www.rfc-editor.org.
ESP uses
HMAC-MD5, HMAC-SHA, HMAC-SHA-256, and AES-XCBC-MAC algorithms to provide
authentication functions. Each of the algorithms take variable-length input
data and a secret key to produce fixed-length output data (called
a hash or MAC value). If the hashes of two messages match, then it
is likely that the messages are the same.
The Internet Engineering Task Force (IETF) formally defines the algorithms in the following Request for Comments (RFC):
- HMAC-MD5 in RFC 2085, HMAC-MD5 IP Authentication with Replay Prevention
- HMAC-SHA in RFC 2404, The Use of HMAC-SHA-1-96 within ESP and AH
- HMAC-SHA_256 in RFC 4868, Using HMAC-SHA-256,
HMAC-SHA-384, and HMAC-SHA-512 with IPsec
- AES-XCBC-MAC in RFC 3566, The AES-XCBC-MAC-96 Algorithm and
Its Use With IPsec
You can view these RFCs on the Internet at the following Web address: http://www.rfc-editor.org.
