Capacity Planning for WebSphere DataPower B2B Appliance XB60

Answer

EDI documents transferred through the B2B Gateway object are stored on the appliance hard disk and are eventually transferred to a permanent archive. Each B2B Gateway object can have a separate archiving policy, but all B2B Gateway objects together share space on the hard disk. Additionally, metadata that is extracted from each transaction is retained in a separate B2B persistence store on the hard disk.

The archive process removes both copies of messages and message metadata. Archived data cannot be re-imported into the appliance and cannot be viewed through the Transaction Viewer.

B2B payload storage

Up to half of the space on the encrypted portion of the hard drive can be used to store copies of message documents (payload). This space is shared across all B2B Gateway objects. Processing a message generally requires saving a copy of it as it is received "off the wire", plus a copy of the message after AS2 and AS3 headers and any S/MIME compression and encryption have been handled. One or more copies of the MDN might also be saved, particularly if asynchronous MDNs are in use.

Metadata storage

Metadata from each transaction is held on the appliance in the persistence store, which is on the unencrypted part of the hard disk. This space can be shared with configuration and log files if the appliance is otherwise configured to use the local:///ondisk/ directory. The default setup does not use the hard disk for anything besides B2B payload and metadata storage.

The B2B Persistence object can be accessed by device administrators in the default domain. Its "Storage Size" parameter controls the size of the persistence store. The default is 1 GB to allow other use of the hard disk. The size can be safely increased to 64 GB.

Note: After this value is increased, it cannot be decreased. Consider whether you will need to use the appliance hard disk for anything else before increasing the size.

During testing, transaction metadata appears to require about 4 KB per transaction. One transaction here includes an inbound or outbound EDI message. If an outbound message is sent that requests an asynchronous MDN, receipt of that MDN will be processed as a separate transaction that requires separate metadata storage.

Calculating capacity

To properly plan for archiving and the persistence store size, you need to estimate the traffic characteristics for your appliance.

• How many documents do you expect to receive per day?
• How large, on average, do you expect these documents to be?

For illustrative purposes, assume 10,000 EDI messages per day with an average size of 10 KB. Therefore, you are expecting 100 MB of content per day. This capacity requires 360 MB of document storage per day, which equates to approximately 84 days. This corresponds to:
• 200 MB of stored data for these messages. 2 copies of each message -- one copy for "off the wire" and one for the processed request.
• 40 MB (4 KB per message) for protocol responses
• 120 MB for the raw MDN (4 KB per message), HTTP request containing MDN (4 KB per message), and HTTP response containing MDN (4 KB per message)

If the messages per day are split evenly between inbound and outbound and that every outbound message requests an asynchronous MDN, the space for metadata storage adds up to 15,000 transactions per day, which equates to 60 MB (15000 * 4 KB) of metadata storage per day. At the default 1 GB persistence store size, this will fill in approximately 16 days. If the persistence store is increased to 64 GB, over 1,000 days of message metadata can be persisted.

Using the default persistence store size in this example, setting the "Archive Document Age" property to 7 days will remove document metadata from the persistence store before it fills.

WebGUI controls

Each B2B Gateway object has an "Archive" tab. Each gateway can be independently configured to archive documents and metadata off of the appliance before purging or to purge without archiving. Relevant settings on the "Archive" tab include:

Archive Minimum Size -- Archiving will not occur unless this amount of data is stored in copies of message content

Archive Document Age -- Documents that were received at least this many days ago will be archived

Archive Minimum Documents -- Archiving will keep at least this many documents on the appliance after archiving is completed; if fewer than this many documents have been received archiving will not occur

Disk Use Check Interval -- Specifies how often to check for documents to archive, defaults to hourly

Maximum Disk Usage for Documents -- Archiving will occur unconditionally if this amount of data is stored in copies of message content

Commands to monitor space

You can use a pair of commands to monitor the disk space and the size of the persistence store. From the command prompt, type:

xb60# top
xb60# service show b2bp encrypted-space
xb60# service show b2bp size

The "service show b2bp encrypted-space" command shows how much of the 30 GB encrypted disk space the message contents has used.

The "service show b2bp size" command shows how much space message metadata has used in the persistence store.

IBM Redbook

See Chapter 7.3.1 of DataPower SOA Appliance Administration, Deployment, and Best Practices for further details.