ProtecTier and Domino: How to get your de-dup back
anthonyv 2000004B9K Visits (4695)
The IBM ProtecTIER performs in-line de-duplication of your backup data, enabling much faster backups and much faster restore times. De-duplicating your backups allows you to store a lot "more on the floor".
One of the advertised capabilities of ProtecTIER is that you can get a de-duplication ratio of up to 25 to 1. This sounds great, but advertising this sort of ratio is a blessing and a curse. On the one hand it shows the potential capability of the device, but it can also create very high expectations. In reality the ratio you will achieve is tota
Backing up Lotus Domino databases is a good case in point. When backing up your Lotus Notes databases you may only see a 2.5 :1 dedup ratio, which is clearly not a good result. The issue may well be with a function called compaction. Compaction re-arranges all of the data contained in the NSF (Notes Storage Format) databases to reclaim space. While this function helps to reduce space utilization from the perspective of Lotus Domino, it also changes the layout and data pattern of every single NSF. So the next time ProtecTIER receives blocks from these databases, they all look unique, so the de-dup ratio naturally ends up being very low. However running compaction is a best practice for Lotus Domino, so disabling it is not a solution.
The solution involves using a tool called DAOS (Domino Attachment and Object Service), which removes all the email attachments from the NSF files and stores them separately. This not only provides substantial space savings for Domino (because it only stores each unique attachment once) but also means that compaction can still run on the NSF files (which are now attachment free). The result at one customer? The combined de-dup ratio went up to 8.5:1 (which was about 2.5:1 on the NSF but almost 20:1 on the attachment files).
The only caveat is that Lotus Domino needs to be at version 8.5 to use DAOS. More information on performing backups with DAOS can be found here.
Thanks to Francois Morin for sharing this with me.