Wrapping up my week on All-Flash arrays, I thought I would cover some of the Enterprise Reliability features of the IBM FlashSystem.
On Monday, [IBM FlashSystem versus EMC XtremeIO all-Flash Arrays], I discussed some of the features of the IBM FlashSystem that differentiate it from EMC's ExtremeIO and other all-Flash arrays. On Tuesday, [IBM 2013 Storage Announcements for November 19] included discussion of the all-Flash model of the IBM System Storage DS8870 disk system.
Just as light bulbs burn out eventually after repeatedly being turned on and off, Flash does not last forever either.
A set of transistors can represent a single bit of informaiton (Single-level cell, or SLC for short), or multiple bits (Multi-level Cell, MLC). MLC typically refers to two bits, with a new "Triple-level cell" or TLC technology, able to store three bits per set of transistors.
SLC is faster and can endure more "Program-erase" write cycles, but MLC is cheaper and therefore used in most consumer products, like digital cameras, smart phones, music players and USB memory sticks. To learn more on this, see this 6-page IBM whitepaper on [Comparison of NAND Flash Technologies Used in Solid-State Storage].
As a compromise, "Enhanced MLC" (or eMLC for short) uses clever algorithms to increase the endurance, such as wear-leveling algoritms, while maintaining lower cost.
In the IBM FlashSystem, DRAM cache is used to buffer the writes first, then written out to the Flash. This helps to increase the endurance. As a result, eMLC has 10x the endurance of regalar MLC, and SLC is three times better than eMLC.
For enterprise reliability, each Flash chip on the IBM FlashSystem has Error Correcting Codes (ECC), and then each set of 10 chips is placed in a 9+P RAID-5 configuration.
The chips are sub-divided into 16 planes. In the event a cell fails, the data for that plane can be reconstructed from parity, and written to spare space on the other planes of that same chip set. That plane is then reformated as an 8+P RAID-5, bypassing the failed plane.
In this manner, a cell failure only results in losing a small portion of one chip. If the same plane fails another failure on another chip, it will drop down to 7+P, 6+P, 5+P, and finally 4+P. This is known as "Variable Stripe RAID" or VSR for short.
IBM FlashSystem can survive over 1,000 such cell failures without an outage. By comparison, a single cell failure on an SSD often marks the entire drive as a failure.
But wait, there's more. Why stop at just RAID-5 across 10 chips. The chips are organized into modules, and IBM FlashSystem can perform RAID-5 across modules, in a 10+P+S RAID-5 configuration. This is referred to as "Two dimensional RAID" or 2D-RAID for short.
Even if you lost an entire module, the system will automatically rebuild on the spare module, and you can replace the bad one non-disruptively.
Many use cases for all-Flash arrays do not require such high levels of Enterprise reliability. Several of the all-Flash competitors have adopted a "design-for-failure" approach common among Cloud Service Providers like Amazon Web Services.
The idea is to assume that the data stored on them is just a copy from some other storage media. In the event of a Flash failure, it can easily be restored from a mirrored copy or backup.
For the IBM FlashSystem, the 700 series are based on SLC, with higher endurance for write-intensive workloads. The 800 series are based on eMLC, for more read-intensive workloads. Typical business applications, databases and virtual machine images placed on all-Flash arrays have 70 percent read ratio or higher.
Within each series, the "tens" models (710, 810) offer RAID-0 striping across ECC and VSR protected modules. For higher levels of availability, the "twenties" models (720, 820) offer ECC, VSR and 2D-RAID protection.
technorati tags: IBM, FlashSystem, SSD, Flash, endurance, high availability, enterprise reliability, SLC, MLC, eMLC, ECC, VSR, 2D-RAID, EMC, XtremeIO, DS8870