This blog is for the open exchange of ideas relating to IBM Systems, storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
Tony Pearson is a Master Inventor, Senior IT Architect and Event Content Manager for [IBM Systems for IBM Systems Technical University] events. With over 30 years with IBM Systems, Tony is frequent traveler, speaking to clients at events throughout the world.
Lloyd Dean is an IBM Senior Certified Executive IT Architect in Infrastructure Architecture. Lloyd has held numerous senior technical roles at IBM during his 19 plus years at IBM. Lloyd most recently has been leading efforts across the Communication/CSI Market as a senior Storage Solution Architect/CTS covering the Kansas City territory. In prior years Lloyd supported the industry accounts as a Storage Solution architect and prior to that as a Storage Software Solutions specialist during his time in the ATS organization.
Lloyd currently supports North America storage sales teams in his Storage Software Solution Architecture SME role in the Washington Systems Center team. His current focus is with IBM Cloud Private and he will be delivering and supporting sessions at Think2019, and Storage Technical University on the Value of IBM storage in this high value IBM solution a part of the IBM Cloud strategy. Lloyd maintains a Subject Matter Expert status across the IBM Spectrum Storage Software solutions. You can follow Lloyd on Twitter @ldean0558 and LinkedIn Lloyd Dean.
Tony Pearson's books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
The developerWorks Connections Platform is now in read-only mode and content is only available for viewing. No new wiki pages, posts, or messages may be added. Please see our FAQ for more information. The developerWorks Connections platform will officially shut down on March 31, 2020 and content will no longer be available. More details available on our FAQ. (Read in Japanese.)
Well, it's Tuesday, which means IBM makes its announcements!
This week, IBM announces that it now supports 50GB Solid State Disk (SSD) in its [IBM System Storage EXP3000] disk systems.IBM has already made announcements about SSD enablement in the DS8000 and SAN Volume Controller (SVC), but now the EXP3000 brings SSD technology down to smaller System x server deployments.
Adoption of this new exciting technology is still in the early stages, despite the fact that IBM and other vendors have been touting this technology for a while. (For a quick blast to the past, here was my first post on the subject back from December 20, 2006: [Hybrid, Solid State and the future of RAID])Recently, fellow blogger BarryB admitted that EMC have only sold SSD to [hundreds of their customers], and to be fair, I suspect IBM's sales of SSD in its BladeCenter servers [available since July 2007] have been in similar single-digit percentage territory as well.
The advantage of today's announcement is that you can mix and match SSD drives with SAS and SATA drives in the EXP3000. You won't have to buy the entire drawer of SSD, you can start with just a few, depending on your business needs. On the other extreme, you can have up to two drawers, with 12 SSD drives each, for a total of 24 drives directly attached to System x servers via the ServeRAID MR10M SAS/SATA controller adapter.
Well it's Tuesday again, and you know what that means.. IBM announcements! Today, IBM announces that next Monday marks the 60th anniversary of first commercial digital tape storage system! I am on the East coast this week visiting clients, but plan to be back in Tucson in time for the cake and fireworks next Monday.
1925 - masking tape (which 3M sold under its newly announced Scotch® brand)
1930 - clear cellulose-based tape (today, when people say Scotch tape, they usually are referring to the cellulose version)
1935 - Allgemeine Elektrizitatsgesellschaft (AEG) presents Magnetophon K1, audio recording on analog tape
1942 - Duct tape
1947 - Bing Crosby adopts audio recording for his radio program. This eliminated him doing the same program live twice per day, perhaps the first example of using technology for "deduplication".
According to the IBM Archives the [IBM 726 tape drive was formally announced May 21, 1952]. It was the size of a refrigerator, and the tape reel was the size of a large pizza. The next time you pull a frozen pizza from your fridge, you can remember this month's celebration!
When I first joined IBM in 1986, there were three kinds of IBM tape. The round reel called 3420, and the square cartridge called 3480, and the tubes that contained a wide swath of tape stored in honeycomb shelves called the [IBM 3850 Mass Storage System].
My first job at IBM was to work on DFHSM, which was specifically started in 1977 to manage the IBM 3850, and later renamed to the DFSMShsm component of the DFSMS element of the z/OS operating system. This software was instrumental in keeping disk and tape at high 80-95 percent utilization rates on mainframe servers.
While visiting a client in Detroit, the client loved their StorageTek tape automation silo, but didn't care for the StorageTek drives inside were incompatible with IBM formats. They wanted to put IBM drives into the StorageTek silos. I agreed it was a good idea, and brought this back to the attention of development. In a contentious meeting with management and engineers, I presented this feedback from the client.
Everyone in the room said IBM couldn't do that. I asked "Why not?" The software engineers I spoke to already said they could support it. With StorageTek at the brink of Chapter 11 bankruptcy, I argued that IBM drives in their tape automation would ease the transition of our mainframe customers to an all-IBM environment.
Was the reason related to business/legal concerns, or was their a hardware issue? It turned out to be a little of both. On the business side, IBM had to agree to work with StorageTek on service and support to its mutual clients in mixed environments. On the technical side, the drive had to be tilted 12 degrees to line up with the robotic hand. A few years later, the IBM silo-compatible 3592 drive was commercially available.
Rather than put StorageTek completely out of business, it had the opposite effect. Now that IBM drives can be put in StorageTek libraries, everyone wanted one, basically bringing StorageTek back to life. This forced IBM to offer its own tape automation libraries.
In 1993, I filed my first patent. It was for the RECYCLE function in DFHSM to consolidate valid data from partial tapes to fresh new tapes. Before my patent, the RECYCLE function selected tapes alphabetically, by volume serial (VOLSER). My patent evaluated all tapes based on how full they were, and sorted them least-full to most-full, to maximize the return of cartridges.
Different tape cartridges can hold different amounts of data, especially with different formats on the same media type, with or without compression, so calculating the percentage full turned out to be a tricky algorithm that continues to be used in mainframe environments today.
The patent was popular for cross-licensing, and IBM has since filed additional patents for this invention in other countries to further increase its license revenue for intellectual property.
In 1997, IBM launched the IBM 3494 Virtual Tape Server (VTS), the first virtual tape storage device, blending disk and tape to optimal effect. This was based off the IBM 3850 Mass Storage Systems, which was the first virtual disk system, that used 3380 disk and tape to emulate the older 3350 disk systems.
In the VTS, tape volume images would be emulated as files on a disk system, then later moved to physical tape. We would call the disk the "Tape Volume Cache", and use caching algorithms to decide how long to keep data in cache, versus destage to tape. However, there were only a few tape drives, and sometimes when the VTS was busy, there were no tape drives available to destage the older images, and the cache would fill up.
I had already solved this problem in DFHSM, with a function called pre-migration. The idea was to pre-emptively copy data to tape, but leave it also on disk, so that when it needed to be destaged, all we had to do was delete the disk copy and activate the tape copy. We patented using this idea for the VTS, and it is still used in the successor models of IBM Sysem Storage TS7740 virtual tape libraries today.
Today, tape continues to be the least expensive storage medium, about 15 to 25 times less expensive, dollar-per-GB, than disk technologies. A dollar of today's LTO-5 tape can hold 22 days worth of MP3 music at 192 Kbps recording. A full TS1140 tape cartridge can hold 2 million copies of the book "War and Peace".
(If you have not read the book, Woody Allen took a speed reading course and read the entire novel in just 20 minutes. He summed up the novel in three words: "It involves Russia." By comparison, in the same 20 minutes, at 650MB/sec, the TS1140 drive can read this novel over and over 390,000 times.)
If you have your own "war stories" about tape, I would love to hear them, please consider posting a comment below.
We have a new member of the ever-growing IBM Spectrum Storage family! IBM Spectrum Discover is modern metadata management software that delivers data insight for petabyte-scale, unstructured data.
IBM Spectrum Discover easily connects to IBM Cloud Object Storage (COS) and IBM Spectrum Scale and Elastic Storage Server (ESS) to rapidly ingest, consolidate, and index metadata for billions of files and objects, providing a rich layer of metadata on top of these storage sources. IBM plans to extend support to other platforms next year.
This metadata enables data scientists, storage administrators, and data stewards to efficiently manage, classify, and gain insights from massive amounts of unstructured data. The insights gained accelerate large-scale analytics, improve storage economics, and help with governance to create competitive advantage, speed critical research, and mitigate risk.
This initial release is labeled v2.0 as IBM has deployed this in beta form already at various client locations. Here are some key highlights:
Event-notifications and policy-based workflows to automate metadata ingestion and metadata indexing at a petabyte scale
Fine-grained views of storage consumption based on a wide range of system and custom metadata
Fast, efficient search through petabytes of data, resulting in highly relevant results for large-scale analytics
Ability to quickly differentiate mission-critical business data from data that can either be deleted or moved to a cheaper, colder tier
Policy-based custom tagging that enables organizations to classify and categorize data, and align this data with the needs of the business
A software developers kit (SDK) to build action agents that extract metadata from file headers and content, automate data movement, and provide integration to open source software, such as Apache Spark, Apache Tika, PyTorch, Caffe and TensorFlow, to facilitate data identification and speed large-scale data processing
The latest IBM FlashSystem 900 comes in two models, the AE3 "full purchase" model, and the UF3 "storage utility pricing" model where you pay less initially, and then more as you consume more of the capacity. They are the same hardware, just licensed differently.
Currently, IBM offers FCP or InfiniBand host attachment, with up to twelve 3.6TB, 8.5TB or 18TB modules (PCiE card). A full 2U drawer would be configured as 10+P+S RAID5 for high availability and data protection.
Each module offers embedded compression chip, but modules only had enough DRAM cache to allow a maximum of compressed 22TB effective data, so while the 3.6TB and 8.5TB could compress data up to 2.5x, the 18TB card was somewhat limited at 1.2x, which might be fine for some already-compressed data like MP3 audio, or JPEG photos.
This month, IBM offers new XL MicroLatency Modules, 18TB cards with enough DRAM cache to support 44TB compressed data, up to an effective 2.4x compression ratio. A full twelve-module drawer could hold up to 440TB of effective capacity.
IBM also now offers a quad-port 16Gb FCP card that supports both SCSI and NVMe commands over fabric. This is often denoted as either FC-NVMe or NVMe/FC. The FlashSystem 900 already supported NVMe-OF for InfiniBand (see my blog post [IBM February 2018 Announcements])
IBM Cloud Tape Connector for z/OS is a software-defined storage solution that provides an alternative to virtual tape libraries like the TS7760. Here are some highlights:
Robust virtual tape emulation solution with e-vaulting to cloud-based offsite storage for cold, archival, or backup data. Virtual tape emulation simulates IBM compatible tape controllers, tape drives, and tape volumes, maintained on any IBM z/OS-compatible disk system, such as IBM DS8000. IBM Cloud Tape Connector for z/OS provides several vault, transfer, and recovery options to support business continuity and resiliency.
Sequential z/OS data set cloud storage and retrieval. Sequential data sets stored on disk or flash storage can be moved to the cloud by IBM Cloud Tape Connector for z/OS without the requirement of performing a tape-write operation.
Automatic application recall of data from cloud, whether e-vaulted through virtual tape emulation or copied directly to the cloud.
Pervasive encryption support. This feature enables enterprises to ensure that any data copied to the cloud is encrypted before it is transmitted, automatically protecting and handling the encryption keys.
Support for IBM Cloud Object Storage using S3 protocol, as well as Amazon S3, Hitachi HCP protocol, and EMC Elastic Cloud Service Protocol.
Well, it's Tuesday again, and you know what that means? IBM announcements!
Today's announcements are all about the Storwize family, IBM's market-leading Software Defined Storage offerings. Having sold over 55,000 systems, and managing over 1.6 Exabytes of data, IBM continues to be the #1 leader in storage virtualization solutions. The Storwize family consists of the SAN Volume Controller (SVC), Storwize V7000, Storwize V7000 Unified, Flex System V7000, Storwize V5000, Storwize V3700 and V3500.
SAN Volume Controller 2145-DH8
The new 2145-DH8 model is a complete repackaging of this popular storage system. The previous model, the 2145-CG8, was 1U-high x86 server per node, and each node required a separate 1U-high UPS to provide battery protection for its cache. Nobody liked this. The new 2145-DH8 instead is a 2U-high node with two hot-swappable batteries, eliminating the need for UPS altogether. Thus, an SVC node-pair using the 2145-DH8 models takes up the same 4U space, but with fewer cables. The SVC can now also support standard office 110/240 voltage sources.
The new model sports an 8-core processor with 32GB RAM. Since these are 2-socket servers, IBM offers that option to add a second 8-core processor and additional 32GB RAM to help boost Real-time Compression. Each node can have optionally one or two hardware-assisted compression cards which use the Intel QuickAssist chip to boost compression performance.
While the Real-time Compression was in fact, real-time, performed in-line to the read/write I/O process, at latency comparable to uncompressed data for applications, the compression process on older models was entirely software-based, consuming some of the CPU resources, which lowered the maximum IOPS of the solution. With the added cores, added RAM, and hardware-assisted compression chips, IBM resolves that concern. In fact, the new 2145-DH8 with compression can provide more IOPS than an older 2145-CG8 without compression.
The previous model 2145-CG8 allowed you to put up to 4 small SSD drives in the node itself, which were treated the same as externally Flash drives for purposes of having a high-speed storage pool for select volumes, or automated sub-LUN tiering with Easy Tier. The new model 2145-DH8 allows you to attach up to 48 Solid State Drives (SSD) via 12Gb SAS cables. These are housed in the new 2U-high 24F enclosures that can offer up to 38.4 TB of Flash per SVC I/O group.
IBM also re-designed the host/device ports to use Hardware Interface Card (HIC) slots. In the 2145-CG8, you had four FCP ports, two 1GbE Ethernet ports, with options to add two 10GbE Ethernet ports or four additional FCP ports. If you had mostly an FCoE or iSCSI environment, you didn't need the FCP, and if you were mostly a FCP Storage Area Network (SAN) environment, then most of the Ethernet ports went unused. To solve this, the 2145-DH8 can allow you to have up to six HIC cards that are either FCP, Ethernet, or SAS. There are three 1GbE fixed Ethernet ports which can be used for iSCSI and administration.
If you have SVC today, you can upgrade non-disruptively by either swapping out your current SVC engines with the new 2145-DH8 engines, or you can add the new 2145-DH8 engines to your existing SVC cluster. Either way, there is no outage to your applications!
This is the next generation of the popular Storwize V7000. The previous generation had a 4-core processor and 8GB RAM per canister. The new model has an 8-core processor with 32GB of RAM per canister, with the option to double these to boost Real-time compression. There are two canisters per control enclosure, which gives you 64GB to 128GB of RAM per Storwize V7000 I/O group.
The new Storwize V7000 comes with one hardware-assisted compression chip on the mother board of each canister, with the option to add a second chip per canister.
Each canister offers three HIC slots, which can be used for the additional hardware-assist compression chip, FCP or Ethernet ports.
To accommodate these HIC slots, new canisters were needed. Instead of the flat wide style top and bottom, we now have taller, thinner canisters that sit side to side. This side-to-side design is similar to our existing Storwize V5000 and V3700 models.
The previous model could support up to 9 expansion enclosures per control enclosure. The Storwize V7000 can have up to 24 drives in its control enclosure, and now attach up to 20 expansion enclosures, which allows up to 504 drives per control enclosure, and up to a maximum of 1,056 drives per Storwize cluster.
If you have previous models of Storwize V7000, you can add the new Storwize V7000 into the same cluster, or virtualize the previous storage for migration purposes.
The new software applies new capabilities to both new generation hardware as well as the older models, so people with existing gear can benefit as well.
In prior releases, the sub-LUN automated tiering was limited to two levels: Flash and HDD. This lumped all 15K, 10K and 7200 RPM drives into a common HDD category. In the new v7.3.0 code, you can now have three levels: Flash, Enterprise HDD, and Nearline HDD, or two HDD levels: Enterprise and Nearline. The Enterprise level combines 15K and 10K RPM drives, similar to what is done on the IBM System Storage DS8000 disk systems.
The new code is also able balance your storage pools, and can be used with uniform or mixed storage pools to eliminate performance hot spots.
The new code has been enhanced to detect the hardware-assisted compression chip on the new SVC and Storwize V7000 models, and use those if available.
For the Storwize V3700 and V5000 models, the new code allows up to nine expansion enclosures per control enclosure. In the previous models, the V3700 allowed only four expansions, and the V6000 only six expansions per control enclosure. The V3700 can now support up to 240 drives, and the V5000 can support up to 480 drives.
IBM Storwize V7000 Unified File Module software v1.5
For Storwize V7000 Unified clients, there is new software for the File Modules that provide NFS, CIFS, FTP, HTTPS and SCP protocol capability. The new v1.5 code now adds NFS v4 and SMB 2.1 levels of support. Most NFS users are still on NFSv3, but about 20 percent of NFS users are using NFS v4 which offers stateful access. The SMB 2.1 for CIFS was introduced by Microsoft in Windows 7 and Windows Server 2008 R2.
Deterministic ID mapping allows you to map Windows userids to UNIX/Linux group and owner id numbers. In the past, the problem is that this mapping is different on each machine, so people often had to stand up a Windows System for Unix Services (SFU) server to provide consistent ID mapping. Now, with v1.5 code, you will no longer have to do this. The deterministic ID mapping will can now replicate the mapping to each machine without an SFU server.
Active Cloud Engine allows up to ten Storwize V7000 Unified to be connected across distance to form a single global name space. WAN caching, however, was restricted to a single site having write capabilities, while the others were read-only. In v1.5 release, IBM now supports multiple independent writers at different locations on the same fileset.
Security enhancements include multi-tenancy, configurable password policies, session policies, and hardened boot and SSH configurations. With NFS v3/v4, you can now use [Kerberos] for security.
Finally, I am please to see that we now have Cinder support for files on the Storwize V7000 Unified on the OpenStack Havana release that just came out last month. The OpenStack Cinder interface can assign LUNs to virtual machines, but the new Havana release allows NAS systems to dole out files that act as LUNs, such as OVA or VMDK files. The advantage is that these files can managed by Active Cloud Engine, cached locally across global name space, have policies place them on appropriate storage tiers, and inactive Virtual Machine images can be migrated to less expensive disk or tape.
When IBM designed the DS8870, it changed the bulk power supplies and batteries used in the previous DS8800 model to highly energy-efficient [DC-UPS]. In addition to reducing the overall energy consumption of the DS8870, it also gave the engineers space above the units to put 4U of standard 19-inch rack equipment.
The High Performance Flash Enclosure provides an ultra-dense and ultra-high-performance option. Each HPFE can delivers up to 250,000 IOPS and up to 3.4 GB/s bandwidth.
Up to thirty 387 GB Enterprise Multi-Level Cell (eMLC) flash cards provide up to 11.6 TB of raw capacity, about 9.2 TB usable, in only 1U of 19-inch rack space. A pair of very powerful integrated SAS RAID engines manage RAID-5 across the flash cards. The HPFE attaches directly to GX++ slots in the two DS8870 POWER7+ controllers, rather than using the Device Adapter (DA) loops.
You can have up to four of these HPFE in the "A" frame of your DS8870. Each HPFE can have either 16 or 30 flash cards. For 16 cards, you would have two spare plus two 6+P RAID-5 ranks. For 30 cards, you would add another two 6+P RAID-5 ranks.
Easy Tier Enhancements
Easy Tier is IBM's market-leading sub-volume automated tiering inside the DS8870 disk system. There were several enhancements in this announcement.
The first enhancement is to "Easy Tier Server", a feature that coordinates caching of active blocks of data inside the server's own internal Flash. This had supported Power Systems with EXP30 Ultra drawers, and now expanded to support IBM [Flash Adapter 90].
The second enhancement is to the three-level (Flash,Enterprise,Nearline) tiering algorithm. Inside the DS8870, the new HPFE flash cards will be part of the "Flash Tier" along with solid state drives (SSD) attached to the DA loops. Internal inter-tier load-balancing will take into account the faster nature of the flash cards in the HPFE, and move the busiest blocks accordingly. We we refer to this as "micro-tiering" within the Flash Tier.
Broader Solid State Drive options
Not everybody likes the 400GB solid state drives IBM offered for the DA loops, so IBM is now offering a smaller 200GB and a larger 800GB options as well.
Enhanced Concurrent Code Load
The new DS8870 R7.3 firmware release drastically cuts the activation time of concurrent code load in half.
Nobody likes warmstarts either. These are a necessary evil for some error conditions, but the clever engineers upstairs have figured out ways to reduce the number of warmstarts and eliminate the need to perform a warmstart after certain events to prevent any application impact to the attached host.
Multi-Target Remote Mirror
By now you know that IBM has the market-leading remote mirroring services for high-end disk systems, using less bandwidth and maintaining better concurrency than high-end systems from other vendors.
The DS8870 R7.2.7 firmware release can now support multi-target remote mirror. In previous releases, if you wanted three-site disaster recovery, you relied on Metro/Global Mirror, where site "A" had a Metro Mirror to a bunker site "B", and then site "B" had a Global Mirror to site "C". Not everybody liked this.
Some clients have asked for a "star" configuration, where "A"-to-"B" and "A"-to-"C" are independent of each other. A SCORE request is available for the following configurations:
Two Metro Mirror
One Metro Mirror and one Global Copy
Two Global Copy
While Metro Mirror can support up to 300km distance, and Global Copy can go any distance around the planet, there is no reason why you can't have one or both copies in the same building, or on campus nearby, for use with HyperSwap.
OpenStack Cinder interface support
Last but not least, the DS8870 now offers full support for OpenStack Havana and Icehouse releases. Support is provided through the OpenStack Cinder driver currently available for download. IBM is a platinum sponsor of the OpenStack foundation.
The new TS1155 enterprise tape drive can write up to 15 TB uncompressed data to existing JD/JZ/JL media.
It can read/write existing 10TB-formatted JD media, and 7TB-formatted JC media, written by former TS1150 drives. It also can offer read-only support for older 4TB-formatted JC media from TS1140 drives.
These are uncompressed capacities, and some clients achieve 2x or 3x compression on top of these capacities. This depends heavily on the type of data. Your mileage may vary, as they say.
Most of the rest of the features of the TS1150 drives carry forward., The performance 360 MB/sec is similar, encryption via IBM Security Key Lifecycle Manager (SKLM) is similar, and support for IBM Spectrum Archive via Linear Tape File System (LTFS) format is similar.
An interesting development is that the TS1155, in addition to standard 8Gb Fibre Channel attach, is the first IBM enterprise drive to also offer 10Gb Ethernet support. IBM will offer both RDMA over Converged Ethernet (RoCE) as well as iSCSI support.
The newest member of the IBM Spectrum Storage software family, IBM Spectrum Copy Data Management automates the creation of snapshot images (FlashCopy for those familiar with IBM terminology) on IBM, NetApp and EMC storage arrays. These copies can be made for various uses, such as DevOps, Dev/Test, Backup/Restore, and Disaster Recovery.
At some data centers, these copies can consume as much as 60 percent of your total storage space, because often each developer and tester are generating their own copies. Instead, having copies automated, registered, cataloged, and made available to developers and testers eliminates rogue copies.
This release adds support for additional databases, including Microsoft SQL Server on physical machines, SAP HANA in-memory databases, and Epic/Caché from InterSystems used in Electronic Health Records (EHR) management systems.
IBM also adds support for long-distance Vmotion for VMware virtual machine images. The target for this movement is IBM Spectrum Accelerate running on IBM Bluemix Cloud, supporting Hybrid Cloud configurations.
Well, it's Tuesday again, and you know what that means? IBM Announcements! This week I am in San Francisco, California speaking to clients. A bit colder than Tucson, Arizona!
(FTC Disclosure: I work for IBM. Special thanks to Mark Larson (IBM SAN team), and both Craig Nelson and Peter Schmelter from Broadcom, for their assistance with this post. I have no personal financial interest in Broadcom. This blog post can be considered a "paid celebrity endorsement" of the IBM products mentioned below.)
Spectrum Control v5.3
Back in 2003, I was the chief architect of Spectrum Control v1, formerly called TotalStorage Productivity Center, and later Tivoli Storage Productivity Center. IBM Spectrum Control is part of the IBM Spectrum Storage Suite.
There are two editions: Standard Edition and Advanced Edition.
(What happened to the other editions? The "Base Edition" is now called IBM Spectrum Connect. The "Spectrum Control Storage Insights" service in the IBM Cloud is now just called IBM Storage Insights and Storage Insights Pro.)
The Standard Edition v5.3 offers the following:
Capacity visualization and management, Performance troubleshooting, Health and performance alerting, Application modeling, and support for VMware data sources
Create, save, and send reports directly in the web UI. The reports can be run now, or scheduled to be run later. When a report is run, it can be sent by email or exported and saved in different file types.
Support IBM FlashSystem 900 AE3 models using compression, and the new IBM FlashSystem 9100
Improved automation of counting the licenses for enclosure-based storage devices
The latest IBM Copy Services Manager (CSM) v6.2 for managing remote mirroring, replacing the previous IBM Tivoli Storage Productivity Center for Replication.
The Advanced Edition v5.3 provides all of the above, as well as the following.
Tiered storage optimization with intelligent analytics
Service catalog with policy-based provisioning
Self-service provisioning with restricted use logins
Analysis of reclaimable space
Showback and Chargeback reports
Application-based snapshot management using IBM Spectrum Protect Snapshot (formerly known as IBM FlashCopy Manager, FCM)
Clients with v5.2.x version of IBM Spectrum Control can upgrade to this new release.
Clients with IBM Spectrum Virtualize-based appliances can bundle Spectrum Control v5.3 with the latest Spectrum Virtualize v8 code. This bundle is referred to as "IBM Virtual Storage Center", or VSC for short. VSC supports SAN Volume Controller, FlashSystem 9100 and V9000, Storwize V7000 and V5000 models.
IBM's announcement of NVMe-capable FlashSystem 9100 has caused many to re-evaluate their SAN infrastructure. All IBM b-type Gen5 and Gen6 switches and directors are NVMe-ready!
(Last year, Broadcom completed its acquisition of Brocade. I am thankful both start with the letter "B", so we won't have to rename our B-type switches to another letter!)
There are two new products in this announcement. The SAN 128B-6 is a Gen6 switch in a 2U container. The other is a 64-port Blade that fits into existing Gen6 Directors, like the 256B-6 or 512B-6 models.
But the 128B-6 doesn't have 128 standard ports ! It actually has 96 standard ports, plus eight "Q-Flex" ports (that can be used to create a total of 128 ports) . Likewise, the 64-port blades have 16 Q-Flex ports (that can be used to create 64 ports).
What is going on? The Q-Flex ports can actually run four channels in different colors of light over the same fiber optic cable, reducing the wiring mess. These Q-Flex can be used for host or device traffic, but are often used as "Inter-Switch Links" or ISL for short.
All of the standard and Q-Flex ports are 32Gbps, but can are capable of autosensing 4, 8, 16, and 32 Gbps port speedsm depending on the SFPs used , for interoperability with existing servers and storage devices. In the case of Q-Flex, all four colors must be run at the same speed, so a Q-Flex represents either 4x32, 4x16, 4x8 or 4x4 Gbps links. You cannot mix different speeds on a single Q-Flex.
In addition, the 64-port blade also supports 10 GbE, 25 GbE, and 40 GbE using the appropriate QSFP transceivers.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
Here is a quick recap of the October 9, 2018 announcements this week.
IBM Elastic Storage Server V5.3.2
The new IBM Elastic Storage Server v5.3.2 offers support for new drawers, non-disruptive upgrades of older models, and an optional 100GbE switch.
When the ESS was first announced, we had GSx models and GLx models, where x represented the number of storage drawers. The "S" stood for small 2U-24 drive drawers, so for example the GS4 had two Power8 servers combined with four 2U-size flash SSD drawers. The "L" stood for large 4U-60 drive nearline HDD drawers.
The second generation models append "S" for Second, so we had GS4S and GL6S. The large models changed to larger 5U-84 drive drawers. As with the previous "L" models, two slots per system contain Solid State Drives for internal use and caching, leaving the rest for slower spinning HDD disk.
Before this week, to upgrade from one model to another meant moving the data off, installing and configuring the additional drawers, and then move the data back. With today's announcements, you can now non-disruptively upgrade GS1S to GS2S to GS4S models, and GL1S to GL2S to GL4S to GL6S.
While you can federate as many GS and GL models together, that may mean having to spend more for Power8 servers than you are comfortable with, so IBM added "GHxy" Hybrid models, with x 2U-24 drive drawers, and y 5U-84 drive drawers. Initial models included the GH14 and GH24, which had one or two flash drawers, and four large drawers. This week, IBM announced a new GH12 model. The SSD flash in the 2U drawer can be 3.84TB or 15.36TB, and the nearline drives in the 5U drawers can be 4TB, 8TB or 10TB capacities.
What did IBM call the third generation GL models? Instead of using "T" which is both the next letter in the alphabet after "S", and the initial letter of the word "third", IBM instead decided to use "C" to designate CORAL project, the Collaboration of Oakridge, Argonne, and Lawrence Livermore national labs. Since the change applied only to the GL models, not the GS models, this makes sense.
To meet the requirements to build the world's fastest supercomputer for the CORAL project, IBM created a modified Elastic Storage Server model with 4U drawers that contained 106 drives. Now, these are available to the general public! IBM announced GL1C, GL2C, GL4C and GL6C models. In these, there are 2 SSD drives, and the rest are 10TB nearline drives.
The new optional 100GbE switch has 32 ports with a total of 6.4 Tbps. These can support 10, 40, 50 and 100GbE data rates, with 300 nsec latency for 100 GbE port to port
Spectrum Scale is licensed two ways: Standard Edition based on the number of sockets, with different prices for NSD servers, FPO servers and NSD clients; and the "Data Management" edition which offered advanced features, and was based on capacity of NSD, independent of the number of servers and clients attached.
Clients liked the capacity-based license model, but did not necessarily need the advanced features. In response, IBM now offers the "Data Access" edition, which offers the same features and functions of Standard Edition, but with capacity-based licensing.
For ESS models, you can chose to license by disk as before, or by capacity in combination with Spectrum Scale capacity-based deployments.
Hortonworks Data Platform v3.0.1 has followed suit. With the merger between Hortonworks and Cloudera, Hortonworks now offers capacity-based licensing for shared storage, like the IBM Elastic Storage Server.
IBM FlashSystem A9000/A9000R software version 12.3
There are three enhancements in this release: Three-site replication, a new model of A9000R, and raising a previous pool size limit.
For three-site replication, you can now combine HyperSwap which maintains two identical copies at distance, with a third asynchronous mirroring. The first two are typically within 100 km, but the third copy can be a much greater distance, across the continent if you like.
The A9000 "Pod" had three x86-based controller and one FlashCore drawer. The A9000R "Rack" had four, six or eight x86-based controllers and two, three or four FlashCore drawers, respectively, as well as a Power Distribution Unit (PDU) and pair of InfiniBand switches to connect everything together. The new "Grid Starter" model is very much like the "Pod" with three controllers and one FlashCore drawer, but adds the PDU and IB switches. The idea is that you can start with a "Grid Starter", then later upgrade to the larger A9000R models as you grow.
Back in XIV days, the architectural limit per pool of 1PB was plenty big. But with the new capacities on the A9000 and A9000R, the 1PB limit was starting to draw complaints. This limit was lifted, so that now a single pool can be made with the entire capacity of the box.
In the mainframe world, IBM Geographically Dispersed Parallel Sysplex, now just GDPS, provide the highest BC-7 business continuity tier, providing end-to-end coordination with servers, networks and storage devices. For IBM Power Systems, similar BC-7 support is provided by IBM Geographically Dispersed Resiliency.
In this week's announcement, IBM Geographically Dispersed Resiliency (GDR) for Power Systems has been renamed and now offered in two editions: VM Recovery Manager HA and VM Recovery Manager DR. The "HA" edition provides high availability using Power Systems Live Partition Mobility for AIX, IBM i and Linux operating systems.
The "DR" edition provides both High Availability and Disaster Recovery capabilities, supporting mirrored storage systems like IBM DS8000, SAN Volume Controller, FlashSystem 9100 and V9000, and Storwize systems, as well as competitive storage from Dell EMC and Hitachi.
Next week, I will be in Hollywood, Florida for IBM Technical University (Oct 15-19), and then Rome for the IBM Technical University (Oct 22-26). I will be covering many of these announcements above, and more!
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(OK, yes, today is Friday, but I was busy getting married on Tuesday, so IBM pushed the announcements out one day to Wednesday, and technically I am writing this blog post during my honeymoon vacation, so the IBM marketing team and my new wife both cut me some slack. Work/Life balance is all about compromises, right?)
IBM DS8880 Storage System
The IBM DS8880 comes in three models, the DS8884 entry level, the DS8886 enterprise level, and the DS8888 all-flash array. IBM offers 1, 2, 3 and 4 year warranties.
The new High Performance Flash Enclosure (HPFE) Gen2 delivers more capacity than Gen1. The 2U flash enclosures are configured in pairs with each enclosure supporting up to twenty-four 2.5-inch flash cards in capacities 400 GB, 800 GB, 1.6 TB and 3.2 TB.
The HPFE Gen2 are currently available for both the DS8884 and DS8886 models. The maximum flash capacity for the DS8886 increases from 96 TB to 614.4 TB, delivering reduced storage costs through lesser cost per IOPS with this new flash enclosure. IBM has made a statement of direction to offer these HPFE Gen2 on the DS8888 as well.
To improve security, IBM DS8880 now supports customer-defined digital certificates for authentication, and configurable Hardware Management Console (HMC) firewall support.
For IBM's mainframe clients, IBM now offers "Extents-level" space release support for z/OS®, DSCLI (Command Line Interface) support for z/OS environment, and FICON® Information Unit (IU) pacing improvements.
IBM Spectrum Virtualize™ V7.8 delivers support for the latest SAN Volume Controller, FlashSystem V9000 and Storwize® product family, and adds new software functionality and improvements
In conjunction with [IBM Spectrum Copy Data Management], Spectrum Virtualize v7.8 offers flexible data protection with transparent cloud tiering to leverage the cloud as FlashCopy targets and restore these snapshots from the cloud on select platforms.
However, the encryption keys are kept on USB thumb drives, which are either left in the USB ports on the back of the hardware, or locked away in a safe, only to be retrieved as needed when rebooting the systems or upgrading the firmware.
Now, IBM Spectrum Virtualize v7.8 supports the IBM Security Key Lifecycle Manager (SKLM) to manage encryption keys. IBM continues to support USB thumb drives if you prefer, but SKLM is used to manage keys for most of the rest of IBM products, and provides centralized management.
The SVC and Storwize models can directly attach via 12Gb SAS to expansion drawers. At the time, we supported 2U-high 12-bay that support Large Form Factor (LFF) 3.5-inch Nearline (7200 rpm) drives, and 2U-high 24-bay that support the Small Form Factor (SFF) 2.5-inch drives (SSD, 15K, 10K and 7200 rpm).
With Spectrum Virtualize v7.8, IBM now offers a third option, the 5U-high 92-bay that supports both LFF and SFF drives. This new expansion can be attached to Storwize V5000 Gen2, Storwize V7000 (models 524/Gen2 and 624/Gen2+), and SVC (models DH8 and SV1).
For the 12-bay and 92-bay, IBM now supports 10TB capacity 3.5-inch Nearline drives. For the 24-bay and 92-bay, IBM now supports 7.68 TB and 15.36 TB capacity Solid State Drives (SSD).
For those concerned about the phrase "lower endurance" in the press release, let me explain. SSD have a bit of extra capacity included. If you write the full capacity of the drive every day for a year, you will "burn up" about one percent of the capacity.
To handle ten "Full Drive Writes per Day" (10 FDWP) over the course of five years, IBM adds 50 percent extra spare capacity above the 400 GB, 800 GB, 1.6 TB and 3.2 TB capacities. So, a 400GB full-endurance drive is really 600 GB inside. These were sometimes referred to as "Enterprise" SSD.
For the larger device sizes, the IT industry has determined that 1 FDWP is sufficient, so instead of 50 percent spare capacity, IBM adds only 5 percent extra. The 7.68 TB is really 8.06 TB inside. These were earlier referred to as "Read-Intensive" SSD. These come in 1.92 TB, 3.84 TB, 7.68 TB and 15.36 TB capacities.
IBM is also offering non-disruptive model conversions. Storwize V5010 can now be converted to V5020, and V5020 can be converted to V5030. The Storwize V7000 Model 524 (Gen2) can be converted to model 624 (Gen2+).
The DeepFlash 150 is the perfect JBOF addition to the ESS family. The current ESS models had either 2U-high 24-drive bays, or 4U-high 60-drive bays. This new model is 3U-high with 64 high-capacity (8 TB) Board Solid State Drives (BSSD).
The ESS includes all the features of IBM Spectrum Scale, including both 8+2 and 8+3 Erasure Coding data protection. This provides file and object access to data, including POSIX compliance for Windows, Linux and AIX operating systems, as well as HDFS-compliant access for big data analytics.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
IBM Storwize V5030F and V7000F all-flash high-density expansion enclosure
The 5U-high, 92-drive expansion enclosure introduced for the IBM Storwize V5000 and V7000 is now available for the all-flash models V5030F and V7000F. High-density expansion enclosure Model A9F requires IBM Spectrum Virtualize Software V7.8, or later, for operation.
The enclosure allows any mix of "Tier 0" write-endurance SSD at 1.6TB and 3.2TB capacities, and "Tier 1" read-intensive SSD at 1.92TB, 3.84TB, 7.68TB and 15.36TB capacities.
Storwize V5030F control enclosure models support attachment of up to 40U of expansion enclosures, which equates to eight high-density expansion enclosures, up to 760 drives per control enclosure, and up to 1,056 per clustered system.
Storwize V7000F control enclosure models support attachment of up to eight high-density expansion enclosures, up to 760 drives per control enclosure, and up to 3,040 drives per clustered system.
IBM has adopted "Agile" process for all of its IBM Spectrum Storage software. Spectrum Virtualize is offered in a variety of forms. IBM offers the FlashSystem V9000, SAN Volume Controller, Storwize family, and Spectrum Virtualize as software that runs on Lenovo and SuperMicro servers. This means quarterly delivery of new features and functions!
Lots of small enhancements were added in this release:
Apply Quality-of-Service (QoS) to a Host Cluster in terms of IOPS and or MB/s throughput.
SAN Congestion reporting, via buffer credit starvation reporting in Spectrum Control and via the XML statistics reporting, for the 16Gbps FCP Host Bus Adapter (HBA).
Resizing for Metro Mirror and Global Mirror remote copy services of thin provisioned volumes.
Consistency Protection for Metro Mirror and Global Mirror. You can now define "Change Volumes" to be used in the event of problems with MM or GM, it will switch over to GMCV mode.
Increased FlashCopy Background Copy Rates
Proactive Host Failover during temporary and permanent node removals from cluster
IBM Aspera® Files cloud service helps to enable fast, easy, and secure exchange of files and folders of any size between users, even across separate organizations. Aspera Files is currently available in three all-inclusive editions of Personal, Business, and Enterprise. Clients can subscribe either to a committed amount of data transferred on a monthly or annual basis or as a pay-per-use option.
Personal edition now includes 20 authorized users and a single workspace.
Business edition now includes 100 authorized users, 100 workspaces, support for IBM Aspera Drive, support for IBM Mobile applications, and support for Single-Sign-On.
Enterprise edition now includes 500 authorized users, no limit on number of workspaces, support for IBM Aspera Drive, support for IBM Mobile applications, and support for Single-Sign-On.
IBM is now introducing a new "Elite edition" includes 2500 authorized users, no limit on number of workspaces, support for IBM Aspera Drive, support for IBM Mobile applications, support for Single-Sign-On, and access to IBM Aspera Developer Network and nonproduction organization.
With the addition of the new Elite edition, clients have the flexibility to subscribe to additional functionality in Aspera Files that helps provide higher value and greater differentiation. The Elite edition is available as a subscription and on a pay-per-use basis.
In addition to the existing charge metric of data transferred, a user subscription metric is now included for all four editions. Each edition comes with an included number of authorized users in addition to other key features and capabilities.
The new [IBM System Storage Tape Controller 3592 Model C07] is an upgrade to the previous C06 controller. Like the C06, the new 3592-C07 can have up to four FICON (4Gbps) ports, four FC ports, and connect up to 16 drives. The difference is that the C07 supports 8Gbps speed FC ports, and can support the [new TS1140 tape drives that were announced on May 9]. A cool feature of the C07 is that it has a built-in library manager function for the mainframe. On the previous models, you had to have a separate library manager server.
Crossroads ReadVerify Appliance (3222-RV1)
IBM has entered an agreement to resell [Crossroads ReadVerify Appliance], or "RV1" for short. The RV1 is a 1U-high server with software that gathers information on the utilization, performance and health for a physical tape environment, such as an IBM TS3500 Tape Library. The RV1 also offers a feature called "ArchiveVerify" which validates long-term retention archive tapes, providing an audit trail on the readability of tape media. This can be useful for tape libraries attached behind IBM Information Archive compliance storage solution, or the IBM Scale-Out Network Attached Storage (SONAS).
As an added bonus, Crossroads has great videos! Here's one, titled [Tape Sticks]
Linear Tape File System (LTFS) Library Edition Version 2.1
While the hardware is all refreshed, the overall "scale-out" architecture is unchanged. Kudos to the XIV development team for designing a system that is based entirely on commodity hardware, allowing new hardware generations to be introduced with minimal changes to the vast number of field-proven software features like thin provisioning, space-efficient read-only and writeable snapshots, synchronous and asynchronous mirroring, and Quality of Service (QoS) performance classes.
The new XIV Gen3 features an Infiniband interconnect, faster 8Gbps FC ports, more iSCSI ports, faster motherboard and processors, SAS-NL 2TB drives, 24GB cache memory per XIV module, all in a single frame IBM rack that supports the IBM Rear Door Heat Exchanger. The results are a 2x to 4x boost in performance for various workloads. Here are some example performance comparisons:
Disclaimer: Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. Your mileage may vary.
In a Statement of Direction, IBM also has designed the Gen3 modules to be "SSD-ready" which means that you can insert up to 500GB of Solid-State drive capacity per XIV module, up to 7.5TB in a fully-configured 15 module frame. This SSD would act as an extension of DRAM cache, similar to how Performance Accelerator Modules (PAM) on IBM N series.
IBM will continue to sell XIV Gen2 systems for the next 12-18 months, as some clients like the smaller 1TB disk drives. The new Gen3 only comes with 2TB drives. There are some clients that love the XIV so much, that they also use it for less stringent Tier 2 workloads. If you don't need the blazing speed of the new Gen3, perhaps the lower cost XIV Gen2 might be a great fit!
As if I haven't said this enough times already, the IBM XIV is a Tier-1, high-end, enterprise-class disk storage system, optimized for use with mission critical workloads on Linux, UNIX and Windows operating systems, and is the ideal cost-effective replacement for EMC Symmetrix VMAX, HDS USP-V and VSP, and HP P9000 series disk systems, . Like the XIV Gen2, the XIV Gen3 can be used with IBM System i using VIOS, and with IBM System z mainframes running Linux, z/VM or z/VSE. If you run z/OS or z/TPF with Count-Key-Data (CKD) volumes and FICON attachment, go with the IBM System Storage DS8000 instead, IBM's other high-end disk system.
Well, it's Tuesday again, and you know what that means! IBM Announcements! Typically, IBM System Storage has three to five major product launches per year. Making announcements every Tuesday would have been two frequent, and having one big announcement every two or three years would be too far apart. Worldwide combined revenues for storage hardware and software grew double digits last year, comparing full-year 2011 to the prior 2010 year, and I am sure that 2012 will also be a good year for IBM as well! This week we have announcements for both disk and tape, but since 2012 is the 60th Diamond Anniversary for tape, I will start with tape systems first.
TS1140 support for JA/JJ tape cartridges
The TS1140 enterprise tape drive was announced at the [Storage Innovation Executive Summit] last May. It supported a new E07 format on three different new tape cartridges. Models "JC" was 4.0TB standard re-writeable tapes, "JY" was 4.0TB WORM tapes, and "JK" were 500GB economy tapes that were less expensive, but offered faster random access.
Generally, IBM has adopted an N-2 read, N-1 write [backward compatibility]. This means that the TS1140 could read E05 and E06 formatted tapes on JB and JX media, and could write E06 format on JB and JX media. However, there are a lot of older JA and JJ media, especially as part of TS7740 environments, so IBM now supports TS1140 drives to read J1A formatted JA and JJ media. This is not just for TS7740 environments, any TS1140 in stand-alone or tape library configurations will support this as well.
TS7700 R2.1 enhancements
IBM is a leader in tape virtualization with or without physical tape as back-end media. There are two hardware models of the [IBM Virtualization Engine TS7700 family] for the IBM System z mainframe. These virtual libraries are referred to as "clusters" in IBM literature.
The TS7740 Virtual Tape Library supports putting virtual tape images on disk first, then move less-active data to physical tape, which I covered in my blog post [IBM Announcements - July 2007].
A unique feature of the TS7700 series is support for a Grid configuration, which allows up to six different TS7700 clusters to be grouped into a single instance image. These clusters can be in local or remote locations, connected via WAN or LAN connections.
R2.1 is the latest software release of this successful IBM's TS7700 series.
True Sync Mode Copy. Before R2.1, the TS7700 offered "immediate mode copy". An application would write to a virtual tape, and when it was done with the tape and performed an unmount, the TS7700 would then replicate the tape contents to a secondary cluster on the grid. With True Sync Mode, data contents are replicated per implicit or explicit SYNC points. This is another IBM first in the IT tape industry.
Remote Mount Fail-over. When you have two or more TS7700 clusters in a grid configuration, you can do remote mounts. We've added fail-over multi-pathing up to four paths, so that if a link to a remote cluster is down, it will try one of the others instead.
Parallel Copies and Pre-Migration. On of my 19 patents is for the pre-migration feature for the IBM 3494 Virtual Tape Server (VTS) that carries forward into the TS7700, and is also used in the SONAS and Information Archive products. However, when the grid architecture was introduced, the engineers decided not to allow pre-migration and copies to secondary clusters to occur concurrently. Now these two operations can be done in parallel.
Merge two grids into one grid. Now that we can support up to six clusters into a single grid, we have people with 2-cluster and 3-cluster grids looking to merge them into one. Of course, all the logical and physical volume serials (VOLSER) must be unique!
Accelerate off JA/JJ Media. There are a lot of older JA and JJ media still in TS7700 libraries. This feature allows customers to speed up the transition to newer physical tape media.
Copy Export to E06 format on JB media. This one is clever, and I have to say I would have never thought about it. Let's say you have a TS7740 with TS1140 drives, but you want to export some virtual tapes to physical media to be sent to someone who only has a TS7740 connected with older TS1130 drives. These older drives can't read new JC media nor make sense of the E07 format. This feature will let you export to older JB media in E06 format so that it will be fully readable at the new location on the TS1130 drives.
Copy Export Merge service offering. Thanks to mergers and acquisitions, it is sometimes necessary to split off a portion of data from a TS7700 grid. In the past, IBM supported sending this export to a completely empty TS7700 library, but this new service offerings allows the export to be merged into an existing TS7700 that already contains data.
LTFS-SDE support for Mac OS X 10.7 Lion
How do people still not yet know about the Linear Tape File System [LTFS]? I mentioned this in my blogs back in 2010 in [April], [September], and [November]. Last year, LTFS was the [NAB Show Pick Hits Award] and an [Emmy] for revolutionizing the use of digital tape in Television broadcasting.
In layman's terms, the Single Drive Edition [LTFS-SDE] allows a tape cartridge to be treated like USB memory stick. It is supported on the LTO5 tape drives for systems running various levels of Windows, Linux and Mac OS X. Prior to this announcement, IBM supported Snow Leopard (10.5.6) and Leopard (10.6), and now supports Mac OS X 10.7 "Lion" release.
IBM first introduced Solid-State Drives (SSD) back in 2007 where it made sense the most, in [drive-for-drive replacements on blade servers in the IBM BladeCenter]. Blade servers typically only have a single drive, and SSD are both faster and use less energy on a drive-for-drive comparison, so this provided immediate benefit. Today, SSD are available on a variety of System x and POWER system servers.
In 2008, IBM rocked the world by being the first to reach [1 Million IOPS with Project Quicksilver]. This was an all-SSD configuration which many considered unrealistic (at the time), but it showed the potential for solid state drives.
When the [XIV Gen3 was Announced - July 2011], each module included an 1.8-inch "SSD-Ready" slot in the back. IBM made a Statement of Direction that IBM would someday offer SSD drives to put in these slots. Today's announcement is that IBM has finalized the qualification process, so now XIV Gen3 clients can have 400GB of usable non-volatile SSD read cache added to each module. This SSD can be added to existing XIV Gen3 boxes in the field, or it can be factory-installed in new shipments. If you have a 15-module XIV, that's 6TB of additional read cache! This SSD is entirely managed by the XIV Gen3, so you won't have to spend weeks reading manuals or specifying configuration parameters.
When you carve volumes on the XIV, you now have an option to enable or disable use of the SSD cache for each volume. Since XIV is being used in private and public cloud deployments, this offers the ability to offer premium performance at premium prices. The use of SSD is complementary to IBM XIV Quality of Service (QoS) performance levels, which are determined by host instead.
Well, that's the first major IBM System Storage launch of 2012. Let me know what you think in the comment section below.
Well, it's Tuesday again, and you know what that means? IBM Announcements! There were lots of announcements today, so I have split this up into two posts. One for the Tape and Cloud announcements, and the other for the Spectrum Storage family.
IBM TS7700 Virtual Tape System
IBM TS7700 release 4.1.1 now supports seven- and eight-way grids with approved RPQs. Before this, grids could only have up to six TS7700 systems connected together.
IBM also plans to extend the capacity of the TS7760 base frame to over 600 TB, and to extend the capacity of a fully configured TS7760 system to over 2.45 PB, before compression, by supporting 8 TB disk drives. This is a huge increase over the 4TB and 6TB drives used today.
IBM offers the IBM Cloud Object Storage System in three ways: as software, as pre-built systems, and as a cloud server on IBM Bluemix (formerly known as SoftLayer).
For those not familiar with IBM Cloud Object Storage (IBM COS), consider it "Valet Parking" for your storage. In a valet parking environment, you have valet parking attendants that drive the cars, parking garages that hold the cars, and a manager that oversees the operation. With IBM COS, you have Accesser® nodes that receive and retrieve your data like valet parking attendants, you have Slicestor® nodes that store your objects like cars in a parking garage, and you have IBM COS Manager to oversee the operation.
Today, IBM announced new HDD options for their S01, S03 and S03 models of Slicestor nodes. These are all 7200 rpm, 3.5-inch Nearline drives, at capacities of 4 TB, 6 TB, 8 TB and 10 TB.
In addition, a short-range 40 GbE SFP+ transceiver is available for ordering on IBM Cloud Object Storage Accesser models A00, A01, and A02, and IBM Cloud Object Storage Slicestor models S01 and S02. This improves the performance of data transfer between the Accesser nodes and the Slicestor nodes. Think of it like shortening the distance valet parking attendants have to drive your car to the garage and run back.
I have been presenting Cloud Storage for nearly 10 years now. People are often shocked to learn that most of the major cloud providers -- including Amazon, Google, Microsoft -- do not offer "Data at Rest" encryption on their storage offerings.
Why not? Because it would mean investing in Self-Encrypting Drives, Key management software, and other related technology to make it happen. Instead, Cloud Service Providers (CSPs) expect you to encrypt the data in software. Most users encrypt data before it lands on the cloud, but what if you create the data in the cloud?
IBM solved this by offering IBM Cloud Object Storage in its IBM Cloud (formerly known as SoftLayer). It has integrated encryption software that takes care of this for you.
This new product, IBM Multi-Cloud Data Encryption V1.0, enables you to encrypt files, folders, and volumes in any cloud while maintaining local control of encryption keys. It integrates with IBM Security Key Lifecycle Manager (SKLM). This is designed to allow you to move cipher data between clouds that are running Multi-Cloud Data Encryption without decrypting and re-encrypting the data.
For example, you can use IBM Multi-Cloud Data Encryption to protect your data on Amazon, Google or Microsoft, then later realize that you can save a ton of money moving to IBM Cloud instead, and you are now able to move the data over seamlessly!
Well, it's Tuesday again, and you know what that means? IBM Announcements! There were lots of announcements today, so I have split this up into two posts. One for the Tape and Cloud announcements, and the other for the Spectrum Storage family.
IBM Spectrum Virtualize Software V7.8.1
IBM Spectrum Virtualize&trade: V7.8.1 is the latest software for FlashSystem V9000, SAN Volume Controller and Storwize products.
Last release, IBM introduced "Host Groups" for clusters that needed to share a common set of volumes. This release offers "Host cluster I/O throttling": I/O throttling can be managed at the host level (individual or groups) and at managed disk levels for improved performance management,and GUI support.
Increased background FlashCopy transfer rates: This feature enables you to increase the rate of background FlashCopy transfers, providing faster copies as the infrastructure allows. This takes advantage of the higher performance capabilities of today's systems, processing the copy in a shorter period of time. The default was 64 MB/sec, and now we can go up to 2 GB/sec, for those who want their FlashCopy to be done as fast as possible.
Port Congestion Statistic: Zero buffer credits help detect SAN congestion in performance-related issues, improving support in high-performance environments. IBM had this for the 8Gbps FCP cards, but not for the 16Gbps cards, so now that's fixed.
Resizing of volumes in remote mirror relationships: Target volumes in remote mirror relationships will be automatically resized when source volumes are resized. Lots of clients asked for this, and IBM delivered!
Consistency protection for Metro/Global Mirror relationships: An automatic restart of mirroring relationships after a link fails between the mirror sites improves disaster recovery scenarios, helping to ensure the applications are protected throughout the process.
When IBM introduced "Global Mirror with Change Volumes" (GM CV), I wanted to call it "Trickle Mirror", because the primary site takes a FlashCopy, trickles the data over, then FlashCopy at the remote site. Now, clients using traditional Metro or Global Mirror can add "Change Volumes" as protection. In the unlikely event a network disruption occurs, it drops down to GMCV until the link resumes full speed.
Support of SuperMicro servers for the Spectrum Virtualize as Software Only offering: Support for x86-based Intel™ servers by SuperMicro for Spectrum Virtualize Software is available with this release.
Last year, IBM offered Spectrum Virtualize as software that could run on Lenovo servers. However, now there are clients who want alternative server choices.
Supermicro SuperServer 2028U-TRTP+ is supported to run Spectrum Virtualize Software. This is a great option for end clients, managed service or cloud service providers deploying private clouds, building hosted services, or using software-defined storage on third party Intel servers. This a fully inclusive license with all key features available on Spectrum Virtualize in a single, downloadable image.
IBM Spectrum Control V5.2.13 and IBM Virtual Storage Center V5.2.13
We often joke that IBM Virtual Storage Center is the [Happy Meal] combining storage virtualization with Spectrum Virtualize hardware like FlashSystem V9000, SAN Volume Controller or Storwize as the "hamburger", Spectrum Control as the "fries" and "Spectrum Protect Snapshot" as the "soft drink". Storage Analytics was included as a "prize inside" only available in the VSC bundle to entice clients to chose this option.
Whenever IBM updates Spectrum Control, they often put out a new version of the Virtual Storage Center bundle as well. I was the Chief Architect for Spectrum Control 2001-2002, and Technical Evangelist for SVC in 2003 when we first introduced the product, so I have long history with both products.
This release provides additional information and performance metrics on Dell EMC VMAX and EMC VNX devices. This is done natively, they do not need to be virtualized by Spectrum Virtualize as was often done in the past.
IBM now offers better visibility of drives within IBM Cloud Object Storage Slicestor® nodes. IBM acquired Cleversafe 18 months ago, and are working to get it under the Spectrum Control management umbrella.
IBM Spectrum Scale™ file system to external pool correlation. Spectrum Scale can migrate data to three different type of "external pools":
Cloud Object pool, either on-premise Object Storage or off-premise Cloud Service Provider storage.
Spectrum Protect pool, where Spectrum Protect manages the migrated data on one of 700 supported devices, including tape, virtual tape, optical, flash, disk, object storage or cloud.
Spectrum Archive pool, where data is written directly to physical tape using the Industry-standard LTFS format.
This release provides additional information on the copy data panel about SAN Volume Controller (SVC) HyperSwap® and vDisk mirror.
While the "Virtual Storage Center" bundle is an awesome deal, some clients have asked for the "Vegetarian Option" (Fries and Drink only). Why? Because they want the advanced storage analytics (prize inside) for other devices like DS8000, XIV, etc. So, IBM created the "IBM Spectrum Control Advanced Edition", which has everything in VSC except the Spectrum Virtualize itself.
Advanced edition adds improvements to the chargeback report. It also includes IBM Spectrum Protect™ Snapshot V8.1 release.
IBM Spectrum Control Storage Insights Software as a Service
Storage Insights is IBM's "Software-as-a-Service" reporting-only offering subset of Spectrum Control Advanced Edition. It includes direct support for Dell EMC VMAX, VNX, and VNXe storage systems. This is huge! Now, clients who have only EMC hardware can now, on a monthly basis, figure out where they are wasting money and decrease their costs.
Other features carried over include the enhanced drive support for IBM® Cloud Object Storage, enhanced external capacity views for IBM Spectrum Scale™ and additional replication views for vDisk mirror and HyperSwap® relationships for SAN Volume Controller (SVC) and Storwize® devices that I mention above.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(FTC Disclosure: I work for IBM, and have either written code and/or presented the DS8000 storage system and Spectrum Storage products in my professional capacity. This blog post can be considered a "paid celebrity endorsement" for the IBM DS8000 Storage System and Spectrum Storage software.)
IBM DS8880 and DS8880F Storage Systems
For those not up on the DS8000 nomenclature, here's a quick recap:
DS8880 supports a hybrid mix of Flash cards, SSD, 15K, 10K and 7200 rpm drives.
This includes the DS8884 and DS8886. The Flash cards are held in High Performance Flash Enclosures (HPFE) directly attached to the controllers, whereas the SSD and spinning disk are in shelves connected via the Device Adapters.
DS8880F is an all-flash array, with Flash cards only in HPFE. This includes the DS8884F, DS8886F and DS8888F models.
DS8880/F is convenient shorthand to refer to both the hybrid and all-flash models collectively.
Today, IBM announces new 7.68TB flash cards for the High Performance Flash Enclosures of the IBM DS8880/F. These are double the capacity of the 3.84TB cards currently available, doubling the total capacity to 368.6TB per HPFE.
Different DS8880 models support a different number of HPFE. An HPFE is a pair of 2U drawers, holding a total of 48 flash cards. You can purchase flash in groups of 16 cards, with the option to mix and match within the HPFE. For example, you can have 16 cards at the 1.6TB capacity, 16 cards with 3.68TB and 16 cards of the new 7.68TB capacity, all in a single HPFE.
The new 7.68TB support 1 Drive Write Per Day (DWPD). Some people call these "Read-Intensive" drives, but IBM refers to them as "High-Capacity Drives", to differentiate them from the "High Performance Drives" that support 10 DWPD.
In reality, the read performance is similar in both types of Flash cards offered, but the write performance is slightly slower for the High-Capacity drives due in part to additional garbage collection performed in the background. Our studies found that over 90 percent of workloads might find the High-Capacity drives good enough to handle I/O requirements.
IBM Easy Tier was updated to distinguish between High-Performance and High-Capacity flash cards, so that blocks of data that have higher or lower I/O characteristics will be relocated to the appropriate level of storage.
The newest level of IBM Spectrum Storage Suite simplifies procurement by bringing together the latest releases of the following software:
IBM Spectrum Accelerate V11
IBM Spectrum Archive Enterprise Edition V1 (Linux edition)
IBM Spectrum Control Advanced Edition V5
IBM Spectrum Protect Suite V8 (including Spectrum Protect Plus!)
IBM Spectrum Scale Data Management Edition V5
IBM Spectrum Virtualize Software for SAN Volume Controller V8 (including FlashCopy and Remote Mirror, Real-time Compression and Encryption Software)
IBM Spectrum Virtualize Software-only V8
IBM Cloud Object Storage System V3
Instead of buying software products separately, a single license enables administrators to deploy IBM Spectrum Storage Suite software when and where they need it, without having to wait. Simplified capacity pricing can significantly reduce software costs and time spent on license management.
The Spectrum Storage suite also offers a "sandbox" approach for try-and-buy. Since you have access to all the software listed, you can set up a sandbox to experiment with the functionality, without having to pay for the added capacity, until you deploy it to dev/test, quality assurance, or production.
The suite is licensed per Tebibyte [TiB]. For those not familiar with international standards, here is a comparison table:
Always decimal, 10 to the 12th power
Always binary, 2 to the 40th power
The two terms sound similar and represent nearly the same quantity within 10 percent of each other, so it is understandable when people mistakenly use the terms interchangeably.
From farm to fork, IBM Food Trust platform is a collaborative network of growers, processors, wholesalers, distributors, manufacturers, retailers and others enhancing visibility and accountability in each step of the food supply.
Powered by the IBM Blockchain Platform on IBM servers and storage systems, IBM Food Trust directly connects participants through a permissioned, permanent and shared record of food origin details, processing data, shipping details and more.
(This reminds me of a funny story: the man sitting next to me on my flight back from an IBM Systems conference in New Orleans asked me, "You look familiar. Didn't I see you at the conference this week?" I responded "Yes, were you there for the "server" or "storage" side?" He thought about it for a while, and said "I guess the server side". "Too bad", I replied, "I am on the storage side."
It took us a while, but I realized he worked in the food and restaurant industry, and that he was at a completely different conference. It happened to also have both a "server" and "storage" side!)
The IBM Food Trust platform provides new levels of transparency, quicker recalls, better standardized communication and protection of brand value. As an authorized user, you have immediate access to shared, actionable food supply data through integrated IBM Blockchain-powered modules for faster traceability and more confidence in provenance.
Today, IBM announces new services to enable clients to successfully connect to and make use of the IBM Food Trust Platform.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(This week I am in Pennsylvania and New York speaking to clients. The weather this week has not been cooperative!)
Spectrum Protect Plus 10.1.2
Just in time for the upcoming VMworld conference, IBM announces the following features added to Spectrum Protect Plus, a snapshot-based backup software for VMware, Hyper-V and databases.
Data-at-Rest Encryption for local backups stored in the vSnap repository
IBM Db2 support with point-in-time recovery
VMware vSphere 6.7 support
Alerting for backup and restore jobs and storage thresholds limits
Drill-down capabilities for dashboard widgets
Spectrum Protect 8.1.6
IBM also continues to enhance its traditional file-based backup product. Here are some of the features:
Tier data by backup state for container pools. When you have multiple backup versions, the most recent version is called the "active", the older versions are called "inactive" versions. Rarely do you recover inactive versions, so this feature allows them to be migrated off to object or cloud storage.
Ransomware detection for Virtual Environment workloads. This is an enhancement of the "Ransomware detection" introduced earlier this year, but for VMware and Hyper-V images.
IBM DS8882F All-Flash Array
When IBM announced the DS8880, it shocked folks that it changed them from the previous 33-inch wide, to a standard 19-inch width. The IBM Z team followed up with 19-inch wide models of its mainframe servers.
Now, IBM can bring these together. There are two flavors of the new DS8882F:
The "Rackless" model is 17U in height with the optional keyboard/monitor, and can be put into existing 19-inch racks. These can be used with VMware, Linux, Windows, AIX and z/OS.
The "Flex Frame" model, which is 16U, allowing it to fit nicely inside a single-rack IBM Z Z14 ZR1 model, or LinuxOne RockHopper II model. It is 16U instead of 17U because it shares the existing 1U-high keyboard/monitor unit.
Like the DS8888F, DS8886F, and DS8884F models, the new DS8882F uses the High Performance Flash Enclosure (HPFE) gen2 drawers, supporting either high-performance/high-endurance drives (400GB to 3.2TB each), or high-capacity/standard-endurance drives (3.8TB to 15.3 TB each).
The R8.5 release of firmware that accompanies this announcement also supports data-in-flight encryption for Transparent Cloud Tiering. It also supports a new feature called "Safeguarded Copies", up to 500 copies to protect against hackers and ransomware.
IBM Spectrum Access blueprints have been extended to support IBM Z and LinuxOne. These blueprints show how to run IBM Cloud Private with Spectrum Connect with IBM block storage, including IBM DS8880/F, SVC, Storwize and FlashSystem models.
IBM Storage Solutions for Virtual Desktop Infrastructures (VDI)
IBM offers a new blueprint to configure Virtual Desktops with its newly announced IBM FlashSystem 9100 device. The low latency/high IOPS capability of the FlashSystem 9100 is perfect for the type of "boot storms" that are often encountered with VDI deployments.
IBM Spectrum Scale 5.0.2 and Elastic Storage Server
At recent IBM Technical University, I joked that the IBM Elastic Storage Server is only "part of a complete breakfast" because it only supported the NSD POSIX interface. To make it useful in most situations, you needed to buy additional servers outside of the ESS to run Spectrum Scale protocol nodes to provide industry-standard file and object protocols.
Today, IBM announced that you can order a new "IBM Elastic Storage Server Data Server" (5148-22L) which is a POWER server with the Spectrum Scale software pre-installed for protocol node support. It has [similar specifications] to the IBM Elastic Storage Server Management Server (5148-21L).
If you prefer to run Spectrum Scale in the cloud, you can "Bring your own license" (BYOL) to Amazon Web Services.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(This week in the USA there are big elections. However, I am far, far away in Zurich, Switzerland writing IBM Redbooks.)
Storwize V7000 Model 724
The next generation of the Storwize V7000, officially model 724, supports NVMe and FC-NVMe (NVMe/FC) on 16Gbps adapters, and iSER on 25GbE adapters. You can cluster Storwize V7000 Gen2+, Storwize V7000 model 724, and FlashSystem 9100 series into the same cluster. There are some differences, shown in this table below:
The latest software is now available for SVC, FlashSystem 9100 and V9000, and Storwize V7000 and V5000.
NVMe over Fibre Channel support on 16 Gb Fibre Channel adapters extends the simplicity, efficiency, and end-to-end NVMe model where NVMe commands and structures are transferred end to end, requiring no translations. This is often written as FC-NVMe or NVMe/FC.
Full IP-based quorum delivers support for administrators looking to consolidate their infrastructure over Ethernet. Previously, clients used a mix of IP-based and physical quorum disks.
Increased host mappings to 64K, more than triple from the previous limitation of 20K.
IBM Spectrum Insights provides a call home protocol that uses IBM IP to deliver a more robust path and higher bandwidth/higher frequency data transmission, with end-to-end confirmation of receipt.
Single copy vdisk expand with format enables administrators to expand a vdisk without migrating the data off and back on. For example, if you want to expand a 100GB LUN to 150GB, you can do this, and the hardware will format the additional 50GB capacity.
iSER support for host attachment with 25 GbE adapters expands the host connectivity options for SVC, FlashSystem V9000 and Storwize V5000. This is similar to the v8.2 support for Storwize V7000 and FlashSystem 9100.
Clustering support over Ethernet using RDMA enables IBM Spectrum Virtualize software to run over multiple types of technology, including FlashSystem 9100 and V9000, SVC, and Storwize V7000. This was the last hold-out for all-Ethernet shops. IBM supported Ethernet host attachment, and Ethernet-based back-end storage, and Ethernet-based replication mirroring, so the last piece is now complete, Ethernet-based node-to-node communications needed for clustering.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
The Collaboration of Oak Ridge, Argonne, and Livermore [CORAL] is a joint procurement activity among three of the Department of Energy's National Laboratories launched in 2014 to build state-of-the-art high-performance computing (HPC) technologies that are essential for supporting U.S. national nuclear security and are key tool s used for technology advancement and scientific discovery.
Of course, when you hear "state-of-the-art technology", IBM is probably the first company that comes to mind!
The new IBM Spectrum Scale 5.0 has been greatly enhanced to meet CORAL requirements:
Dramatic improvements in I/O performance
Significant reduction in internode software path latency to support the newest low-latency, high-bandwidth hardware such as NVMe
Improved performance for many small and large block size workloads simultaneously from new 4 MB default block size with variable sub-block size based on block size choice
Improved metadata operation performance to a single directory from multiple nodes
Spectrum Scale 5.0 now handles automatically tuning more than twenty communication protocol and buffer management parameters, aiding setup for optimal performance. The enhanced GUI features many capabilities including performance, capacity, network monitoring, AFM (multicluster management), transparent cloud tiering, and enhanced maintenance and support, including interaction with IBM remote support.
Spectrum Scale 5.0 now offers file-level immutability. Previous releases supported immutability at the file set granularity, so this allows greater granularity. Immutability can be an effective tool as part of an overall Non-Erasable, Non-Rewriteable [NENR] compliance policy.
Spectrum Scale comes in both "Standard Edition" and "Data Management Edition". The latter offers some additional features, including Transparent Cloud Tiering, Asynchronous AFM Disaster Recovery support, and Encryption. Some additional enhancements to Data Management Edition in Spectrum Scale 5.0 are:
File audit logging capability to track user accesses to file system and events supported across all nodes and all protocols
Parseable data stored in secure retention-protected fileset
Data security following removal of physical media protected by on-disk encryption
The new IBM Storage Utility Offerings include the IBM FlashSystem 900 (9843-UF3), IBM Storwize V5030 (2078-U5A), and Storwize V7000 (2076-U7A) storage utility models that enable variable capacity usage and billing.
These models provide a fixed total capacity, with a base and variable usage subscription of that total capacity. IBM Spectrum Control Storage Insights is used to monitor the system capacity usage. It is used to report on capacity used beyond the base subscription capacity, referred to as variable usage.
The variable capacity usage is billed on a quarterly basis. This enables customers to grow or shrink their usage, and only pay for configured capacity.
Suppose you only need 300 TB today, but expect this to grow to 1 PB (1000 TB) over the course of three years. You install 1000 TB (1 PB) of capacity, and pay for the base 300 TB, plus whatever above this 300 TB you might be using during each subsequent quarter. After 36 months, you pay for the rest of capacity installed.
(There are comparable offerings from IBM's competitors, but they often require that you pay for at least 75 to 85 percent of the installed amount, and then you would need to continue to disrupt your operations with additional capacity installed throughout the 12 to 36 month period. IBM's approach allows you to avoid installation disruption during the entire 36 month period!)
IBM Spectrum Virtualize for Public Cloud V8.1.1 delivers a powerful solution for the deployment of IBM Spectrum Virtualize software in public cloud, starting with IBM Cloud. This new capability provides a monthly license to deploy and use Spectrum Virtualize in IBM Cloud to enable hybrid cloud solutions
Remote replication will be supported between Spectrum Virtualize-based appliances (including SAN Volume Controller (SVC), the Storwize family, IBM FlashSystem V9000, and VersaStack with Storwize family or SVC), or Spectrum Virtualize Software, to the IBM Cloud.
Using IP-based replication with Metro Mirror, Global Mirror, or Global Mirror with Change Volumes, clients can create secondary copies of on-premises data in the public cloud for disaster recovery. IBM has over 25 data centers around the world available to chose from. Remote copy services can also be used between two IBM Cloud data centers for improved availability.
The solution is based on bare metal servers. You can create either two- or four-node high availability clusters.
Spectrum Virtualize on-premise SVC and Storwize now also support 2.4 TB 10K rpm 2.5-inch SAS hard disk drives.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
IBM Elastic Storage Server
Replacing the older "GSn" and "GLn" models, IBM announces the "Second Generation" GSnS and GLnS models (the second "S" stands for Second Generation), the "n" continues to refer to the number of storage drawers. All of these have a pair of POWER8 servers to drive amazing performance at a low price point.
The "GSnS" models are based on smaller 2U, 24-drive storage drawers, with 3.84 and 15.36 TB Tier-1 Read-intensive Solid-State Drives (SSD). The "GLnS" models are based on larger 5U, 84-drive storage drawers, with 4TB, 8TB and 10TB nearline (7200 rpm) spinning disk.
These new models have the latest IBM Spectrum Scale software pre-installed.
In addition to IBM's two existing Hyperconverged offerings--IBM Spectrum Accelerate for x86 servers, and IBM Spectrum Scale for x86, POWER and z Systems servers--IBM Power Systems now offers a third option. This integrated offering combines Nutanix's Enterprise Cloud Platform software with IBM Power Systems™ hardware to deliver a turnkey hyperconverged solution that targets critical workloads in large enterprises.
Nutanix is offered and will be defaulted/required on these Power® servers only:
While "Hyperconvergence" is still fairly new, and only about 1 percent of data centers have deployed this new technology, I am glad that IBM is a leader in this space with multiple offerings across both x86 and POWER systems platforms.
This last one on how to build your own Watson, Jr. has gotten over 69,000 hits! While several people told me they plan to build their own, I have not heard back from anyone yet, so perhaps it is taking longer than expected.
IBM and Wellpoint announced this week that it will be [putting Watson to work] in healthcare. [Wellpoint] is one of the largest health benefits company in the United States, with over 70 million people served through its affiliate plans and its various subsidiaries. I am one of the development lab advocates for Wellpoint, and have been proud to work with the account team to help Wellpoint achieve their goals.
This marks the first commercial deployment of IBM Watson. This is a joint effort. IBM will develop the base IBM Watson for healthcare platform, and Wellpoint will then develop healthcare-specific solutions to run on this platform. Watson's ability to analyze the meaning and context of human language, and quickly process vast amounts of information to suggest options targeted to a patient's circumstances, can assist decision makers, such as physicians and nurses, in identifying the most likely diagnosis and treatment options for their patients.
Is this going to put doctors out of business? No. Physicians find it challenging to read and understand hundreds or thousands of pages of text, and put this into their practice. IBM Watson, on the other hand, can scan through hundred of millions of pages in just a few seconds to help answer a question or provide recommendations. Together, doctors armed with access to IBM Watson will be able to improve the quality and effectiveness of medical care.
From an insurance point of view, improving the quality of care will help reduce medical mistakes and malpractice lawsuits. This is a win-win for everyone except ambulance-chasing lawyers!
Long-time readers of my blog know that typically IBM makes its announcements on Tuesdays, but this week, we had an announcement today, Wednesday!
IBM announced agreements with Brocade, Cisco and Juniper Networks to help build more dynamic infrastructures. An IBM study estimates that the "digital footprint" of each person will grow from 1TB today to 16TB by the year 2020, and all of that data will need bandwidth to get around.IBM’s Data Center Networking (DCN) Initiative is focused on providing clients with solutions to address these three key areas in networking:
Partner for choice of networking products based and open standards to ensure clients can select from a full range of hardware options for data center networks. IBM will build on its strong partnerships with leading networking vendors to provide greater customer choice.
Integrate with IBM Data Center Products and Services. IBM continues to deliver a comprehensive portfolio of services for network design, integration and management from IBM Global Technology Services (GTS).
IBM differentiates itself with Common Data Center Management, where we can bring together software products, such as IBM Systems Director and Tivoli portfolio, to create what we refer to as a "Unified Service Management" software platform.
Here's a sample of what IBM announced:
IBM continues its strategic support for Fibre Channel over Ethernet (FCoE) with a Converged B32 switch, and Converged Network Adapters (CNA) for IBM System x servers. A CNA card does the job of both the Ethernet Network Interface Card (NIC) as well as the Fibre Channel Host Bus Adapter (HBA), reducing the number of cables from each server.
IBM and its Business Partners will resell the Cisco Nexus 5000 Series Switches, a leading family of high-performance, low-latency switches for data center networks supporting lossless 10GbE, Fibre Channel and Fibre Channel over Ethernet (FCoE).
IBM will OEM selected Juniper EX and MX switches and routers. This expands upon IBM and Juniper's long-term relationship that includes a reseller agreement with IBM Global Technology Services, collaboration on Juniper's Stratus Project, and IBM's ten worldwide Cloud Labs.
IBM wins lots of awards, but this time is unique: [IBM and Fox Networks Group have jointly won an Engineering Emmy® Award] for Innovation from the Academy of Television Arts & Sciences. According to the Academy, by improving the ability of media companies to capture, manage and exploit content in digital form, IBM and Fox have fundamentally changed the way that audio and video content is managed and stored. Here's an excerpt from the IBM Press Release:
"By standardizing technologies in this way, Fox can now use open-standard, file-based tape in all aspects of production, post-production and distribution functions – displacing costly proprietary tape formats and/or disk subsystems. This provides media companies with the consumer equivalent of having their entire library of DVDs online and available at any time, and the ability to go to a specific scene, in any one of the movies, in an instant.
In the early stage of the technology initiative, the IBM/Fox team applied IBM-patented technologies invented by IBM Research for high-speed data movement. They also integrated traditional broadcast transport and encoding standards with IT industry open standards. This allowed either Standard Definition (SD) or HD programming to be available in real time for digital recording and repurposing -- with improved economics."
Unfortunately, people didn't like the name, but they loved the acronym, so it was renamed to Linear Tape File System. IBM offers LTFS single-drive edition on its LTO-5 and TS1140 tape drives, and LTFS library-edition across all of its tape libraries. Since everyone hates proprietary vendor lock-in, IBM has graciously shared LTFS as an open source standard with the rest of the Linear Tape Open consortium.
(Note: I was not there at the awards ceremony. The pictures were taken by Ed Childers, David Pease and Rainer Richter of each other. Additional photos are available on this [Flicr photo album].)
(1) Rainer Richter, Media Technology Market Partners LLC [MTMP], presenting the Emmy to Steve Canepa, IBM General Manager for Media and Entertainment industry. MTMP is an IBM Business Partner that offers integrated solutions for LTO and LTFS, consulting, services, and technology to the media and entertainment industry…
(2) Ed Childers, IBM manager of the Tape Drive Development team, holding the Emmy. Fellow IBM blogger Steve Hamm credits Ed on coming up with the idea for LTFS seven years ago, in his blog post [Coding and Loading in Las Vegas: How a Team of IBM Researchers Helped Transform the Way Video is Stored]. Ed wanted to make tape storage easier to use and to integrate it into the workflow of networks and studios, and suggested using an indexing system that would allow people to write software that would make video more accessible.
(3) David Pease, IBM Senior Technical Staff Member from the IBM Almaden Research Center, holding the Emmy. Along with Lucas Villa Real (IBM Brazil) and Michael Richmond (IBM Almaden), David and his team were able to come up with a working prototype in just four months. Michael discusses this in his posts [Tape? Does anyone care about Tape anymore?"] and [the Emmy goes to... LTFS].
Of course, Technology is only worthwhile if you put it to use. Our friends at FOX initially partnered with IBM to develop this video archive solution for the National Football League (NFL). If there is one place that "re-purposes" a lot of video footage, it is sports television. The technology proved so useful that FOX has since expanded it to other types of programming.
Can you believe it has been a year already since IBM announced VersaStack?
In my May 2012 blog post, [EMC Strikes Back], I poked fun at the fact that Cisco had two girlfriends "significant others": EMC and NetApp.
Cisco originally partnered with EMC to create a converged system called Vblock which combined Cisco UCS servers and switches with EMC storage. The partnership between VMware, Cisco and EMC was dubbed Virtual Computing Environment (VCE).
However, Cisco then partnered with NetApp to create Flexpod, a converged system that combined Cisco UCS servers and switches with NetApp storage. Many of my clients felt that Flexpod was an improvement over Vblock.
Before VersaStack, IBM had its own converged system, PureSystems, which combined IBM POWER and x86 servers with IBM storage. The x86 server portion of this business was sold off to Lenovo, but IBM continues to sell POWER-only and blended x86-and-POWER PureFlex systems, as well as PureApplication and PureData systems.
The [VersaStack] collaboration between IBM and Cisco offers an alternative to Vblock and Flexpod converged systems. Cisco is a leader in x86 blades and networking switches, and IBM is #1 in Flash and Software Defined Storage, including Storage Virtualization. VersaStack gives you the best of both worlds!
The VersaStack has Cisco Validated Designs for use with IBM's Spectrum Virtualize products:
Storwize V7000 Unified
This week, February 11, 2016, 12pm EDT, IBM and Cisco are hosting a webinar on VersaStack. Join us for the one year anniversary of VersaStack in a discussion with IBM, Cisco and VersaStack customers.
The speakers will be discussing VersaStack progress to date and the value VersaStack brings to client workloads. Topics of discussion will include how VersaStack can lower TCO, administrative overhead, reduce downtime and improve resource utilization, and allow for business innovation. The speakers include:
Jonathan Cox, Medicat, Director, Technology Services
Susan Martens, IBM, Director, VersaStack Sales, North America
Kent Hixson, Cisco, Sales Business Development Manager
(Back in 2010, I poked fun at EMC with my post [VPLEX: EMC's Latest Wheel is Round]. I pointed out that EMC's announcement of "new features" that already existed in IBM's SAN Volume Controller. Oops! They did it again!)
Basically, Dell EMC is working on a new "2 Tiers" approach that combines high-performance flash tier with high-capacity object storage. Guess what? IBM already offers this! Why wait?
IBM Spectrum Scale, formerly known as the General Parallel File System (GPFS), supports POSIX, HDFS, OpenStack Swift, Amazon S3, NFS, SMB and iSCSI protocols.
Spectrum Scale can provide this front-end abstraction layer between flash and object storage, including IBM Cloud Object Storage system and IBM Bluemix (formerly SoftLayer) cloud services.
But why limit yourself to just two tiers? IBM Spectrum Scale can also support 15K, 10K and 7200 RPM spinning disk drive tiers, as well as virtual or physical tape tier, the ultimate low-cost high-capacity tier!
Several years ago, IBM coined the phrase "FLAPE" to discuss the two-tier approach of combining Flash with Tape using Spectrum Scale as the front-end abstraction layer.
Perhaps we should call combinations of Flash and Object "FLobject" storage? If the name catches on, you read it here first!
Well, it's Tuesday again, and you know what that means! IBM Announcements!
Starting today, April 1, 2014, the IBM Executive Briefing Centers (EBC) are adopting a new self-hosted model. In the past, each briefing was assigned a "Briefing Host", a member of the EBC staff, who acted as [master of ceremonies] for the day (or more) for the clients. At some locations, if there were three rooms, there would be three or more briefing hosts so that concurrent briefings could be held.
However, the method does not scale. Having a person per briefing means that you are limited to the number of total concurrent briefings. Inspired by self-service provisioning and scalability of the Cloud, IBM has adopted a new methodology.
In the new model, the visiting client rep, sales rep, or IBM Business Partner will be handed instructions and a map. This will include the agenda, the schedule, biographies of each speaker, the locations of the nearest restrooms, and so on.
I can take partial credit for the idea. In 2012, I made the analogy that having briefing centers at each development lab made a lot of sense, because it allowed clients to interact directly with the engineers and executives that made development decisions. I also made the analogy that having a fully-staffed EBC was like a fire department, whether you have five briefings per month, or fifty, you need a team that is ready, staying abreast of the latest technological changes.
In my post, [Like animals in the zoo], I argued there are two kinds of zoos, the self-guided kind, where visitors are handed a map, versus the docent-guided kind, where a member of the zoo staff introduces you to each animal.
The EBC briefing hosts in this analogy were the docents, and the animals that people came to visit were the engineers and executives.
As for the fire department, IBM management flipped the analogy around. They argued that many smaller communities had "volunteer fire departments", eliminating the need to keep full-time employees doing nothing but playing cards and sliding down brass poles in between fire fighting sessions. When a fire happens, phones calls are made, and this will help get everyone notified to get involved.
In my past 28 years at IBM, I have to say that you know you have good analogies when they can be used in both directions. The zoo analogy was used to prevent management from consolidating all of the EBC staff to Austin, TX. The fire department analogy helped us keep all of our lab equipment to run demonstrations.
The new self-hosted model will address both scheduling and scalability issues. We often had two-day and three-day briefings, and scheduling the rooms, and the briefing managers, based on their availability, was quite challenging.
There are three advantages to the new method:
A coordinator will merely assign rooms, no longer worrying if a briefing host is available for those days. Now, each EBC location can run at full capacity, limited only by real estate and floor space.
Subject matter experts, like myself, that often did double-duty serving as briefing hosts as needed, will have more free time. I personally will be doing more "outbound briefings" to attend conferences and visit clients at their location, eliminating the time I need to be in Tucson to host "inbound" briefings.
The awkward silence that happens when the client rep, sales or IBM Business Partner invites all the clients and presenters, but forgets to invite the briefing host, is completely eliminated.
Well, it's 2008, which could mark the end to RAID5 and mark the beginnings of a new disk storagearchitecture. IBM starts the year with exciting news, acquiring new disk technology from a smallstart-up called XIV, led by former-EMCer Moshe Yanai. Moshe was ousted publicly in 2001 from hisposition as EMC's VP of engineering, and formed his own company. It didn't take long for EMC bloggersto poke fun at this already. Mark Twomey, in his StorageZilla blog, had mentioned XIV before back in August,[XIV], and again todayin [IBM Buys XIV].
To address the new requirements associated with next generation digital content, IBM chose XIV and its NEXTRA™ architecture for its ability to scale dynamically, heal itself in the event of failure, and self-tune for optimum performance, all while eliminating the significant management burden typically associated with rapid growth environments. The architecture also is designed to automatically optimize resource utilization of all the components within the system, which can allow for easier management and configuration and improved performance and data availability.
"We are pleased to become a significant part of the IBM family, allowing for our unique storage architecture, our engineers and our storage industry experience to be part of IBM's overall storage business," said Moshe Yanai, chairman, XIV. "We believe the level of technological innovation achieved by our development team is unparalleled in the storage industry. Combining our storage architectural advancements with IBM's world-wide research, sales, service, manufacturing, and distribution capabilities will provide us with the ability to have these technologies tackle the emerging Web 2.0 technology needs and reach every corner of the world."
The NEXTRA architecture has been in production for more than two years, with more than four petabytes of capacity being used by customers today.
Current disk arrays were designed for online transaction processing (OLTP) databases. The focus was onusing fastest most expensive 10K and 15K RPM Fibre Channel drives, with clever caching algorithmsfor quick small updates of large relational databases. However, the world is changing, and peoplenow are looking for storage designed for digital media, archives, and other Web 2.0 applications.
One problem that NEXTRA architecture addresses is RAID rebuild. In a standard RAID5 6+P+S configuration of 146GB 10K RPM drives, the loss of one disk drive module (DDM) was recovered by reconstructing the data from parity of the other drives onto the spare drive. The process took46 minutes or longer, depending on how busy the system was doing other things. During this time,if a second drive in the same rank fails, all 876GB of data are lost. Double-drive failures are rare,but unpleasant when they happen, and hopefully you have a backup on tape to recover the data from.Moving to slower, less expensive SATA drives made this situation worse. The drives have highercapacity, but run at slower speeds. When a SATA drive fails in a RAID5 array, it could take severalhours to rebuild, and that is more time exposure for a second drive failure. A rebuild for a 750GBSATA drive would take five hours or more,with 4.5 TB of data at risk during the process if a second drive failure occurs.
The Nextra architecture doesn't use traditional RAID ranks or spare DDMs. Instead, data is carved up into 1MBobjects, and each object is stored on two physically-separate drives. In the event of a DDM loss, allthe data is readable from the second copies that are spread across hundreds of drives. New copies aremade on the empty disk space of the remaining system. This process can be done for a lost 750GB drive in under20 minutes. A double-drive failure would only lose those few objects that were on both drives, so perhaps1 to 2 percent of the total data stored on that logical volume.
Losing 1 to 2 percent of data might be devastating to a large relational database, as this could impactthe entire access to the internal structure. However, this box was designed for unstructuredcontent, like medical images, music, videos, Web pages, and other discrete files. In the event of a double-drivefailure, individual files would be recovered, such as with IBM Tivoli Storage Manager backup software.
IBM will continue to offer high-speed disk arrays like the IBM System Storage DS8000 and DS4800 for OLTP applications, and offer NEXTRA for this new surge in digital content of unstructured data. Recognizing this trend, diskdrive module manufacturers will phase out 10K RPM drives, and focus on 15K RPM for OLTP, and low-speedSATA for everything else.
Update: This blog post was focused on the version of XIV box available as of January 2008 that was built by XIV prior to the IBM acquisition. IBM has since made a major revision, made available August 2008 thataddresses a variety of workloads, including database, OLTP, email, as well as digital content and unstructuredfiles. Contact your IBM or IBM Business Partner for the latest details!
Bottom line, IBM continues to celebrate the new year, while the EMC folks in Hopkington, MA will continue to nurse their hangovers. Now that's a good way to start the new year!
(Note: We'll go back to storage tomorrow, but for today, I will talk only about the IBM Red Hat acquisition)
Back in 2007, my blog post [Double Happy Wedding] compared IBM's acquisition for a company that produced data migration software to the practice in Japan of waiting until the bride is five to seven months pregnant to have a wedding.
In business, the best acquisitions are the ones where both parties have been working together already. IBM and Red Hat have been working together for the past 20 years!
From 1999-2002, I was part of the team that help port Linux to the mainframe, based on Red Hat components. I was the first person to install Linux on a mainframe in Arizona, on a z800 machine, if you can remember that far back. My involvement with Linux was three-fold:
Back then, I was the chief architect of DFSMS on the MVS operating system (now called z/OS). We needed a way to backup Linux data on the mainframe, so I helped develop the "Compatible Disk Layout" (CDL) which made the disk volume compatible between MVS, z/OS and Linux operating systems. Linux would read and write data on the volume, and then the Linux volume could be backed up or dumped to tape using existing DFSMS utilities.
I led a team to test and debug all of the disk and tape storage drivers for Linux on the mainframe. One of my colleagues, who worked with Tom West, gave me a copy of Tracy Kidder's book [The Soul of a New Machine] as it seemed similar to our efforts. I highly recommend this book!
I ran a series of roadshows, traveling to promote Linux. At each event we had two speakers. I was the key speaker, and one of my teammates would be working the keyboard to run all of the live demos. I played the role of a reporter, hot on the story of Linux, and my teammate would play the role of my newspaper editor/boss who would ask me questions from a script. I would then answer the question by showing off a Linux demo, while my teammate hit the appropriate keys to make it happen.
At the time, many in IBM did not understand the concept of "open source", or the idea of an operating system written by people on the Internet. I saw that Linux and Open Source was the future, but not everybody I worked with at that time shared that vision. Today, open source is the default choice for business.
On the surface, the deal appears fairly straightforward. IBM paid $34 Billion, and in return gets 13,000 new employees and $3.4 Billion in new annual revenues. But this deal is more than that. This acquisition redefines the cloud market for business. Here are some excerpts from the press release:
"Red Hat's open hybrid cloud technologies are now paired with the unmatched scale and depth of IBM's innovation and industry expertise, and sales leadership in more than 175 countries. Together, IBM and Red Hat will accelerate innovation by offering a next-generation hybrid multicloud platform. Based on open source technologies, such as Linux and Kubernetes, the platform will allow businesses to securely deploy, run and manage data and applications on-premises and on private and multiple public clouds.
Red Hat will continue to be led by Jim Whitehurst and its current management team. Whitehurst is joining IBM's senior management team, reporting to Ginni Rometty. IBM will maintain Red Hat's headquarters in Raleigh, North Carolina, its facilities, brands and practices. Red Hat will operate as a distinct unit within IBM and will be reported as part of IBM's Cloud and Cognitive Software segment.
Most enterprises today are approximately 20 percent into their transition to the cloud. In this first chapter of their cloud journey, businesses made great strides in reducing costs, boosting productivity and revitalizing their customer-facing innovation programs.
The collective ability of IBM and Red Hat to unlock the true value of hybrid cloud for businesses is already resonating among customers moving to the next chapter of digital reinvention.
With Red Hat, IBM has acquired one of the most important software companies in the IT industry. Red Hat's pioneering business model helped bring open source – including technologies like Linux, Kubernetes, Ansible, Java, Ceph and many more – into the mainstream for enterprises. Today, Linux is the most used platform for development. Red Hat Enterprise Linux alone is expected to contribute to more than $10 trillion worth of global business revenues in 2019. By 2023, an additional 640,000 people are expected to work in Red Hat-related jobs.
IBM has committed to scaling and accelerating open source and hybrid cloud for businesses across industries, as well as preserving the independence and neutrality of Red Hat's open source heritage. This includes its open source community leadership, contributions and development model; product portfolio, services, and go-to-market strategy; robust developer and partner ecosystems, and unique culture."
This independence and neutrality works both ways: Red Hat will continue to work with other hardware manufacturers and IBM Power, Z and LinuxONE servers will continue to support all of the same distributions of Linux it did before, including Canonical Ubuntu Linux, and SUSE Linux Enterprise Server (SLES).
To this day, I still run Red Hat Enterprise Linux (RHEL) on my work laptop. This blog post was written using "gedit", a text-based editor that is part of the GNOME platform.
Normally, IBM only makes announcements on Tuesdays, but today, Friday, IBM announces that it acquired Diligent Technologies. What? I got a lot ofquestions about this, so I thought I would start with this...
When I posted in January that[IBM Acquires XIV],fellow EMC blogger Mark Twomey of StorageZilla fame, sent me a comment:
"Ah now Tony I wasn't poking fun. Indeed I find it fascinating that Moshe who's been sitting out on the fringes for years having been banished for being an obstructionist to EMC entering the mid-market is now back.
Which reminds me what happens with Diligent? There his as well aren't they or has he packed his stake in that in?"
As you might have guessed, I am privy to a lot of stuff going on behind the scenes at IBM that I can't talk about in this blog, and all these rumors in the blogosphere about IBM acquisition of Diligent was a topic I couldn't officially recognize, defend or deny, until official IBM announcements were made.
In his latest post, Mark wonders about[the last Tape and Mainframe sales person on earth]. He recounts my interaction with fellow HDS blogger Hu Yoshia about the energy benefits ofVirtual Tape Libraries. Knowing that we were going to announcement IBM's acquisition of Diligent soon, I thoughtthis would be a worthy exchange, driving up the sales of Diligent boxes (whether you buy them from IBM or HDS).Diligent already had reselling arrangements with HDS, and IBM plans to continue thosearrangements going forward with HDS. As I have explained before in my post [Supermarketsand Specialty Shops], IBM and HDS cater to different customers, so if a customer who wants the best technologyfrom a specialty shop, they can buy IBM Diligent products from HDS, but if they want one-stop shopping, they can buyIBM Diligent directly from IBM or its other IBM Business Partners.
(Perhaps a more tricky situation is that Diligent also had an arrangement with Sun Microsystems, which competesdirectly against IBM as another IT supermarket vendor, but I have not heard how IBM has decided to handle thisgoing forward.)
For more on this intricate mess of interconnected companies, alliances and partnerships, read Dave Raffo's article[Data dedupe dance cardfilling up] over at Storage Soup.
So, let's tackle the first question:
Q1. What will happen to IBM's real tape library business?
Come on! IBM is Number one in tape, we've had virtual tape libraries since 1997 (the first in the industry)and continue to do well in both virtual and real tape libraries. Both provide value to the customer, and bothhave their place as part of the overall "information infrastructure". This acquisition provides yet another choicefor clients on our "supermarket" shelf.
(For those following the ["which is greener"] discussion, the robot of the IBM TS3500 real tape library consumes185W per frame (when moving) and each tape drive consumes 50W (when actively working on a tape). Compared to 13W per SATA disk drive, each 6-drive frame of a TS3500 consumes as much electricity as 37 SATA disk drives. If you are not running backups 24x7, the total KWh per day for your tape library is actually quite less, but as several people have pointed out, there are customers that do run backups 80-90 percent of the time. LTO-4 tapes can hold 800GB uncompressed, and SATA disk are now available in 1TB (1000 GB) size, so you can have fun with your own comparisons.)
Meanwhile, Scott Waterhouse, one of the few people at EMC who understand tape workloadslike backup and archive, takes me to task in his Backup Blog with his post[I want a Red Ferrari].For those who are surprised that anyone at EMC might understand backup workloads, EMC did acquire a company calledLegato, and perhaps Scott came from that acquisition. I've never met Scott in person, but based solely only fromhis writings, he seems to know his stuff and makes strong arguments for using IBM Tivoli Storage Manager (TSM) with deduplication and virtual tape libraries.
While TSM does a good job of "deduplicating" at the client first, backing up only changed data, Scott feels database and email repositories must be backed up entirely each time, which is what happens in many other backup software products. Some clients might have 80 percent database/email and only 20 percent files, while others might have less than 20 percent database/email and 80 percent files, so this might influence whether deduplication will have small or big benefit.If TSM has to backup the entire database, even though little has changed since the last backup, that is where deduplication on a virtual tape library can come in handy. For IBM DB2 and Oracle databases, IBM TSM application-aware Tivoli Data Protection module interface backs up only changed data, not the entire file. Thanks to IBM's FilesX acquisition-- (also coincidently from Israel) --IBM can extend this support now to SQL Server databases as well.However, to be fair, Scott is partly correct, TSM does backup some database and email repositories in their entirety, which is why it is a good idea to have BOTH an IBM virtual tape library with deduplication and Tivoli Storage Manager to handle all cases. This brings us to the next question:
Q2. What will happen to IBM's patented "progressive backup" technology?
IBM will continue to use TSM's progressive backup technology. TSM already works great with Diligent virtual tapelibraries. One example is LAN-free backup. In this configuration, the TSM client writes its backups directly toa virtual or real tape library, over the SAN, and then sends the list of files backed up to the TSM server over theLAN to record in its database. This can greatly reduce IP traffic on your LAN during peak backup periods. For more about this, see the IBM Redbook titled["Get More Out of Your SAN with IBM Tivoli Storage Manager"].
Jon Toigo from DrunkenData asks[Did IBM Do Due Diligence Before Making Diligent Acquisition a Done Deal?] which is probably always a valid question. Unlike XIV, I wasn't part of the Diligent acquisition team, so I can't provide first hand account of the process. I am told that the IBM team did all the right things to make sure everything is going to turn out right.Sadly, many companies that make acquisitions in the IT industry fail to make them work. Fortunately, IBM is one of the few companies that has a great success record, with over 60 acquisitions in the past six years.In the Xconomy forum, Wade Rousch writes[IBM and the Art of Acquisitions]and gives some insight why IBM is different. Jon did not understand why Cindy Grossman, IBM VP of tape and archive solutions, ran the analyst conference call for this announcement, which brings me to the next question:
Q3. What is Diligent virtual tape library going to be categorized as, a disk system or a tape system?
IBM organizes its storage systems based on the host application workloads.Products to address disk workloads (SVC, DS8000 series, DS6000 series, DS4000 series, DS3000 series, N series, XIV Nextra) are in our disk systems group. Storage that appears to host applications like a tape system to address workloads like backup and archive (tape drives, libraries and tape virtualization) are in our tape and archive group. IBM Diligent has two products, one for big workloads and one for medium workloads. Both look liketape systems, so our tape and archive team, who understand tape workloads like backup and archive the best, are obviously the best choice to support IBM Diligent in the mix.
IBM will offer both N series and Diligent deduplication capabilities. For disk workloads, IBM N series offers a post-process deduplication feature at no additional charge. For tape workloads, IBM will now offer an in-line deduplication feature with Diligent Technologies. Different workloads, different offerings.
As with any acquisition, there will be some changes. The 100 folks from Diligent will get to learn the IBM wayof doing things. This brings me to our fifth and final question:
Q5. What is the correct spelling: deduplication or de-duplication?
It appears that Diligent has a corporate-wide standard to hyphenate this term (de-duplication), but the "word police" at IBM that control and standardize all "proper spellings, trademarks, and capitalization" have sent me corporate instructions a few days ago that IBM does not to hyphenate this term (deduplication). So, going forward, it will be "deduplication", or "dedupe" for short.I suspect one of the first tasks that our new IBMers from Diligent will be doing is removing all those hyphens fromthe [Diligent Technologies website]!
That's all for now, I'm off to Chicago, Illinois tomorrow!
The marketshare data for external disk systems has been released by IDC for 4Q09. Overall, the market dropped 0.7 percent, comparing 4Q09 versus 4Q08. While EMC was quick to remind everyone that they were able to [maintain their #1 position] in the storage subset of "external disk systems", with the same 23.7 percent marketshare they had back in 4Q08 and revenues that were essentially flat, the real story concerns the shifts in the marketplace for the other major players. IBM grew revenue 9 percent, putting it nearly 5 points of marketshare ahead of HP. HP revenues dropped 7 percent, moving it further behind. Not mentioned in the [IBM Press Release] were NetApp and Dell, neck and neck for fourth place, with NetApp gaining 16.8 percent in revenues, while Dell dropped 13.5 percent. Both NetApp and Dell now have about 8 percent marketshare each. These top five storage vendors represent nearly 70 percent of the marketshare.
Given that HP is IBM's number one competitor, not just in storage but all things IT, this was a major win. Bob Evans from InformationWeek interviews my fifth-line manager, IBM executive Rod Adkins [IBM Claims Hardware Supremacy] where he shares his views and opinions about HP, Oracle-Sun, Cisco and Dell.
I'll add my two cents on what's going on:
Shift in Servers causes Shift in Storage
Hundreds of customers are moving away from HP and Sun over to IBM servers, and with it, are chosing IBM's storage offerings as well. IBM's rock-solid strategy (which I outlined in my post [Foundations and Flavorings]) has helped explain the different products and how they are positioned. HP's use of Itanium processors, and Sun's aging SPARC line, are both reasons enough to switch to IBM's lastest POWER7 processors, running AIX, IBM i (formerly i5/OS) and Linux operating systems.
Thunder in the Clouds
Some analysts predict that by 2013, one out of five companies won't even have their own IT assets. IBM supports all flavors of private, public and hybrid cloud computing models. IBM has its own strong set of offerings, is also the number one reseller of VMware, and has cloud partnerships with both Google and Amazon. HP and Microsoft have recently formed an alliance, but they have different takes on cloud computing. HP wants to be the "infrastructure" company, but Microsoft wants to focus on its ["three screens and a public cloud"] strategy. Microsoft has decided not to make its Azure Cloud operating system available for private cloud deployments. By contrast, IBM can start you with a private cloud, then help you transition to a hybrid cloud, and finally to a public cloud.
In the latest eX5 announcement, IBM's x86-based servers can run 78 percent more virtual machines per VMware license dollar. This will give IBM an advantage as HP shifts from Itanium to an all x86-based server line.
Network Attached Storage
There seems to be a shift away from FC and iSCSI towards NAS and FCoE storage networking protocols. This bodes bad for HP's acquisition of LeftHand, and Dell's acquisition of EqualLogic. IBM's SONAS for large deployments, and N series for smaller deployments, will compete nicely against HP's StorageWorks X9000 system.
Storage on Paper no longer Eco-friendly
HP beats IBM when you include consumer products like printers, which some might consider "Storage on Paper". At IBM, we often joke that 96 percent of HP's profits come from over-priced ink cartridges. With the latest focus on the environment, people are printing less. I have been printing less myself, setting my default printer to generate a PDF file instead. There are several tools available for this, including [CutePDF] and [BullZip]. As IBM employees switch from Microsoft Office to IBM's [Lotus Symphony], it has built-in "export-to-PDF" capability as well. People are also going to their local OfficeMax or CartridgeWorld to get their cartridges refilled, rather than purchase new ones. That has to be hurting HP's bottom line.
Don't Forget About Storage Management
The leading storage management suites today are IBM's Tivoli Storage Productivity Center and EMC's Control Center. HP's Storage Essentials doesn't quite beat either of these, and management software is growing in importance to more and more customers.
Well, it's Tuesday again, and you know what that means? Announcements!
Today, IBM's announcements are designed to change the economics of big data analytics, cloud, mobile and social media.
[Software Defined Environments] require [Software Defined Storage], combining storage virtualization with open, extensible, industry-led interfaces. The IBM SmartCloud Virtual Storage Center (VSC) and IBM Storwize Family are the market leaders in storage virtualization. SmartCloud VSC, Storwize Family, and XIV support the industry-led OpenStack interfaces.
Here are some of the announcements today:
IBM Storwize® Family
The [SAN Volume Controller] was first introduced 10 years ago, in 2003. Today, clients enjoy these storage virtualization capabilities across a variety of offerings, known collectively as the [IBM Storwize Family].
IBM adds a new member to the Storwize Family. In addition to SAN Volume Controller, Storwize V7000, Storwize V7000 Unified, Flex System V7000, Storwize V3700, and Storwize V3500, IBM is announcing the [IBM Storwize V5000]. Here's a quick side-by-side comparison:
Scalability: Maximum configuration
Four control enclosures clustered together, 36 expansion enclosures, 960 drives, 64GB cache
Two control enclosures clustered together, 12 expansion enclosures, 336 drives, 32GB cache
One control enclosure, 4 expansion enclosures, 120 drives, 8GB cache upgradeable to 16GB, optional Turbo performance
8Gbps FCP and 1GbE iSCSI standard; optional 10GbE iSCSI/FCoE.
Can upgrade to Storwize V7000 Unified by adding NAS File Modules to add support for CIFS, NFS, HTTPS, SCP and FTP protocols
1GbE iSCSI, 6Gbps SAS, 8Gbps FCP and 10GbE iSCSI/FCoE Standard
1GbE iSCSI, 6Gbps SAS standard; optional 8Gbps FCP and 10GbE iSCSI/FCoE
Storage virtualization/Data Migration
Internal virtualization, Data Migration standard; optional external virtualization
Internal virtualization, Data Migration standard; optional external virtualization
Internal virtualization, Data Migration (external devices can be attached to ingest data only) standard
optional Metro Mirror, Global Mirror, Global Mirror with Change Volumes
optional Metro Mirror, Global Mirror, Global Mirror with Change Volumes
optional Metro Mirror, Global Mirror, Global Mirror with Change Volumes
Sub-LUN Automated Tiering
Easy Tier standard
optional Easy Tier
optional Easy Tier
VMware VAAI, VASA, vCenter plug-in, and OpenStack Cinder APIs standard
VMware VAAI, VASA, vCenter plug-in, and OpenStack Cinder APIs standard
VMware VAAI, VASA, vCenter plug-in, and OpenStack Cinder APIs standard
Storwize V7000, V5000 and V37000 now support larger 800GB SSD drives. Previously, they only support SSD drives up to 400GB.
VMware 5.5 and VASA support. VMware ships every release with built-in support for all members of the IBM Storwize Family, but it bears repeating here just in case you were interested. IBM is a leading reseller of VMware, so it makes sense for IBM's storage devices to support everything that VMware customers could possibly want in terms of VMware integration. IBM SmartCloud VSC, Storwize Family, and XIV Storage System are no exception!
New IP-based replication driving lower costs for replication. Previously, Metro Mirror, Global Mirror and Global Mirror with Change Volumes were FCP-based, and many clients bought extra equipment to run FCP packets over long-distance IP (known as FCIP). Now, clients can replicate across distnace natively without FCIP routers, and use IP-based connections natively.
In my blog posts covering [Edge 2013 - Day 3 Solution Center], I mentioned that IBM has certified Bridgeworks' SANSlide 150SVCV7K unit that provides a Riverbed-like WAN Optimization for long-distance replication. Now, IBM has fully integrated Bridgeworks' SANSlide network optimization technology directly into Storwize Family!
All members of the Storwize Family will support 1GbE remote disk replication, and this will be extended to 10GbE support at a later date.
The [Storwize V3700] is now offered in 48-volt Direct Current (DC) models, [NEBS/ETSI compliance] for Telecommunications companies that require this, and now support 4TB drives.
When we introduced [IBM SmartCloud Storage Access] in February, it was to offer self-service, automated policy-based provisioning for file storage on the SONAS and Storwize V7000 Unified. Today, we add self-service, automated policy-based provisioning for block storage. The first products to be supported are SmartCloud VSC, the entire Storwize Family, and XIV Storage Systems. In addition to the web portal, the Storage Cloud Integration API enables 3rd party ISV applications to support SmartCloud Storage Access.
Storage admins will no longer need to be bothered with tedious provisioning requests, freeing up more time for them to work on more strategic, transformational projects.
[IBM SmartCloud Virtual Storage Center] was introduced last year, combining SAN Volume Controller, Tivoli Storage Productivity Center, Tivoli FlashCopy Mangaer and the Storage Analytics Engine into a single license. The initial offering provided the cross-platform "Tiered Storage Optimization" that provided recommendations for what LUNs should be moved from one disk array to another to manage performance vs. cost. Today, IBM is first to market with an automated version, moving LUNs automatically from one disk array to another.
[SmartCloud Enterprise Object Storage] is switching from 3rd-party Nirvanix to its internal IBM Softlayer. This one involves more in-depth explanation which I will save for another post.
IBM XIV Storage System
As part of the [due diligence] team for IBM to acquire the XIV company back in 2007, I am glad to see how this system has evolved since then. I have certainly [blogged quite a bit on XIV] over the years.
Earlier this year, IBM introduced Hyper-Scale Mobility which allows the storage admin to move LUNs non-disruptively from one XIV frame to another. Today, Hyper-Scale Cross-system Consistency Groups allows you to have snapshots of collections of volumes across multiple XIV frames, up to 3PB of capacity snapped at the same instance of time.
The current supported releases of OpenStack are Folsom and Grizzly, and the newest release is Havana. XIV now offers OpenStack Cinder interfaces at the Havana level.
XIV now offers a RESTful API for monitoring and provisioning. [REST] is a de-facto standard in WEB services and cloud implementations. XIV's RESTful API is a programmatic management interface that follows REST principles:
Resources are identified by global identifiers (URIs)
Data is sent as JSON/XML over HTTP
Manipulations of resources are done by HTTP methods (GET, PUT, POST, DELETE)
The interface is Stateless and Hypertext driven
The interface is universally supported, programming language and platform agnostic. For monitoring, the following GET example could show the list of volumes on a particular XIV storage system:
For provisioning, the following PUT example could create "vol1" on that XIV storage system.
IBM SmartCloud Storage Access to allow self-service provisioning (see the SmartCloud section above).
Data-at-Rest encryption, using Self-Encrypting Drives (SED). XIV will encrypt the data, and IBM's Security Key Lifecycle Manager (SKLM) or Tivoli Key Lifecycle Manager (TKLM). If you have an XIV already, you may already have SED drives ready to use! The XIV will also encrypt the data on the SSD drives used for persistent read-cache.
Other new and enhanced offerings
For our mainframe clients, the Virtualization Engine TS7700 now supports 60 percent more capacity, and can now support 8Gbps FICON attachement.
N series N3000, N6000 and N7000 support new disk drive types and sizes, as well as Data OnTap 8.2 Cluster mode. You can now lash together up to 16 N series together into a SONAS-like single system image.
Cisco MDS 9710 Multilayer Director for IBM® System Networking is a new 16 Gbps SAN director with robust security to support multi-tenancy cloud configurations.
Whew! That is a lot of things to discuss in one post. Since they were all related, I did not want to split it up into parts.