Tony Pearson is a Master Inventor and Senior IT Architect for the IBM Storage product line at the
IBM Systems Client Experience Center in Tucson Arizona, and featured contributor
to IBM's developerWorks. In 2018, Tony celebrates his 32th year anniversary with IBM Storage. He is
author of the Inside System Storage series of books. This blog is for the open exchange of ideas relating to storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
My books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
Well, it's Tuesday again, and you know what that means? IBM Announcements! There were a lot of IBM Power System announcements on Tuesday, so the IBM Power team asked us to wait until Thursday to post about all of the IBM storage announcements, to avoid overwhelming excitement levels with the press and analysts.
(FTC Disclosure: I work for IBM. I have either worked on the code, developed marketing materials, and/or represented each of the products below in my professional capacity. This blog post can be considered a "paid celebrity endorsement")
A few months ago, IBM re-factored its internals. Spectrum Virtualize will continue to support its legacy storage pools, but also offered "Data Reduction Pools", or "DR pools" for short. At the time, this supported only Thin Provisioning and Compression. See fellow blogger Barry Whyte's post on [Data Reduction Pools] for more details.
Spectrum Virtualize 8.1.3 release now adds Data Duplication and RESTful API support for the Spectrum Virtualize family, including SAN Volume Controller, FlashSystem V9000 and Storwize products. These features also apply to Spectrum Virtualize as software only, and to Spectrum Virtualize for the Public Cloud.
Data Deduplication is a form of data footprint reduction. Like the deduplication in Spectrum Protect and FlashSystem A9000/R products, Spectrum Virtualize will use SHA1 hash codes to identify duplicate 8K blocks. If the hash code of the block about to be written does not match any existing hash code previously written to the cluster, it is considered unique data.
Legacy storage pools supported three kinds of volumes: fully-allocated, thin-provisioned, and compressed-thin volumes. The new DR pools support five kinds: fully-allocated, thin-provisioned, deduped-thin, compressed-thin, and deduped-compressed-thin volumes.
The new deduplication feature is included at no additional charge with the base Spectrum Virtualize license.
The RESTful API enables storage admins to easily automate common tasks with industry-standard tools. RestAPI support is available to interface with the command-line interface (CLI), create vDisk volumes and generate views normally available through the CLI, and secure authentication to the IBM Spectrum Virtualize family.
The SAN Volume Controller, FlashSystem V9000 and Storwize family now also support 12TB drives for internal storage. These are 7200 rpm 3.5 inch drives that can be in the 2U 12-bay or 5U 92-bay expansion drawers, or directly in the 12-bay Storwize controllers. Spectrum Virtualize 7.8.1 is the minimum level to support these high-capacity disks.
IBM Spectrum Virtualize for Public Cloud, available on IBM Cloud, has been enhanced to support a full eight node cluster (four node-pairs, or "I/O Groups" as they are called). This can be used as a target for remote mirror from your Spectrum Virtualize cluster on premises.
IBM offers data footprint reduction, high availability, and technical refresh guarantee programs for these products. See Ernie Pitt's blog post on [Peace of Mind with IBM Storage].
IBM Spectrum Scale 5.0 is highly scalable file and object storage system. It is available as software, pre-built appliances, and in the Cloud.
The pre-built appliances are called "Elastic Storage Server", combining Spectrum Scale software on two IBM Power servers with drawers of flash or disk drives.
IBM introduces two new "Hybrid" models to the ESS family. GH14 has one 2U drawer with 24 Solid State Drives (SSD) combined with four 5U drawers with 7200rpm spinning disk. The GH2R has two 2U drawers with four 5U drawers.
Like the GS models, the SSD are either 3.84TB or 15.3TB capacities. The 5U drawers are similar to those in the GL models, either 4TB, 8TB or 10TB capacities.
A new Enterprise Slim Rack (S42) is now available to hold these. The S42 is available for all ESS orders, including the GS, GL and new GH models.
IBM has shortened the name of "Spectrum Control Storage Insights" to just "Storage Insights" and made it available in two flavors: Storage Insights, and Storage Insights Pro.
Storage Insights is a no-cost cloud Artificial Intelligence (AI) service that provides common monitoring capabilities to all of your IBM block-level storage, including IBM FlashSystem, SAN Volume Controller (SVC), Storwize, DS8000 models and IBM XIV Storage Systems. Here are some of the capabilities offered:
View the health, performance, and capacity of all your IBM-supported devices from a single place
Filter storage device events to help you focus on the things that require your immediate attention
Act on predictive insights provided by device intelligence before anomalies have an impact on service levels
Use actionable data you get to resolve more issues on your own
Open and view IBM support tickets
Enable IBM Support to automatically collect log packages with no interaction with the client
IBM Storage Insights Pro includes everything in Storage Insights as well as these additional capabilities. This is a fee-based cloud service, licensed per TiB per month, for the added functionality:
Business impact analysis
Data placement optimization with tier planning
Capacity optimization with reclamation planning
Supports file and object storage, including IBM Spectrum Scale, Elastic Storage Server (ESS), and IBM Cloud Object Storage (IBM COS)
Both Storage Insights and Storage Insights Pro use a "data collector" that runs on premises. This can be any bare metal server or Virtual Machine running Windows, Linux or AIX operating system connected to the SAN, with access to the Internet to upload the data to the IBM Cloud.
If you have IBM block storage today, there is no reason not to try this out. You can download the "data collector" and start using Storage Insights right away. If you like it, consider upgrading to Storage Insights Pro, or the full on-premise Spectrum Control product.
This session had four parts. First, an overview of "Data Footprint Reduction" technologies, like compression, data deduplication, space-efficient snapshots and thin provisioning.
Second, a look at how these technologies can get storage administrators in trouble. Much like airlines selling more tickets than seats on the airplane, storage administrators may over-provision based on data reduction estimates, and then suddenly run out of storage capacity.
Third, an overview of IBM FlashSystem A9000 and A9000R products, often referred to as "A9000/R" to cover both as a family. These models offer data footprint reduction for all data.
Finally, I explain how the Hyper-Scale Manager GUI can help with reporting and analytics to avoid these risks. This GUI is available for the FlashSystem A9000/R, as well as XIV Gen3 and Spectrum Accelerate software clusters.
Special thanks to Rivka Matosevich for her help in preparing this presentation.
The Pendulum Swings: Understanding Converged and Hyperconverged Integrated Systems
With IBM's partnership with Cisco for VersaStack, and Nutanix for the IBM Power systems, this has become a particularly popular topic.
I started with an overview of the last 50 years of storage evolution, from internal storage and external storage to NAS and SAN storage networks. An estimated 96 percent of the storage in corporate data centers are connected via NAS or SAN networks.
More recently, people have been willing to give up all those gains for something simpler, less powerful, less reliable, less expensive. Enter Converged and Hyperconverged Systems. IBM PureApplication and VersaStack lead the pack for Converged Systems, along with IBM Spectrum Scale, Spectrum Accelerate and Nutanix on IBM Power Systems for Hyperconverged Integrated Systems.
We had 1,600 attendees, much higher than expected. This is a good sign, when you consider IBM just had its "Think 2018" conference last March, and Dell EMC had their big conference the same week in Las Vegas.
When people asked me what was the main difference between "Think 2018" and "IBM Technical University", I explain it as follows:
Think 2018 is a big conference focused on uni-directional communication. IBM executives present the corporate line repeatedly to large audiences. Its size and scale means they can have big name bands and celebrity speakers.
IBM Technical University is a smaller conference focused on bi-directional communication. Audiences are small and encouraged to ask questions. Demos, Labs and Meetups allow for conversations with IBM technical experts. There are no crowds in the hallways to hamper ad-hoc side conversations. The IBM speakers listen to the clients concerns and bring that feedback to development.
Confused yet? Fortunately, the speaker Mark Rader, IBM Z, focused entirely on the z/OS platform version of these tools.
Storage Meetup: Cloud and Object Storage
In past years, the conference would organize three huge rooms, one for IBM Z, one for IBM POWER, and the last for IBM Storage. These would be Q&A panels, with a dozen experts at the front of the room, and a large audience asking questions.
This year, these were all split up into smaller "Meetup" sessions. There were Meetups for Spectrum Protect, Spectrum Scale, Encryption, IBM i, z/OS, Power, FlashSystem, TS7700, DS8000, Blockchain, DevOPS, Machine Learning, Spectrum Virtualize.
Andy Kutner and I led the Storage Meetup for "Cloud and Object Storage". Andy and I had both done several presentations on these topics during the week, and we were able to handle the questions that came from the audience. Here is a sample:
Is IBM Spectrum Virtualize Transparent Cloud Tiering mature enough to use now?
It is unfortunate that IBM chose "Transparent Cloud Tiering" (TCT) to describe four implementations on four different product families. The one thing they have in common is that they send data to the Cloud or IBM Cloud Object Storage. The TCT feature of Spectrum Virtualize, including SAN Volume Controller (SVC), Storwize and FlashSystem V9000 was made generally available for use last year.
Our use of S3FS does not perform well?
S3FS was developed to allow file system access through the FUSE driver for Linux and MacOS operating systems. It was not optimized for performance, and was meant as a convenience instead.
Does Spectrum Copy Data Management support the Cloud?
Yes, Spectrum CDM works with a variety of IBM and non-IBM storage to take volume snapshots that can be used for DevOPS, dev/test, and other purposes. These snapshots can be moved to the cloud to make them more broadly accessible to developers.
How do we size backup configurations to the Cloud
Several backup software support IBM Cloud and IBM Cloud Object Storage, including Veritas NetBackup and Commvault Simpana. For IBM Spectrum Protect, IBM has published "blueprints" now that include Cloud configurations that have done all the sizing work for you.
When should we consider using a Co-location facility instead of Public Cloud
There are pros and cons for each, and there are also options in between. IBM Cloud offers "Dedicated" bare-metal servers, giving you complete control, but still publicly accessible. IBM Cloud also offers "Private" bare-metal behind your firewall.
Can IBM COS Vault Mirroring employ the Aspera file transfer protocol?
Andy and I were stumped on this one. Andy agreed to get back to the attendee on this.
Can IBM COS support Spark Analytic workloads
Yes, IBM COS supports [Stocator], a high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.
We are currently using IBM FileNet to Centera, can the data be moved to IBM COS?
Absolutely, not only does FileNet offer utilities to make this possible, IBM has partnered with service providers that offer fast data movement from Centera to IBM COS for a variety of specific applications, including FileNet.
What are the key selling points over Dell EMC Centera?
Basically, IBM COS is more scalable, performs faster, at a lower Total Cost of Ownership (TCO). Contact your local storage seller or IBM Business Partner for a full presentation!
Is there a database to quickly search through all of the valuable metadata stored in IBM COS and other repositories?
Great idea! I will pass this on to development.
How does IBM COS protect against disk failure or data corruption?
IBM COS slowly rolls through all the data checking the integrity. Checksums are validated, and any corrupted or missing slices are reconstructed using Erasure Coding.
Who competes with IBM COS?
IBM ranks #1 in Object Storage. There are several competitors. Established storage companies like Dell EMC, NetApp, Hitachi Vantara, and Hewlett Packard Enterprise (HPE) Simplivity have object storage offerings. Companies also have attempted to build object stores from open source code, OpenStack Swift and Ceph.
How can IBM Business Partners demonstrate IBM COS to clients?
The IBM Systems Client Experience Portal [ISCEP] provides a list of demos available for all IBM storage products, including IBM COS.
Event Night: Universal Studios
IBM rented out a section of Universal Studios! We were welcomed by various characters to take pictures with! Here are Scooby Doo and Shaggy!
There were several rides to chose from. While waiting for the "Race through New York with Jimmy Fallon" ride, we were entertained by these five acapella singers doing popular rap songs. Laura poses for a picture with two Minions.
The other two rides were "Transformers" and "The Mummy". After Transformers ride, we could take pictures with Bumblebee, one of the auto-bots. I posed with Ahmanet, the mummy played by Sofia Boutella in the 2017 movie.
There was also plenty of food, representing Italian, Mexican, Greek, Chinese and American fare.
You can follow along with Twitter hashtag #IBMtechU, or follow me at @az990tony.
This week, I am in Orlando, Florida for the [IBM Technical University], with focus on IBM storage, IBM Z mainframes and IBM Power servers. This is my recap for Day 3 breakout sessions.
VersaStack for Containers: IBM Cloud Private and Spectrum Access
Chris Vollmar, IBM Canada, presented all day today. In this session, he explained how "Spectrum Access" was not a product, but rather a blueprint of best practices on how to install the IBM Cloud Private and Spectrum Connect software on the VersaStack solution.
Leveraging IBM Cloud Object Storage for z/OS
Louis Hanna, IBM Z Software, presented this session. I was expecting it to either cover the DS8000 Transparent Cloud Tiering, or direct access to IBM Cloud Object Storage, but it turns out neither!
Instead, Louis talked about [IBM Cloud Tape Connector for z/OS], which mimics tape drive interfaces that can be used to move disk and tape data to a public cloud, or to IBM Cloud Object Storage on premises.
Information Lifecycle Management: Why Archive is Different than Backup
Can you believe there are still companies out there keeping backup tapes for seven years and pretending that this meets their long-term retention requirements?
What happens when you try to recover those tapes, and you need the right server, the right operating system, and the right application software to make sense of it all?
Backups should not be used in this manner. Rather, backups are to recover from recent hardware failure or data corruption only. If you have are keeping backups longer than 90 days, you are probably doing something wrong.
Archiving, on the other hand, is an intelligent process for managing inactive or infrequently accessed data, that still has value, while providing the ability to preserve, search and retrieve the information during a specified retention period.
However, some of the product names have changed, so I thought it would be good to do a fresh update on this topic for this conference.
Becoming the person you published on LinkedIn
Frank Degilio, IBM Distinguished Engineer, presented this "Career Development" session. IBM Technical University is not just for technical education, it also offers sessions of general interest to help round out personal skills.
Frank explained that to rise the corporate ranks, you need to learn to communicate, to collaborate, and to network with others. Technical workers should be "T-shaped", with the top part of the letter "T" representing broad, general skills, and the lower part representing deep technical skills in a specific area.
You can follow along with Twitter hashtag #IBMtechU, or follow me at @az990tony.
This week, I am in Orlando, Florida for the [IBM Technical University], with focus on IBM storage, IBM Z mainframes and IBM Power servers. This is my recap of afternoon breakout sessions on Day 2.
Spectrum NAS 101 and key use cases
Chris Maestas presented IBM's latest addition to the Spectrum Storage family of Software-Defined Storage. Spectrum NAS was written from scratch in C/C++ language, instead of using open source code like SAMBA. It supports both NFS and SMB protocols.
Like IBM Cloud Object Storage, the Spectrum NAS software is shipped with the operating system, so you have a single ISO to run everything. You start with four nodes and can grow capacity and performance as needed by adding more nodes. All nodes have identical roles.
All of the storage is internal. Spectrum NAS uses DRAM memory, NVMe-based Solid State Drives (SSD), and spinning disk HDD. The NVMe drives must support at least five Drive Writes per Day (DWPD).
Each Spectrum NAS node can handle 2,000 connections, and up to 4,000 connections during fail-over processing. With 10GbE bandwidth, you can migrate 100 TB/day from other NAS devices to Spectrum NAS. If you want to try out Spectrum NAS yourself, there is a 60-day free trial offer now available. There are a collection of videos on the [Spectrum NAS YouTube channel] to walk you through the installation process.
Clients are Hyper for Hyperconverged
Marc Richardson and Bruce Jones, both from IBM Cognitive Systems, presented this client case study on successful deployment of IBM Hyperconverged Systems powered by Nutanix, often referred to as the "IBM CS" models of the POWER server line. The covered three use cases:
Modernize to Private Cloud
IBM CS models use the Nutanix Acropolis Hypervisor (AHV) to run Ubuntu and CentOS little-Endian virtual machines on POWER. The speakers claimed that they can run 50 percent faster, and 88 percent more workloads per core, than traditional x86 methods. IBM has made statement of direction that IBM CS models will support AIX 7.2 virtual machines later this year.
The IBM CS models can also run IBM Cloud Private, a collection of software that supports Docker and Kubernetes.
Simplify the Data Center
The client was not happy with the high prices of their external, high-end storage systems. When you add another IBM CS models to the cluster, you get more storage capacity and CPU capability at the same time, in lock step. What could be simpler?
Infrastructure for Modern Data Workloads
IBM CS models can run traditional Db2 and WebSphere applications. The client also reduced their costs by switching from expensive Oracle databases to open source databases like MongoDB and EnterpriseDB Postgres.
I was honored with being selected for this week's poster session. I was poster 16, explaining the What, Why and How of IBM Cloud Object Storage. Here is am posting with my colleague Heather Allen, IBM.
Kelly Groff, IBM FlashSystem, had poster 15 on how the embedded compression on the latest FlashSystem 900 models have almost no performance impact. Jeff Barnett, IBM, had poster 14 for IBM's Pay-as-you-grow Storage Utility Pricing.
Barry Whyte drew large crowds with his poster 13 on NVMe. Andy Kutner, IBM, had poster 11 on IBM Cloud Object Storage.
Fahima Zamir, IBM, had poster 29 on VersaStack solution, which combines best-of-breed x86 servers and switches from Cisco with IBM storage into a converged system. Sharie Mims from VSS is an IBM Business Partner.
You can follow along with Twitter hashtag #IBMtechU, or follow me at @az990tony.