This blog is for the open exchange of ideas relating to IBM Systems, storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
Tony Pearson is a Master Inventor, Senior IT Architect and Event Content Manager for [IBM Systems for IBM Systems Technical University] events. With over 30 years with IBM Systems, Tony is frequent traveler, speaking to clients at events throughout the world.
Lloyd Dean is an IBM Senior Certified Executive IT Architect in Infrastructure Architecture. Lloyd has held numerous senior technical roles at IBM during his 19 plus years at IBM. Lloyd most recently has been leading efforts across the Communication/CSI Market as a senior Storage Solution Architect/CTS covering the Kansas City territory. In prior years Lloyd supported the industry accounts as a Storage Solution architect and prior to that as a Storage Software Solutions specialist during his time in the ATS organization.
Lloyd currently supports North America storage sales teams in his Storage Software Solution Architecture SME role in the Washington Systems Center team. His current focus is with IBM Cloud Private and he will be delivering and supporting sessions at Think2019, and Storage Technical University on the Value of IBM storage in this high value IBM solution a part of the IBM Cloud strategy. Lloyd maintains a Subject Matter Expert status across the IBM Spectrum Storage Software solutions. You can follow Lloyd on Twitter @ldean0558 and LinkedIn Lloyd Dean.
Tony Pearson's books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
The developerWorks Connections Platform is now in read-only mode and content is only available for viewing. No new wiki pages, posts, or messages may be added. Please see our FAQ for more information. The developerWorks Connections platform will officially shut down on March 31, 2020 and content will no longer be available. More details available on our FAQ. (Read in Japanese.)
Now the Thanksgiving Holidays are over, its time to catch up on some Storage Announcements! These actually came out November 20, but many readers were busy with Thanksgiving Thursday, Black Friday, Small Business Saturday, Cyber Monday and Give Back Tuesday, they might have been missed.
IBM DS8880/F Storage Systems
There were three items this time for the DS8880 family, for both Generation 1 and Generation 2 models.
The first is the expansion on the type of high capacity flash drives available for the system. With this announcement, IBM has added the high-capacity 15.3 TB flash drive set as an available option. With this, the maximum flash storage capacity supported per high performance flash enclosure (HPFE) pair is now 737 TB raw.
The second is the addition of write operations for zHyperLink. zHyperLink is a short distance, mainframe attach link that is intended to accelerate Db2 v12 for z/OS. Low I/O latencies deliver value through improved workload elapsed times and faster transactional response times, and contribute to lower scaling costs.
The third is the ability to leverage IBM Storage Insights cloud-based service for your DS8880 systems. It supports the next-generation Call Home infrastructure that enables continuous updates of the system's configuration and health. IBM Storage Insights simplifies uploading logs, speeds resolution with online configuration data, and provides an overview of open tickets, all in one place. Clients who have been using IBM Storage Insights with their their IBM Storwize and IBM FlashSystem A9000/R systems can now extend this to include their DS8880 systems.
The TS4500 library now supports IBM TS1080 tape drive model F8S. This is an LTO8 Ultrium tape drive enabled with single-mode fiber and designed for the heavy demands of backup and archive tape storage applications that require high bandwidth over long distances. TS1080 Model F8S can be installed in a TS4500 tape library and delivers maximum throughput with a native data transfer of up to 360 Mbps.
There were two other enhancements for the TS4500 were:
A new action "Export logs direct to IBM" on the web. The TS4500 will collect library logs and send via Call Home to IBM, eliminating download to the laptop or system. This improves how clients engage with IBM support teams.
A new intelligent Power Distribution Unit (PDU) with three-phase [wye] or single-phase input to provide power to the local frame control assembly. This new PDU will offer power and environmental monitoring features in the future. So stay tuned.
IBM continues its lead in tape innovation with the IBM TS1160 Enterprise Tape Drive model (60F)! Delivering a 100% increase over the TS1150 series of tape drives, the TS1160 comes in at 20TB of cartridge capacity on the new JE media type with a native data rate of 400 Mbps, or up to 900 Mbps compressed. The new TS1160 will be available for the TS4500, the TS3500 (Model 60G) as well as a stand alone design with Fibre Channel interfaces.
Of course, the new TS1160 will support the reuse of existing JD/JZ/JL media with the capability to read and write TS1150 and TS1155 formatted media, and format the same media with up to 15 TB uncompressed capacity. It will also support IBM Spectrum Archive with the LTFS format for tape. Leveraging these larger tape capacities for active archives, backups, and other long term retention requirements.
IBM Spectrum Archive can move data from Spectrum Scale flash and disk storage to less expensive tape. IBM Spectrum Archive is based on IBM Linear Tape File System (LTFS) and enables fast and easy data retention to reduce the total cost of ownership for active archive storage. Both the IBM Spectrum Archive Enterprise Edition (EE) V1.3, and the Library Edition (LE) V2.4.1, will support the new enterprise-class TS1160 tape drive.
The Data Management Edition (DME) of Spectrum Scale does not require you to separately purchase client, server, or FPO socket-based, MIPS-based, or PVU-based licenses. Instead, you can simply license the total storage capacity managed by IBM Spectrum Scale. DME is now available through IBM Shopz. Shopz is a web service to order z Systems software, manage software licenses, and view software inventory. It provides an easy way to plan and order your product ServerPac or CBPDO, running on z.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(This week in the USA there are big elections. However, I am far, far away in Zurich, Switzerland writing IBM Redbooks.)
Storwize V7000 Model 724
The next generation of the Storwize V7000, officially model 724, supports NVMe and FC-NVMe (NVMe/FC) on 16Gbps adapters, and iSER on 25GbE adapters. You can cluster Storwize V7000 Gen2+, Storwize V7000 model 724, and FlashSystem 9100 series into the same cluster. There are some differences, shown in this table below:
The latest software is now available for SVC, FlashSystem 9100 and V9000, and Storwize V7000 and V5000.
NVMe over Fibre Channel support on 16 Gb Fibre Channel adapters extends the simplicity, efficiency, and end-to-end NVMe model where NVMe commands and structures are transferred end to end, requiring no translations. This is often written as FC-NVMe or NVMe/FC.
Full IP-based quorum delivers support for administrators looking to consolidate their infrastructure over Ethernet. Previously, clients used a mix of IP-based and physical quorum disks.
Increased host mappings to 64K, more than triple from the previous limitation of 20K.
IBM Spectrum Insights provides a call home protocol that uses IBM IP to deliver a more robust path and higher bandwidth/higher frequency data transmission, with end-to-end confirmation of receipt.
Single copy vdisk expand with format enables administrators to expand a vdisk without migrating the data off and back on. For example, if you want to expand a 100GB LUN to 150GB, you can do this, and the hardware will format the additional 50GB capacity.
iSER support for host attachment with 25 GbE adapters expands the host connectivity options for SVC, FlashSystem V9000 and Storwize V5000. This is similar to the v8.2 support for Storwize V7000 and FlashSystem 9100.
Clustering support over Ethernet using RDMA enables IBM Spectrum Virtualize software to run over multiple types of technology, including FlashSystem 9100 and V9000, SVC, and Storwize V7000. This was the last hold-out for all-Ethernet shops. IBM supported Ethernet host attachment, and Ethernet-based back-end storage, and Ethernet-based replication mirroring, so the last piece is now complete, Ethernet-based node-to-node communications needed for clustering.
We have a new member of the ever-growing IBM Spectrum Storage family! IBM Spectrum Discover is modern metadata management software that delivers data insight for petabyte-scale, unstructured data.
IBM Spectrum Discover easily connects to IBM Cloud Object Storage (COS) and IBM Spectrum Scale and Elastic Storage Server (ESS) to rapidly ingest, consolidate, and index metadata for billions of files and objects, providing a rich layer of metadata on top of these storage sources. IBM plans to extend support to other platforms next year.
This metadata enables data scientists, storage administrators, and data stewards to efficiently manage, classify, and gain insights from massive amounts of unstructured data. The insights gained accelerate large-scale analytics, improve storage economics, and help with governance to create competitive advantage, speed critical research, and mitigate risk.
This initial release is labeled v2.0 as IBM has deployed this in beta form already at various client locations. Here are some key highlights:
Event-notifications and policy-based workflows to automate metadata ingestion and metadata indexing at a petabyte scale
Fine-grained views of storage consumption based on a wide range of system and custom metadata
Fast, efficient search through petabytes of data, resulting in highly relevant results for large-scale analytics
Ability to quickly differentiate mission-critical business data from data that can either be deleted or moved to a cheaper, colder tier
Policy-based custom tagging that enables organizations to classify and categorize data, and align this data with the needs of the business
A software developers kit (SDK) to build action agents that extract metadata from file headers and content, automate data movement, and provide integration to open source software, such as Apache Spark, Apache Tika, PyTorch, Caffe and TensorFlow, to facilitate data identification and speed large-scale data processing
The latest IBM FlashSystem 900 comes in two models, the AE3 "full purchase" model, and the UF3 "storage utility pricing" model where you pay less initially, and then more as you consume more of the capacity. They are the same hardware, just licensed differently.
Currently, IBM offers FCP or InfiniBand host attachment, with up to twelve 3.6TB, 8.5TB or 18TB modules (PCiE card). A full 2U drawer would be configured as 10+P+S RAID5 for high availability and data protection.
Each module offers embedded compression chip, but modules only had enough DRAM cache to allow a maximum of compressed 22TB effective data, so while the 3.6TB and 8.5TB could compress data up to 2.5x, the 18TB card was somewhat limited at 1.2x, which might be fine for some already-compressed data like MP3 audio, or JPEG photos.
This month, IBM offers new XL MicroLatency Modules, 18TB cards with enough DRAM cache to support 44TB compressed data, up to an effective 2.4x compression ratio. A full twelve-module drawer could hold up to 440TB of effective capacity.
IBM also now offers a quad-port 16Gb FCP card that supports both SCSI and NVMe commands over fabric. This is often denoted as either FC-NVMe or NVMe/FC. The FlashSystem 900 already supported NVMe-OF for InfiniBand (see my blog post [IBM February 2018 Announcements])
IBM Cloud Tape Connector for z/OS is a software-defined storage solution that provides an alternative to virtual tape libraries like the TS7760. Here are some highlights:
Robust virtual tape emulation solution with e-vaulting to cloud-based offsite storage for cold, archival, or backup data. Virtual tape emulation simulates IBM compatible tape controllers, tape drives, and tape volumes, maintained on any IBM z/OS-compatible disk system, such as IBM DS8000. IBM Cloud Tape Connector for z/OS provides several vault, transfer, and recovery options to support business continuity and resiliency.
Sequential z/OS data set cloud storage and retrieval. Sequential data sets stored on disk or flash storage can be moved to the cloud by IBM Cloud Tape Connector for z/OS without the requirement of performing a tape-write operation.
Automatic application recall of data from cloud, whether e-vaulted through virtual tape emulation or copied directly to the cloud.
Pervasive encryption support. This feature enables enterprises to ensure that any data copied to the cloud is encrypted before it is transmitted, automatically protecting and handling the encryption keys.
Support for IBM Cloud Object Storage using S3 protocol, as well as Amazon S3, Hitachi HCP protocol, and EMC Elastic Cloud Service Protocol.
I was in Hollywood Florida for the IBM Systems Technical University. Here is my recap of the final two days, day 4 and 5.
The Pendulum Swings Back: Understanding Converged and Hyperconverged Systems
Once again, I presented my popular session on converged and hyperconverged systems. For converged, IBM offers IBM PureApplication systems with Power and x86 servers, as well as partnership with Cisco called VersaStack. Both support IBM Cloud Private as a platform for running applications.
For Hyperconverged, IBM offers Spectrum Accelerate and Spectrum Scale, as well as partnerships with SuperMicro that combines Spectrum Accelerate on SuperMicro x86 servers, and partnership with Nutanix for CS-models of Power servers pre-installed with Nutanix software.
Unlike other converged and hyperconverged solutions that act as isolated islands of compute and storage, IBM's solutions can be incorporated into an existing datacenter with IBM Cloud Private for orchestration, and IBM Spectrum Scale to provide common access to data.
The Seven Tiers of Business Continuity and Disaster Recovery
With all the natural disasters that happened last year in the USA, and the more recent ones all over the world, this session continues to draw a crowd.
The seven tiers range from the least expensive to most expensive. The least expensive involves restoring data from tapes stored in an offsite vault. Tape continues to be the least expensive storage medium, and can be used to bring up a company in a few days.
For faster recovery, there are options like electronic vaulting to virtual tape libraries, and now the use of Cloud storage for ubiquitous access to data from different locations.
Snapshots of entire volumes, virtual machines or databases are also quite popular. IBM offers IBM Spectrum Protect Snapshot, Spectrum Protect Plus, and Spectrum Copy Data Management for this.
Faster recovery is possible with remote mirroring. This involves sending all of the updates to a secondary location. In the event of a disaster, clients can switch processing with the data already there. IBM has over 800 clients able to do just that in less than 30 minutes.
Event Night by the Pool
Photography by Mo Reyes
While Hurricane Michael raged in upper Florida the week prior, the event coordinators were a bit nervous to offer an evening dinner event by the pool, but the weather cooperated!
Photography by Mo Reyes
I was a social butterfly, moving from table to table to talk to all of the various attendees. A light breeze and excellent food and music made for an enjoyable night!
The pool reception went on to about 10:00pm at night. IBM had lit up its logo into the pools for a great view from above. Perhaps just 30 minutes after arriving back to my hotel room, we had quite the thunderstorm! How incredibly lucky this did not happen during the event!
The following day, I presented my session on "Managing Risk with Data Footprint Reduction, a repeat of the session I did earlier that week.
This was a pleasant way to end the week! Aside from the heat and humidity being above average for October, it was a beautiful hotel in a lovely city.
Last week, I was in Hollywood Florida for the IBM Systems Technical University. Here is my recap of days 2 and 3.
Information Lifecycle Management: Why Archive is Different than Backup
Some companies keep backup copies for years and years. They think this is all they need to do to comply with government regulations for data retention. They could not be more wrong!
This session explained why keeping backups for more than a few months is a bad idea, and how to fix it with proper Information Lifecycle Management practices, the proper use of archive as an alternative to keeping backups to long, and the advantages of archives versus backup.
Storage for Rookies: Introduction to IBM Cloud Object Storage
My session on IBM COS was so popular, we repeated for the "Storage for Rookies" track. In this track, registrants attend specifically selected topics to complete a "degree". This is "University" after all!
My session was organized into three sections. First, a general overview of "Object Storage" that can be accessed via HTTP over TCP/IP networks, and how this is different from traditional block or file storage.
Second, a review of the architecture and features of IBM Cloud Object Storage, and how these can be deployed on-premises, in a hybrid cloud configuration, or use in the public IBM Cloud.
Third, how to use IBM Cloud Object Storage for various use cases, including programming languages that support object storage, NAS gateways, and backup software like IBM Spectrum Protect.
Managing Risks with Data Footprint Reduction
What happens when airlines sell more tickets than actual seats on the airplane? Travelers get upset, and sometimes the airline has to forcibly drag people off the plane.
Likewise, storage admins who over-provision storage run the risk of having application outages from out-of-space conditions. This session explained how thin provisioning, deduplication and compression can help, but at other times make things more complicated.
IBM Spectrum Scale Users Group
We had a great turn-out for this "Users Group". IBM Spectrum Scale and Elastic Storage Server grew substantially last year, and we are keeping up the momentum!
We had several presenters cover various updates, followed by cocktails!
After all that excitement, we went to Jimmy Buffett's Margaritaville for a "Storage Team" dinner. There were karaoke singers, accompanied by a live band. A fun time was had by all!