Tony Pearson is a Master Inventor and Senior IT Architect for the IBM Storage product line at the
IBM Systems Client Experience Center in Tucson Arizona, and featured contributor
to IBM's developerWorks. In 2018, Tony celebrates his 32th year anniversary with IBM Storage. He is
author of the Inside System Storage series of books. This blog is for the open exchange of ideas relating to storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
My books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(This year, both my birthday and Christmas land on Tuesday, so I am thankful that fellow IBMer Chris Vollmar was able to write this post!)
The year 2018 is not over yet, this week, IBM Storage ushers in a new era of Artificial Intelligence (AI), machine learning (ML) and deep learning (DL).
IBM Spectrum AI
IBM Spectrum AI was introduced to drive AI development and streamline the AI Data Pipeline since Data scientists need access to large, readily accessible quantities of data supported by a wide variety of data tools. Together, IBM and NVIDIA have partnered on this solution that leverages IBM Storage and the NVIDIA DGX platform.
As companies increase their focus on leveraging AI initiatives they begin by investing in powerful GPU-based servers, and looking across existing and new data sources to develop their next innovations. Most begin with a modest project or proof-of-concept, but quickly recognize the need to expand their work with more data, more initiatives and additional models efficiently. A complete solution for clients looking to leverage the x86-based NVIDIA DGX software stack, the DGX-1 is optimized for maximized GPU-accelerated training performance and including the new RAPIDS framework to accelerate data science workflow.
IBM Spectrum AI is a converged system, that combines IBM Spectrum Scale, the leading software-defined storage file system, with the leading GPU server, the NVIDIA DGX-1. It is a solution, like VersaStack, on which clients can get started and later scale-out as their needs change. This allows customers who want to work with tools like TensorFlow, PyTorch and Spark to support the entire AI data pipeline from ingest to inference and archive underpinned by the same high-performance, multi-protocol shared storage solution that provides the storage to the [#1 and #2 fastest supercomputers: Summit and Sierra].
A converged solution that is tested and tuned, IBM Spectrum AI with NVIDIA DGX provides the ready-to-deploy robust infrastructure and software that AI projects need to ramp up quickly and grow confidently. IBM Spectrum AI can support configurations from a single IBM Spectrum Scale powered IBM Elastic Storage Server (ESS) to support a few NVIDIA DGX-1 servers, to full rack of nine servers with 72 Tesla V100 Tensor Core GPUs to multi-rack configurations allow Spectrum AI to expand to meet the most strenuous of business requirements. Seamlessly expanded, IBM Spectrum AI with NVIDIA DGX has demonstrated 120 GB/s of data throughput in a single rack to support multiple user and multiple models simultaneously.
On the topic of AI, IBM Storage has also released a reference architecture for advanced driver assistance systems (ADAS) and autonomous driving (AD). The solution enables the design and deployment of end-to-end data pipelines leveraging IBM Spectrum Scale, IBM Cloud Object Storage and the newest addition to the IBM Spectrum family, IBM Spectrum Discover. This solution illustrates how IBM Storage offerings enable clients to build an end-to-end data pipeline serving the varying storage requirements of each stage in the data pipeline, from edge to transient to machine learning and deep learning.
Now the Thanksgiving Holidays are over, its time to catch up on some Storage Announcements! These actually came out November 20, but many readers were busy with Thanksgiving Thursday, Black Friday, Small Business Saturday, Cyber Monday and Give Back Tuesday, they might have been missed.
IBM DS8880/F Storage Systems
There were three items this time for the DS8880 family, for both Generation 1 and Generation 2 models.
The first is the expansion on the type of high capacity flash drives available for the system. With this announcement, IBM has added the high-capacity 15.3 TB flash drive set as an available option. With this, the maximum flash storage capacity supported per high performance flash enclosure (HPFE) pair is now 737 TB raw.
The second is the addition of write operations for zHyperLink. zHyperLink is a short distance, mainframe attach link that is intended to accelerate Db2 v12 for z/OS. Low I/O latencies deliver value through improved workload elapsed times and faster transactional response times, and contribute to lower scaling costs.
The third is the ability to leverage IBM Storage Insights cloud-based service for your DS8880 systems. It supports the next-generation Call Home infrastructure that enables continuous updates of the system's configuration and health. IBM Storage Insights simplifies uploading logs, speeds resolution with online configuration data, and provides an overview of open tickets, all in one place. Clients who have been using IBM Storage Insights with their their IBM Storwize and IBM FlashSystem A9000/R systems can now extend this to include their DS8880 systems.
The TS4500 library now supports IBM TS1080 tape drive model F8S. This is an LTO8 Ultrium tape drive enabled with single-mode fiber and designed for the heavy demands of backup and archive tape storage applications that require high bandwidth over long distances. TS1080 Model F8S can be installed in a TS4500 tape library and delivers maximum throughput with a native data transfer of up to 360 Mbps.
There were two other enhancements for the TS4500 were:
A new action "Export logs direct to IBM" on the web. The TS4500 will collect library logs and send via Call Home to IBM, eliminating download to the laptop or system. This improves how clients engage with IBM support teams.
A new intelligent Power Distribution Unit (PDU) with three-phase [wye] or single-phase input to provide power to the local frame control assembly. This new PDU will offer power and environmental monitoring features in the future. So stay tuned.
IBM continues its lead in tape innovation with the IBM TS1160 Enterprise Tape Drive model (60F)! Delivering a 100% increase over the TS1150 series of tape drives, the TS1160 comes in at 20TB of cartridge capacity on the new JE media type with a native data rate of 400 Mbps, or up to 900 Mbps compressed. The new TS1160 will be available for the TS4500, the TS3500 (Model 60G) as well as a stand alone design with Fibre Channel interfaces.
Of course, the new TS1160 will support the reuse of existing JD/JZ/JL media with the capability to read and write TS1150 and TS1155 formatted media, and format the same media with up to 15 TB uncompressed capacity. It will also support IBM Spectrum Archive with the LTFS format for tape. Leveraging these larger tape capacities for active archives, backups, and other long term retention requirements.
IBM Spectrum Archive can move data from Spectrum Scale flash and disk storage to less expensive tape. IBM Spectrum Archive is based on IBM Linear Tape File System (LTFS) and enables fast and easy data retention to reduce the total cost of ownership for active archive storage. Both the IBM Spectrum Archive Enterprise Edition (EE) V1.3, and the Library Edition (LE) V2.4.1, will support the new enterprise-class TS1160 tape drive.
The Data Management Edition (DME) of Spectrum Scale does not require you to separately purchase client, server, or FPO socket-based, MIPS-based, or PVU-based licenses. Instead, you can simply license the total storage capacity managed by IBM Spectrum Scale. DME is now available through IBM Shopz. Shopz is a web service to order z Systems software, manage software licenses, and view software inventory. It provides an easy way to plan and order your product ServerPac or CBPDO, running on z.
Well, it's Tuesday again, and you know what that means? IBM Announcements!
(This week in the USA there are big elections. However, I am far, far away in Zurich, Switzerland writing IBM Redbooks.)
Storwize V7000 Model 724
The next generation of the Storwize V7000, officially model 724, supports NVMe and FC-NVMe (NVMe/FC) on 16Gbps adapters, and iSER on 25GbE adapters. You can cluster Storwize V7000 Gen2+, Storwize V7000 model 724, and FlashSystem 9100 series into the same cluster. There are some differences, shown in this table below:
The latest software is now available for SVC, FlashSystem 9100 and V9000, and Storwize V7000 and V5000.
NVMe over Fibre Channel support on 16 Gb Fibre Channel adapters extends the simplicity, efficiency, and end-to-end NVMe model where NVMe commands and structures are transferred end to end, requiring no translations. This is often written as FC-NVMe or NVMe/FC.
Full IP-based quorum delivers support for administrators looking to consolidate their infrastructure over Ethernet. Previously, clients used a mix of IP-based and physical quorum disks.
Increased host mappings to 64K, more than triple from the previous limitation of 20K.
IBM Spectrum Insights provides a call home protocol that uses IBM IP to deliver a more robust path and higher bandwidth/higher frequency data transmission, with end-to-end confirmation of receipt.
Single copy vdisk expand with format enables administrators to expand a vdisk without migrating the data off and back on. For example, if you want to expand a 100GB LUN to 150GB, you can do this, and the hardware will format the additional 50GB capacity.
iSER support for host attachment with 25 GbE adapters expands the host connectivity options for SVC, FlashSystem V9000 and Storwize V5000. This is similar to the v8.2 support for Storwize V7000 and FlashSystem 9100.
Clustering support over Ethernet using RDMA enables IBM Spectrum Virtualize software to run over multiple types of technology, including FlashSystem 9100 and V9000, SVC, and Storwize V7000. This was the last hold-out for all-Ethernet shops. IBM supported Ethernet host attachment, and Ethernet-based back-end storage, and Ethernet-based replication mirroring, so the last piece is now complete, Ethernet-based node-to-node communications needed for clustering.
We have a new member of the ever-growing IBM Spectrum Storage family! IBM Spectrum Discover is modern metadata management software that delivers data insight for petabyte-scale, unstructured data.
IBM Spectrum Discover easily connects to IBM Cloud Object Storage (COS) and IBM Spectrum Scale and Elastic Storage Server (ESS) to rapidly ingest, consolidate, and index metadata for billions of files and objects, providing a rich layer of metadata on top of these storage sources. IBM plans to extend support to other platforms next year.
This metadata enables data scientists, storage administrators, and data stewards to efficiently manage, classify, and gain insights from massive amounts of unstructured data. The insights gained accelerate large-scale analytics, improve storage economics, and help with governance to create competitive advantage, speed critical research, and mitigate risk.
This initial release is labeled v2.0 as IBM has deployed this in beta form already at various client locations. Here are some key highlights:
Event-notifications and policy-based workflows to automate metadata ingestion and metadata indexing at a petabyte scale
Fine-grained views of storage consumption based on a wide range of system and custom metadata
Fast, efficient search through petabytes of data, resulting in highly relevant results for large-scale analytics
Ability to quickly differentiate mission-critical business data from data that can either be deleted or moved to a cheaper, colder tier
Policy-based custom tagging that enables organizations to classify and categorize data, and align this data with the needs of the business
A software developers kit (SDK) to build action agents that extract metadata from file headers and content, automate data movement, and provide integration to open source software, such as Apache Spark, Apache Tika, PyTorch, Caffe and TensorFlow, to facilitate data identification and speed large-scale data processing
The latest IBM FlashSystem 900 comes in two models, the AE3 "full purchase" model, and the UF3 "storage utility pricing" model where you pay less initially, and then more as you consume more of the capacity. They are the same hardware, just licensed differently.
Currently, IBM offers FCP or InfiniBand host attachment, with up to twelve 3.6TB, 8.5TB or 18TB modules (PCiE card). A full 2U drawer would be configured as 10+P+S RAID5 for high availability and data protection.
Each module offers embedded compression chip, but modules only had enough DRAM cache to allow a maximum of compressed 22TB effective data, so while the 3.6TB and 8.5TB could compress data up to 2.5x, the 18TB card was somewhat limited at 1.2x, which might be fine for some already-compressed data like MP3 audio, or JPEG photos.
This month, IBM offers new XL MicroLatency Modules, 18TB cards with enough DRAM cache to support 44TB compressed data, up to an effective 2.4x compression ratio. A full twelve-module drawer could hold up to 440TB of effective capacity.
IBM also now offers a quad-port 16Gb FCP card that supports both SCSI and NVMe commands over fabric. This is often denoted as either FC-NVMe or NVMe/FC. The FlashSystem 900 already supported NVMe-OF for InfiniBand (see my blog post [IBM February 2018 Announcements])
IBM Cloud Tape Connector for z/OS is a software-defined storage solution that provides an alternative to virtual tape libraries like the TS7760. Here are some highlights:
Robust virtual tape emulation solution with e-vaulting to cloud-based offsite storage for cold, archival, or backup data. Virtual tape emulation simulates IBM compatible tape controllers, tape drives, and tape volumes, maintained on any IBM z/OS-compatible disk system, such as IBM DS8000. IBM Cloud Tape Connector for z/OS provides several vault, transfer, and recovery options to support business continuity and resiliency.
Sequential z/OS data set cloud storage and retrieval. Sequential data sets stored on disk or flash storage can be moved to the cloud by IBM Cloud Tape Connector for z/OS without the requirement of performing a tape-write operation.
Automatic application recall of data from cloud, whether e-vaulted through virtual tape emulation or copied directly to the cloud.
Pervasive encryption support. This feature enables enterprises to ensure that any data copied to the cloud is encrypted before it is transmitted, automatically protecting and handling the encryption keys.
Support for IBM Cloud Object Storage using S3 protocol, as well as Amazon S3, Hitachi HCP protocol, and EMC Elastic Cloud Service Protocol.
I was in Hollywood Florida for the IBM Systems Technical University. Here is my recap of the final two days, day 4 and 5.
The Pendulum Swings Back: Understanding Converged and Hyperconverged Systems
Once again, I presented my popular session on converged and hyperconverged systems. For converged, IBM offers IBM PureApplication systems with Power and x86 servers, as well as partnership with Cisco called VersaStack. Both support IBM Cloud Private as a platform for running applications.
For Hyperconverged, IBM offers Spectrum Accelerate and Spectrum Scale, as well as partnerships with SuperMicro that combines Spectrum Accelerate on SuperMicro x86 servers, and partnership with Nutanix for CS-models of Power servers pre-installed with Nutanix software.
Unlike other converged and hyperconverged solutions that act as isolated islands of compute and storage, IBM's solutions can be incorporated into an existing datacenter with IBM Cloud Private for orchestration, and IBM Spectrum Scale to provide common access to data.
The Seven Tiers of Business Continuity and Disaster Recovery
With all the natural disasters that happened last year in the USA, and the more recent ones all over the world, this session continues to draw a crowd.
The seven tiers range from the least expensive to most expensive. The least expensive involves restoring data from tapes stored in an offsite vault. Tape continues to be the least expensive storage medium, and can be used to bring up a company in a few days.
For faster recovery, there are options like electronic vaulting to virtual tape libraries, and now the use of Cloud storage for ubiquitous access to data from different locations.
Snapshots of entire volumes, virtual machines or databases are also quite popular. IBM offers IBM Spectrum Protect Snapshot, Spectrum Protect Plus, and Spectrum Copy Data Management for this.
Faster recovery is possible with remote mirroring. This involves sending all of the updates to a secondary location. In the event of a disaster, clients can switch processing with the data already there. IBM has over 800 clients able to do just that in less than 30 minutes.
Event Night by the Pool
Photography by Mo Reyes
While Hurricane Michael raged in upper Florida the week prior, the event coordinators were a bit nervous to offer an evening dinner event by the pool, but the weather cooperated!
Photography by Mo Reyes
I was a social butterfly, moving from table to table to talk to all of the various attendees. A light breeze and excellent food and music made for an enjoyable night!
The pool reception went on to about 10:00pm at night. IBM had lit up its logo into the pools for a great view from above. Perhaps just 30 minutes after arriving back to my hotel room, we had quite the thunderstorm! How incredibly lucky this did not happen during the event!
The following day, I presented my session on "Managing Risk with Data Footprint Reduction, a repeat of the session I did earlier that week.
This was a pleasant way to end the week! Aside from the heat and humidity being above average for October, it was a beautiful hotel in a lovely city.