IBM Spectrum Virtualize for Public Cloud is a brand new offering that provides the IBM Spectrum Virtualize functionality in IBM Cloud. This new capability provides a monthly license to deploy and use Spectrum Virtualize in IBM Cloud to enable hybrid cloud solutions, offering the ability to transfer data between on-premises private clouds or data centers and the public cloud.
The book explains architecture and design considerations, step by step implementation details and typical uses cases of this solution. The following figure depicts the solution architecture used in this book.
In this environment, the on-prem data center is connected to the IBM Cloud using an IPSec VPN that terminates to a network gateway appliance in the cloud. In addition to this, we used native IP replication between a Storwize system and Spectrum Virtualize. This storage based replication allows for data consistency between sites. For guidance on configuring IPSec on the gateway appliance provisioned on the IBM Cloud you can refer to Appendix B, “Basic configuration of the IBM Cloud Network Gateway Appliance”.
The following is the table of contents:
Chapter 1. Introduction
Chapter 2. Architecture of the solution
Chapter 3. Planning and preparation for the IBM Spectrum Virtualize for Public Cloud deployment
Chapter 4. Implementation
Chapter 5. Typical use cases for IBM Spectrum Virtualize for Public Cloud
Chapter 6. Supporting the solution
Appendix A. Approach and guidelines for Disaster Recovery solution in the public cloud
Appendix B. Basic configuration of the IBM Cloud Network Gateway Appliance
I hope you enjoy the book! If you have comments or questions, please use the Comments section of this blog below.
Vasfi Guceris an IBM Redbooks Project Leader. He works with technical experts to create books, guides, blogs, and videos.
VersaStack is an integrated infrastructure solution from Cisco and IBM that combines computing, networking, and storage into a single integrated system. It combines the Cisco Unified Computing System (Cisco UCS) Integrated Infrastructure with IBM Spectrum Virtualize, which includes IBM FlashSystem V9000 and IBM Storwize® storage offerings, for quick deployment and rapid time to value for the implementation of modern infrastructures.
The book explains architecture and design considerations, as well as step by step implementation details of this solution. The following figure depicts the detailed hardware configuration and cabling for the VersaStack solution
used in this book.
This Redbooks has the following chapters:
Chapter 1. Introduction
Chapter 2. Architecture of the solution
Chapter 3. Design considerations for Microsoft Hyper-V and Microsoft SQL Server
Accelerate with IBM Storage: Spectrum Virtualize Encryption
Date: July 20th, 2017
Time: 12 p.m. New York, 5 p.m. London, 6 p.m. Paris, 17:00:00 GMT
Duration: 90 minutes
Abstract: We will go over both internal and external encryption and what each type offers. We will also look a key management options (USB and SKLM) and have a demonstration of encryption using both types of key management from a system in the Washington Systems Center. Plan to join us for this exciting and informative update!
Automation is key to faster deployments with fewer resources. Now Canonical’s JuJu charms can help automate deployments of IBM® Spectrum ScaleTMwith Ubuntu OpenStack cloud. Just in time for the 2017 Boston OpenStack Summit that starts today (08May2017) we have a Redguide to showcase how to deploy an OpenStack environment in an automated fashion with Spectrum Scale as a storage backend by using Juju Charms. We also have a video to give you more insight to how it works and the value it offers.
IBM Spectrum Scale is a proven, enterprise class file system. OpenStack cloud is the leading open-source Infrastructure-as-a-Service cloud platform and Ubuntu is the most used operating system for OpenStack clouds. When used together, these technologies provide a unique cloud platform that accelerates time to value by alleviating many of the storage and management headaches IT administrators, architects, and managers encounter. By integrating the OpenStack storage services within a single storage plane, administrators avoid silos of storage and their corresponding data management headaches, which allows enterprises to realize the full value of their data.
This figure shows IBM Spectrum Scale supports clustered environments and provides file protocol (POSIX, NFS, and SMB) and object protocol (Swift and S3) with unified access capabilities. Spectrum Scale also supports automated tiering of data to cloud object storage providers like IBM Cloud Object Storage (previously known as CleversafeTM) or Amazon S3 using Transparent Cloud Tiering (TCT). By using Canonical's Juju Charms, deployment and management operations can be highly automated, and maintenance operations greatly simplified.
If you happen to be at the Summit stop by the IBM booth in the Marketplace. Also, be sure to stop by Canonical's booth.
Video: Automating IBM Spectrum Scale Deployment with Ubuntu OpenStack Cloud and Juju Charms
The book is a practical guide for IBM Cloud Object Storage Public Cloud Services, the newest addition to the IBM Cloud Object Storage (IBM COS, previously known as Cleversafe) family. It was produced by a team of specialists from IBM around the world working at the IBM Almaden Center and also IBM Business Partner Mark III Systems Team working at their office in Austin, Texas.
With the announcement of the public offering, you now have 4 deployment choices for IBM COS:
Private on-premises object storage
Dedicated object storage (single-tenant)
Public object storage (multi-tenant)
Hybrid object storage (a mix of on-premises, dedicated or public offerings)
The book explains these different deployment models with some details and then focuses on IBM Cloud Object Storage Public Cloud Services and hybrid solutions leveraging this offering.
If you are not familiar with the object storage concepts, start with “Ch1 Introduction to object storage. You will learn about what qualifies business problems for object storage and comparison and positioning to other storage models.
“Ch2 Introducing IBM Cloud Object Store Public Services” compares different COS deployment models and goes into details of the public service capabilities and resiliency options. We don’t discuss the IBM COS internals in this book, since the IBM COS architecture, and actual implementation is transparent to the users of the IBM COS Public Services. You can refer to the newly published IBM Redpaper ”IBM Cloud Object Storage Concepts and Architecture", REDP-5435 for a detailed discussion of the IBM COS architecture, technology and associated benefits.
Ready to start using the service? Follow the instructions in “Ch3 Getting started with IBM Cloud Object Storage” on how to sign up for the service and other details of using it.
“Ch4 Typical applications for IBM Cloud Object Storage” describes several example IBM COS deployments across various industry verticals, provides use cases for IBM COS, and outlines IBM COS integration with other IBM offerings. The aim of this chapter is to provide to system and application owners, architects, and administrators an outline of how IBM COS can fit into or enhance their business.
The rest of the chapters in the book are hands on scenarios and real life solutions with IBM COS, Watson, Bluemix, IoT, Cloudant, Node.js, etc. The chapters are targeted for cloud solution developers and integration architects. We describe step by step implementation and also provide the source code for most of these scenarios.
I hope you enjoy the book! If you have comments, please use the Comments section of this blog post below.
Vasfi Gucer is an IBM Redbooks Project Leader. He works with technical experts to create books, guides, blogs, and videos. Follow Vasfi on Twitter at @VasfiGucer.
For the third version Redbooks update of IBM Spectrum Archive V1.2.2 we added a performance section. Many years ago I had responsibility for a performance department. The team trained me quickly in case I needed to talk to customers. They said all I needed to know was how to say "It depends". We all had a good laugh about my training but there was a lot of truth to the statement. The truth behind the statement was that most environments are unique and there is no one answer to solve every performance issue for seemingly similar environments. You have to dig into the details. I made a visit to a customer during my backup development days to discuss product performance issues. The standard support process including help from the performance team did not resolve the issue so I was asked to visit the customer.
We started the first meeting with the client technical team and management with the support and performance team on the conference phone. The first question was what changed. After getting the answer, "nothing" we talked about the data flow. After about fifteen minutes the "Ah ha!" moment happened when someone from networking remembered they had been doing some switch testing with another location 1,000 miles away and maybe, just maybe, the data from one data center room to another room next to it was taking a 2,000 mile detour. "It depends" became my mantra for many years.
Performance data is critical to help planning, validation of the implementation, and the ongoing success of the solution. As you can see in the figure below the migration performance data between TS1150 and LtO-7 tape drives would be very helpful for planning a Spectrum Archive solution.
The IBM Private, Public, and Hybrid Cloud Storage Solutions Redpaper™ publication takes you on a journey that surveys cloud computing to answer several fundamental questions about storage cloud technology. What are storage clouds? How can a storage cloud help solve your current and future data storage business requirements? What can IBM do to help you implement a storage cloud solution that addresses these needs?
Over the past four versions of this Redpaper there have been many changes regarding types of clouds (private, hybrid, public), how storage products integrate with clouds, and how software-defined storage helps enable storage clouds. The basic architecture of our Redpaper has stayed the same in that we start at the high level of Cloud of Storage Cloud concepts and work our way through the product details, and use cases. In addition to adding new products and use cases we expanded the hybrid cloud discussion, and the discussion about what enables a storage cloud.
Chapter 1. What is cloud computing
Chapter 2. What is a storage cloud
Chapter 3. What enables a storage cloud
Chapter 4. IBM Storage Systems for SDS
Chapter 5. What are others doing in the journey to storage cloud
Chapter 6. Your next steps
Here is a new figure with the seven Hybrid Cloud use cases which to me shows how much progress has been made in the last four years when we published version one of the this Redpaper.
We also updated theIBM Software-Defined Storage GuideRedpaper. There are many similarities and it shows how software-defined storage adds value for traditional workloads and cloud workloads.
This is the fourth edition of our A Deployment Guide for IBM Spectrum Scale Unified File and Object Storage, REDP-5113. It just seems like yesterday when we published the first edition. The title was also updated to show the breadth of IBM Spectrum Scale support of Unified File and Object storage. The latest Redpaper focuses on the scope of object storage with OpenStack Swift and the automation of the installation processes and preferred practices that ensure optimal performance and reliability. We have come a long way from the original paper that included lots of scripts to help with the installation process. Now is a good time to take a look at the Redpaper to see the progress in IBM Spectrum Scale V4.2.2.
A key highlight: This version describes the use of the spectrumscale installation toolkit to automate the steps that are required to install IBM Spectrum Scale, deploy protocols, and install updates and patches.
The Redpaper has the following layout:
Chapter 1. IBM Spectrum Scale Unified File and Object Storage
Chapter 2. Planning for an IBM Spectrum Scale Unified File and Object Storage deployment
Chapter 3. IBM Spectrum Scale Unified File and Object Storage configuration overview
Chapter 4. IBM Spectrum Scale Unified File and Object Storage installation
Chapter 5. System administration considerations
Chapter 6. Swift feature overview
Chapter 7. Data protection: Backup, restore, and disaster recovery
Chapter 8. Summary
Section 2.1.8 Unified file and object access describes some of the use cases for IBM Spectrum Scale Object Storage when unified file and object access is enabled. This figure shows a comparison between traditional analytics with object store versus analytics with Spectrum Scale Unified File and Object access.
This is just one example of new information in the fourth edition.
The IBM SAN42B-R Extension Switch and IBM b-type Gen 6 Extension Blade in Distance Replication Configurations (Disk and Tape), REDP-5404 IBM Redpaper publication is now available.
The IBM SAN42B-R and the IBM b-type Gen 6 Extension Blade for the IBM SAN512B-6 and SAN256B-6 directors are the latest products from IBM that offer IP Extension (IPEX) technology. They provide the following benefits:
A platform for building a high-performance data center extension infrastructure for replication and backup solutions. They can use any type of inter-data center wide area network (WAN) transport to extend open systems and mainframe storage applications over any distance.
Scalable deployment options to extend disaster recovery and data protection storage solutions across any distance. They include a suite of features that help ensure continuous availability of a storage extension network, from pre-deployment validation tools to advanced network failure recovery technologies.
Enterprise class extension benefits that were previously only available for FCIP extension now also apply to IP storage replication.
The SAN42B-R and the IBM b-type Gen 6 Extension Blade incorporate several advanced technologies that are essential to ensuring maximum throughput and bandwidth utilization over distance, including the following features:
An aggressive Transmission Control Protocol (TCP) stack that is optimized for storage, called WAN-Optimized TCP (WO-TCP), a capability that is not available with solutions that use standard TCP stacks alone
An advanced line-rate data compression architecture with three compression
An encapsulation methodology that enables efficient data transport across the WAN
The primary use case for the IPEX technology is with the IBM TS7700 family of products, specifically for the TS7760 or TS7700 Grid solution.
FCIP extension provides a remote data replication solution for Fibre Channel connection (FICON) replication over long distance solutions such as IBM z/OS Global Mirror and IBM tape read/write pipelining. The SAN42B-R and IBM Gen 6 Extension blade are also ideal for FCIP extension of asynchronous DASD remote data replication solutions such as IBM Global Mirror. In addition, the high availability features make them ideal for FCIP extension of synchronous DASD remote data replication solutions such as IBM Metro Mirror.
Implementation scenarios and troubleshooting
Read REDP-5404 for more information about the following topics:
Accelerate with IBM Storage: Elastic Storage Servers (ESS) and Spectrum Scale Software Solution
Date: March 9, 2017
Time: 12 p.m. New York, 5 p.m. London, 6 p.m. Paris, 17:00:00 GMT
Duration: 75 minutes
Abstract: Unstructured data is growing dramatically. Cognitive computing and Big Data analytics are emerging across every industry. These marketplace trends are increasing the need for IBM storage solutions such as Elastic Storage Server (ESS) and Spectrum Scale software. This session will review relevant features and benefits of Spectrum Scale software and the Elastic Storage Server. It will highlight high priority use cases (e.g. back-up / restore, SAP Hana, Big Data for Medical Imaging). It will share representative usage examples.
Matt Drahzal, Storage Product Strategy/Market Management - IBM Technical Computing
JD Zeeman, Worldwide Business Development Executive, Software Defined Storage Solutions
John Sing, Offering Evangelist, IBM Elastic Storage Server, Spectrum Scale, DF150
Cloud Data Sharing with IBM Spectrum Scale draft just published. This IBM® Redpaper™ publication provides information to help you with the sizing, configuration, and monitoring of hybrid cloud solutions using the Cloud data sharing functionality of IBM Spectrum Scale™.
Cloud data sharing is a feature available with IBM Spectrum Scale Version 4.2.2 that allows for transferring data to and from object storage, using the full lifecycle management capabilities of IBM Spectrum Scale information lifecycle management (ILM) policy engine to control the movement of data. This feature is designed to work with many types of object storage allowing for the archiving, distribution, or sharing of data between IBM Spectrum Scale and object storage.
The Redpaper is organized into the following sections:
IBM Spectrum Scale Cloud data sharing use cases
Sizing and scaling considerations
Resource requirements and considerations
Configuration and best practices
Monitoring and lifecycle management
IBM Spectrum Control
The section for use cases describes data distribution/sharing between sites, sharing data between applications, and off-line data archival. Here is a figure from the Redpaper showing the use case example of sharing data between sites.
This Redpaper provides guidance to help plan for Cloud data sharing to allow for the sharing and use of data between various cloud object storage types and IBM Spectrum Scale.
The IBM Redbooks publicationIBM TS7700 Release 4.0 Guide is now available. We've updated the book with the new TS7760 which includes the TS7760D (disk only) and TS7760T (tape attached). Because R4.0 is a new version we decided to create a new book number SG24-8366. R3.3 remains available as SG24-8122. The TS7760 is the newest member of the TS7700 virtual tape solution. IBM TS7760 is a mainframe virtual tape solution that optimizes data protection and business continuance for z Systems data. Here are the highlights:
Support next-generation cloud infrastructures on IBM® z Systems™ servers, backed by virtualized tape storage
Gain private cloud-like functionality that makes all IBM TS7700 data accessible to any host in the grid
Reduce operational expenses by keeping business-critical data secure, continuously available and easy to manage
Simplify z Systems-connected tape management and improve batch window performance
Leverage disk and physical tape technologies in a single, cost-effective solution
In addition updating all the product details in the Redbooks pub we expanded our focus on Disaster Recovery. We split the prior DR chapter and now have Chapter 5 Disaster Recover and Chapter 13 Disaster Recovery Testing. The main reason for the split was to put the information related to DR planning closer to the front of the book with the planning chapters and then put more focus on DR testing of the implemented solution.
Chapter 5 Disaster Recovery describes the use of the TS7700 in disaster recovery (DR) including:.
TS7700 disaster recovery principles
Planning for disaster recovery
High availability and disaster recovery configurations
Disaster recovery testing basics
A real disaster
Geographically Dispersed Parallel Sysplex for z/OS
Chapter 13 Disaster Recovery Testing describes disaster recovery (DR) testing in a TS7700 grid configuration.
DR Testing Overview
DR Testing Methods
DR Testing General Considerations
DR for FlashCopy Concepts and Command Examples
DR Testing Methods Examples
Expected Failures During a DR Test
Based on feedback from the support team we all agreed that this may be helpful to be better prepared if a disaster ever happens.
Everyday we hear about data security breaches regarding social media, companies, governments, organizations, private devices, and on and on. Having worked in the storage management business for most of my career my first thought is how did they leave themselves so exposed. After reading the details there are many that could have been avoided and some where the answer is not so clear. We also don't hear about the ones that successfully prevented unauthorized access to their data. I want my data to be in the last category - successful protection. Understanding your options is a great way to evaluate your data protection environment or help you plan for implementing a secure system for your data. We just published the IBM Redpaper IBM Spectrum Scale Security to describe the security possibilities for Spectrum Scale implementations. Here is an overview of the organization and scope of the Redpaper.
Security for storage systems can be classified as follows:
Data storage (data at rest, which includes data durability and immutability)
Access to data
Movement of data (data in flight)
Management of data
IBM® Spectrum Scale is a software-defined storage system for high performance, large-scale workloads on-premises or in the cloud. IBM Spectrum™ Scale addresses all four aspects of security by securing data at rest (protecting data at rest with snapshots, and backups and immutability features) and securing data in flight (providing secure management of data, and secure access to data by using authentication and authorization across multiple supported access protocols). These protocols include POSIX, NFS, SMB, Hadoop, and Object (REST). For automated data management, it is equipped with powerful information lifecycle management (ILM) tools that can help administer unstructured data by providing the correct security for the correct data. The various aspects of security in IBM Spectrum Scale™, include the following items:
Security of data in transit
Security of data at rest
Security for transparent cloud tiering (TCT)
Security for OpenStack drivers
For example, Chapter 9 describes Security for transparent cloud tiering (TCT). It starts with a high level figure shown here.
Figure 1. Transparent cloud tiering configuration
The Chapter drills down to the following detailed items:
Securing data in flight and at rest
Securing the keys that are used to protect the data
Configuring transparent cloud tiering with an external key manager: IBM Security Key Lifecycle Manager
TCT daemon communications
Security of TCT commands
Data integrity protection
Security considerations while configuring a cloud object storage
Data Security is a never ending process of planning, implementation, and validation.
This week we published IBM Spectrum Scale Deployment on IBM SoftLayer. This IBM® Redpaper™ publication describes various activities that are necessary to deploy IBM Spectrum Scale™ on IBM SoftLayer® . It includes the following topics:
Sizing and configuration guidance
Storage options on SoftLayer
Network considerations on SoftLayer
There are many factors that must be taken into account when configuring a IBM Spectrum Scale cluster on any hardware. Desired performance, cost, cluster size, and workload can influence any hardware and configuration decisions. The Redpaper provide guidance to help you navigate the many options available when choosing hardware and deploying a cluster.
Many of my non technical friends ask me questions about their smartphones data that their apps say is in the cloud. And then they point to the sky as if the data is really up there. While most of the people on the planet don't need to know how it really works many of us really do need to know to make the best decisions so our friends and clients don't need to know.
The SoftLayer platform webpage has a great summary that clearly states how hardware infrastructure is needed to build a cloud. The Redpaper focuses on how to optimize a Spectrum Scale implementation on SoftLayer.
"Cloud is not a commodity. And no matter what you call it, computing doesn’t come out of the sky. It comes from physical hardware inside brick and mortar facilities connected by hundreds of miles of networking cable. And no two clouds are built the same way. SoftLayer gives you the highest performing cloud infrastructure available. One platform that takes data centers around the world that are full of the widest range of cloud computing options, and then integrates and automates everything."
Here is a figure to show SoftLayer's global reach of its hardware infrastructure.
The Spectrum Scale feature called transparent cloud tiering allows cloud object storage providers such as IBM Cloud Object Storage (previously known as Cleversafe), IBM SoftLayer®, and Amazon S3 to be used as a storage tier for IBM Spectrum Scale. Transparent cloud tiering can help cut storage capital and operating costs by moving data that does not require local performance to an on or off premise cloud object storage provider.
Transparent cloud tiering reduces the complexity of cloud object storage by making data transfers transparent to the user or application. This capability can help you adapt to a hybrid cloud deployment model where active data remains directly accessible to your applications and inactive data is placed in the correct cloud (private or public) automatically through Spectrum Scale policies.
The IBM Redpaper Enabling Hybrid Cloud Storage for IBM Spectrum Scale Using Transparent Cloud Tiering draft was published in October and the final publication this week was updated with the latest Spectrum Scale 4.2.2 features related to transparent cloud tiering. One of the management GUI changes added a new Files > Transparent Cloud Tiering page. The Transparent Cloud Tiering page provides both summarized and attribute-specific details of the transparent cloud tiering service, which is integrated with the IBM Spectrum Scale system.
Also, Spectrum Scale introduced component-specific email notifications. The system administrator can now configure the email notifications in the Settings > Event Notifications page to send email notifications to the recipients for reported events. Transparent cloud tiering is one of the functional areas supported with the new feature.
New sections added to the final Redpaper are Restoring files from the cloud tier and Bulk file restore. The following figure (Figure 7 Spectrum Scale cluster with Cloud Service nodes in the redpaper) is updated to show Spectrum Scale transparent cloud tiering components for a typical multi-node configuration.
For more detailed information regarding Spectrum Scale 4.2.2 see the IBM Knowledge Center at: