Modified on by Shashank Shingornikar
After successfully implementing Real-time Compression feature in Storwize V7000, IBM has taken a step further bringing this patented technology in IBM XIV storage system. In a recent announcement of XIV 11.6.0 release, the Real-time Compression feature is seamlessly integrated in XIV storage system. Eliminating the need to add any extra hardware, the IBM Random Access Compression Engine (RACE) technology is now integrated with XIV storage system software stack to compress data before writing it to disk (above cache mechanism) resulting in up to 80% storage capacity savings.
It is designed with transparency in mind so that it can be implemented without changes to applications, hosts, networks, fabrics, or external storage systems. The solution is not visible to hosts, thus users and applications continue to work as is. To estimate the compression savings on an existing XIV non compressed volumes, Comprestimator utility is now integrated with XIV software.
What does Compression has in store for me ?
On the XIV system the compression ratio for all uncompressed volumes in the system is continuously estimated, even before enabling compression. The figure shows the various stages of volumes on the system ranging from uncompressed to potential savings and finally the total amount of compression on a volume
What are the Compression benefits for XIV ?
With the inline implementation of Real-time Compression the IBM XIV now delivers dramatic cost savings without need for extra hardware and provides following benefits :
- Increases usable capacity per rack typically to one Petabyte or more with Real-time Compression, greatly reducing effective cost per capacity
- Replicates compressed data faster and using less bandwidth, freeing up bandwidth for other uses
- Continuously displays predicted or actual compression ratios for all volumes
- Converts non-compressed volumes to compressed non-disruptively
So How does it work ?
Real-time compression implementation in XIV storage uses above cache architecture where data is compressed or de-compressed between the I/O interface and the cache. The compression node runs on every module of XIV taking advantage of parallel architecture of XIV. It compresses the portion of volume which only belongs to the module and thus distributing compression workload across all the modules of XIV. Hence, Real-time Compression implementation in XIV have minimal impact on the performance delivered by XIV.
Whenever write operations happen, data is compressed before they enter cache and acknowledgment is sent back to the host. During read operations, reads are stored compressed in cache and data is de-compressed when they are read from cache using RACE before passing it to the host. During XIV mirroring operation, data is compressed only once and compressed data is sent across the network reducing network bandwidth.
What will benefit more from Compression ?
- Database environments – DB2, Oracle, MS-SQL, and so on
- Database Applications – SAP, Oracle applications, and so on
- Server/Desktop Virtualization – KVM, VMware, Hyper-V, and so on
- Other compressible workloads – seismic, engineering, and so on
- Email – Microsoft Exchange, and so on
Are there any guidelines for Compression?
- IBM Real-time Compression is appropriate for data that has the following characteristics:
- Any data for which the Comprestimator tool estimates 25% or higher savings
- Volumes that contain data that is not already compressed (for example, un-compressed image and video files)
- Data for which application based encryption is not used or data that is not sent encrypted to the XIV.
Anything I can refer to ?
Real-time Compression not only works best with randomly accessed data such as database like IBM DB2, Oracle, MS-SQL Server but it also provides good results with server virtualization solutions like VMware, KVM, Hyper-V. When using Oracle databases, compressed volumes take advantage of above cache architecture compressing the writes seamlessly. A 57% compression has been observed during creation of a terabyte of data with minimal performance penalty. ( Publication : WP102551 )
VMware vSphere virtual machines can be seamlessly deployed on the compressed volumes, often with the compression savings of 50% to75%, allowing customers to reduce the storage capacity required for vitalized environments. ( Publication : WP102552 )
Microsoft Hyper-V virtualization helps customers maximize System x server and other resource use. Included in Windows Server, Microsoft Hyper-V virtualization helps reduce costs by allowing a greater number of application workloads to be hosted on fewer physical servers. When using Microsoft SQL Server 2012 SP1 OLTP data files and VM Windows Server 2012 R2 system files stored in Hyper-V virtual disk and XIV compressed volume achieved 73% compression savings ( Publication : WP102553 )
What about the performance ?
While the team tested the compression benefits and compiled the paper, another team from IBM Tel Aviv lab, had been busy with performance testing of the Oracle database hosted on the IBM XIV compressed volumes.
In the test setup, the team used, both compressed and uncompressed volumes configured on XIV for better parallelism. These volumes were mapped to the ESX system hosting the database server to create multiple VMFS file systems. A 5 TB database was created on the VMFS volumes using the Benchmark Factory tool. During the test run of 12 hour, load starting with 1,000 to a maximum of 30,000 users was made to put the system under a realistic production load. The I/O per second (IOPS) and response time information shown by the Benchmark Factory tool is shown by Figure below. Each point on the graph indicates an addition of 2500 users. The graph clearly indicates that the application has minimum impact in terms of response time when using the compressed volumes.
Blog Authors: Mandar Vaidya, Shashank Shingornikar
Modified on by UdayasuryanKodoly
Real-time collaboration and information sharing are key drivers of an enterprise’s productivity and innovation. Finding solutions to enable such dynamic sharing in an enterprise setting while maintaining control, however, can be a challenge. Some organizations look to consumer-grade, cloud-based file sharing options that offer the scalability, ease of use and access users want but store sensitive company data on external servers. This exposes organizations to risks of data leaks while limiting IT visibility. Other options include using existing enterprise collaboration and content management systems that might be challenging to maintain and cumbersome for users.
What exactly is the solution?
The combined IBM® Spectrum Scale for object storage and ownCloud software technologies helps enterprises to build highly scalable, secure, and flexible on-premise file sync and share solution. The ownCloud provides universal file access through a common file access layer to the IBM Spectrum Scale for object storage. The data files are kept in on-premise Spectrum Scale for object storage. ownCloud allows enterprises IT organizations to regain control of sensitive data with managed file sync and share which gives users universal file access to all of their data:
- Manage and protect data on-premise – Using IBM Spectrum Scale for object storage, with the complete software stack running on servers inside the data center, controlled by trusted administrators, managed to established policies.
- Integrate with existing IT system resources and policies – such as authentication systems, user directories, governance workflows, intrusion detection, monitoring, logging and storage management.
- Provide access through a comprehensive set of application programming interfaces ( API) and mobile libraries to customize system capabilities, meet unique service requirements, and accommodate changing user needs.
Why enterprises wants on-premise files sync and share solution?
Storing data off-premise may strip an organization’s ability to manage and control its data, or to ensure that data can be deleted. Few enterprises, however, are willing to forgo the benefits that cloud services provide in the advancement of agility and improved business processes. That leaves them struggling with how to use these technologies without importing security risks. They also recognize that users are increasingly able to migrate to external services that provide them greater flexibility and mobility than that offered by the enterprise.
By retaining on-premises manageability of file sync and share services, though, IT can use a private cloud solution to reconcile the need for cloud technology with the requirements for security, privacy, and regain control of sensitive data without unwanted exposure. With the ability to enhance control and govern access to files, IT administrators can set sophisticated rules for user and device connections and prevent access based upon those rules. Further, the capabilities and extensibility of on-premise file sync and share match the ease of use and complete access that first drove consumption of cloud services, yet IT controls sensitive assets in its own cloud environment.
Solution Lab testing
This solution consists of multiple servers installed with ownCloud server software. The ownCloud is a PHP web application running on top of Apache on Linux. This PHP application manages every aspect of ownCloud, from user-management to plug-ins, file sharing and storage. Attached to the PHP application is a database where ownCloud stores user information, user-shared file details, plug-in application states, and the ownCloud file cache (a performance accelerator). ownCloud accesses the database through an abstraction layer, enabling support for Oracle, MySQL, SQL Server, and PostgreSQL. Complete webserver logging is provided through webserver logs, and user and system logs are provided in a separate ownCloud log, or can be directed to a syslog file.
In the lab testing environment, an Active Directory (AD) is integrated with the ownCloud for user account provisioning. IBM Spectrum Scale for object storage is configured with local authentication. It is possible to configure IBM Spectrum Scale for object storage with enterprise directory server such as AD or Lightweight Directory Access Protocol (LDAP).
OpenStack Swift is installed on the protocol node(s) of the IBM Spectrum Scale for object storage.
IBM Spectrum Scale is a proven, enterprise-class file system, and OpenStack Swift is a best-of-breed object-based storage system. IBM Spectrum Scale for object storage combines these technologies to provide a new type of cloud storage that includes efficient data protection and recovery, proven scalability, and performance; snapshot and backup and recovery support; and information lifecycle management. Through these features, IBM Spectrum Scale for object storage can help simplify data management and allow enterprises to realize the full value of their data.
ownCloud is a self-hosted file sync and share server. It provides access to on-premises data through a web interface, sync clients while providing a platform to view, sync and share across devices easily, while gives the enterprises the ability to manage and control their data. ownCloud’s open architecture is extensible through a simple but powerful APIs for applications and plug-ins and works with seamlessly with IBM Spectrum Scale for object storage.
The combined IBM Spectrum Scale for object storage and ownCloud server technologies helps enterprises to build highly scalable, secure, and flexible on-premise file sync and share solution.
To learn more about the solution, please see the solution technical paper: https://www-304.ibm.com/partnerworld/wps/servlet/ContentHandler/stg_ast_sto_wp_on-premise-file-syn-share-owncloud
Modified on by Shashank Shingornikar
How far have you gone before in tuning your database ? be it Oracle or DB2. The efforts are never good enough and before you breathe easy the battle begins .. again and again ...
Now you can relax a bit ... With IBM Easy Tier Server functionality available with Easy Tier, you'll be able to get more work done in terms of improvement in transactions per second (TPS).
So what exactly is Easy Tier Server ?
IBM Easy Tier Server is a unified storage caching and tiering solution across AIX servers and supported direct-attached storage (DAS) flash drives. Easy Tier Server allows the most frequently accessed or “hottest” data to be placed (cached) closer to the hosts, thus overcoming the SAN latency. The Easy Tier Server core relies on DS8870 cooperating with heterogeneous hosts to make a global decision on which data to copy to the hosts’ local SSDs, for improved application response time. Therefore, DAS SSD devices play an important role in an Easy Tier Server implementation. Specializing in high I/O performance, SSD cache has the upper hand in cost per input/output operations per second (IOPS).
The Easy Tier technology has evolved over years and is now in its fifth generation. Easy Tier Server is one of several Easy Tier enhancements, introduced with the DS8000 Licensed Machine Code 7.7.10.xx.xx. Both Easy Tier and Easy Tier Server licenses, although required, are available at no cost.
Which workloads are best fit for Easy Tier Server ?
Because Easy Tier Server implements a read-only local DAS cache on the hosts, there are some particular scenarios that can take the best advantage of this feature. These are
- Real-time analytics workload
- Large content data
- Online transaction processing (OLTP) workload
- Virtual machine (VM) consolidation
- Big Data
Under the hood
The Easy Tier Server feature consists of two major components
- The Easy Tier Server coherency server
The Easy Tier Server coherency server runs in the DS8870 and manages how data is placed onto the internal flash caches on the attached hosts. Also, it integrates with Easy Tier data placement functions for the best optimization on DS8870 internal tiers (SSD, Enterprise, and Nearline). The coherency server asynchronously communicates with the hosts system (the coherency clients) and generates caching advice for each coherency client, which is based on Easy Tier placement and statistics.
- The Easy Tier Server coherency client
The Easy Tier Server coherency client runs on the host system and keeps local caches on DAS solid-state drives. The coherency client uses the Easy Tier Server protocol to establish system-aware caching that interfaces with the coherency server. An Easy Tier Server coherency client driver cooperates with the operating system to direct I/Os either to local DAS cache or to DS8870, in a transparent way to the applications.
The POWER system has a DAS attached which is used by Easy Tier Server Coherency Client Driver to create local cache. Easy Tier Server coherency clients are designed to route I/O read hits to the application host DAS, while sending read misses directly to DS8870. In the same way, the write I/Os are routed to DS8870 and cache pages related to the I/O address spaces are invalidated on the client’s local cache to keep cache coherency and data integrity. The coherency client and coherency server share statistics to ensure that the best caching decisions are made.
And the bottom line is ?
In the lab a brokerage OLTP workload was executed simulating maximum amount of read requests. In the beginning of the run the hdisks configured for the ASM DATA disk group showed maximum utilization as no caching was enabled. 60 minutes in the run, the caching was enabled on the database host running the workload. Soon after the caching was enabled, the Easy Tier Server starts migration of hot extents from DS8870 to the database host running the Easy Tier Server coherent client. Over a period, as more and more hot extents are migrated from DS8870, maximum activity was observed on cache devices and lesser activity from DS8870 storage. As more and more extents (containing required data) were cached, the read operations requests were satisfied locally thus eliminating the need to read data from storage. The effective utilization of locally cached data showed 100% improvement in the TPS observed during the test run.
Whether it is a latency sensitive environment, high read/write ratio applications, or a highly parallel processing system, there is an increasing need to process data quickly and Easy Tier Server can be considered for these situations.In cases where the read performance of the storage can lead to a major bottleneck to the environment, there is a high value in faster storage, and therefore, a good fit for the Easy Tier Server.
Publications and Resources
A white paper WP102534 is available on IBM Techdocs website that provides detailed information on the testing effort.
IBM System Storage DS8870: Architecture and Implementation, SG24-8085
IBM System Storage DS8000 Host Attachment and Interoperability, SG24-8887
IBM System Storage DS88870 Product Guide
IBM System Storage DS8000 Easy Tier, REDP-4667
IBM System Storage DS8000 Easy Tier Heat Map Transfer, REDP-5015
IBM System Storage DS8000: Easy Tier Application, REDP-5014
Views / thoughts expressed above are my own, not necessarily of my employer.
Modified on by Prashant Avashia
Lately, patients are expecting their physicians to provide them a higher quality healthcare with intelligent, immediate insights from their radiological images, clinician notes and lab results. They are demanding simple diagnostic guidance, customized treatment options & immediate digital access to their personal medical information on their mobile devices, securely.
The primary driver for the transformation of radiological imaging services from volume-based imaging to patient-centric value-based imaging enables a closer dialogue between the radiologists, physicians & specialists. This transformation delivers superior patient experience, higher clinical accountability, relevant diagnostic insights & clinical decisions, minimizes medical errors, minimizes complexity of care delivery models, establishes consistent outcomes for chronic diseases, achieves tighter integration with non-radiology systems (EMR, Ambulatory, HIS, Lab Services, and more) and improved communication across the entire healthcare enterprise. As defined by American College of Radiology, Imaging 3.0 is a multiphase program initiative covering services, technology, tools and processes. Leveraging this initiative, radiologists adapt how they manage their practices, patient care and their own futures as the industry transforms itself from volume to value based imaging services.
The current reality is that many healthcare systems are not designed to facilitate easy information sharing across the enterprise – and particularly true with medical imaging data. With lack of standards, and with no clear integration and interoperability between imaging and non-radiology systems, it diminishes communication between physicians, radiologists and the specialists, with minimal or no access to real-time symptomatic evidence in the collaborative point-of-care process. Potentially, it contributes to process delays, clinical workflow inefficiencies, and diagnostic errors.
New Era Healthcare Environments with Mobile & Cloud capabilities demand digital transformation with better data economics. It is important to provide patients the ability to securely view, download & transmit health information quickly. IBM realizes many of these transformational challenges in a Healthcare Enterprise. IBM has successfully delivered pre-qualified Medical Imaging & Archiving solutions with leading Business Partners (in Healthcare Industry) on enabling and deploying their Cardiology, PACS and Enterprise Imaging applications on IBM Storage Systems including IBM System Storwize® V7000, DS8870®, IBM XIV®, IBM System Storage® San Volume Controller, and/or the IBM FlashSystemTM. IBM Spectrum StorageTM based Solutions deliver the potential of extracting insights from data volumes, and increase business agility by offering its functional capabilities as software, or as a cloud, or as a managed service offering. These all, are complete, ready-to-deploy, proven, high-performance solutions that help accelerate the time-to-value, with reliability, security and speed.
Many happy, satisfied IBM clients, including emergency centers, radiology departments, multi-specialty clinical groups, and hospital networks are currently running their imaging applications, on a variety of IBM Storage systems. They are running their imaging solutions, on premise at a local datacenter, globally at multiple datacenters, or delivering them from the cloud. They are leveraging DS8870 for enterprise critical applications, Storwize family for virtualization capabilities, FlashSystem for application and patient-data acceleration facilitating life-critical response, or XIV for cloud-specific standard based deployments. IBM storage systems also support built-in data encryption capabilities, instantaneous video imaging for angioplasty, laparoscopy, endoscopy or other clinical procedures, and real-time compression of non-imaging data.
As patient-centric delivery models continually evolve with the transition from volume based imaging services to value based imaging services, the IBM solution choices become very important in designing and implementing flexible storage architectures for imaging applications and facilitate reliable, secure and fast access to patient data, anywhere. IBM will continue to partner with leading Business Partners (in Healthcare) to deliver proven, superior storage solutions that will ultimately improve provider collaboration and better patient outcomes, and at significantly lower costs.
To learn more, I recommend checking out the following paper at URL: http://www.ibm.com/common/ssi/cgi-bin/ssialias?subtype=WH&infotype=SA&appname=SNDE_HL_HL_USEN&htmlfid=HLW03016USEN&attachment=HLW03016USEN.PDF
Modified on by MandarVaidya
Official release of VMware vSphere Virtual Volumes (VVol) in Q1 2015 has generated tremendous interest with customers. VVol extends VMware's software defined story to its storage partners and it completely changes the paradigm in which storage is consumed by the hypervisor. With VVol implementation, storage intensive tasks are off-loaded by the server hypervisor to application-aware, policy-driven storage. It also simplifies storage management, puts the virtual machines in charge of their own storage, and gives more fine-grained control over virtual machine storage. With Virtual Volumes, an individual virtual machine, not the datastore, becomes a unit of storage management, while storage hardware gains complete control over virtual disk content, layout, and management.
IBM is VMware’s strategic alliance partner and is a key design partner for VVol. IBM has announced support of VVol with XIV storage in lock-step with VMware’s general availability of vSphere 6.0 product. IBM’s integration of Virtual Volumes in XIV is based on the VMware API for Storage Awareness (VASA 2.0) delivered by IBM Spectrum Control Base Edition. This integration facilitates off-loading of the following storage-intensive virtual machine operations to IBM XIV storage with predictable performance and effective capacity utilization.
Snapshot operations of a virtual machine using Virtual Volumes datastore
Cloning of virtual machine using Virtual Volumes datastore
Storage migration of virtual machine from non- VVol to Virtual Volumes datastore
The figure below shows a pictorial representation of a Virtual Volumes implementation with XIV using IBM Spectrum Control Base Edition.
IBM Spectrum Control Base Edition implements the VMware Virtual Volumes APIs, providing a separate management bridge between vSphere and XIV storage. This management bridge separates the data path from the management path. IBM Spectrum Control Base Edition enables communication between vSphere stack (ESXi hosts, vCenter server and the vSphere Web Client) and IBM XIV storage. IBM Spectrum Control Base Edition maps virtual disk objects related to virtual machines and their derivatives such as snapshots and clones, directly to the XIV storage system.
ESXi hosts access Virtual Volumes through an intermediate point in data path, called the Protocol Endpoint (PE). It is also referred to as the Administrative Logical Unit (ALU) on XIV storage. ALU allows XIV storage to carry out storage-related tasks on behalf of the ESXi host.
Virtual Volumes reside on storage containers on XIV. Storage containers represent groupings of Virtual Volumes attached to a virtual machine. IBM Spectrum Control Base Edition associates a storage container with a single XIV pool. The storage containers are characterized by a storage service which combines storage capacity along with storage attributes such as encryption, thick/thin provisioning type. This storage container acts as a virtual datastore and matches application specific requirements of a virtual machine .
For detail step by step implementation of VVol in IBM XIV using IBM Spectrum Control Base Edition, refer to this technical paper : https://www.ibm.com/partnerworld/page/stg_ast_sto_wp-vmware-vsphere-virtual-volumes-using-xiv
IBM XIV delivers excellent levels of storage abstraction, easy automated provisioning and policy compliant capabilities through its integration with VVol. IBM Spectrum Control Base Edition delivers the VASA capabilities for XIV’s tight integration with VVol and plays a strategic role in IBM’s software defined storage initiative by providing storage agility and efficiencies required for today’s demanding application workloads.
Here are some videos you might also like to view to hear directly from VMware and IBM on our strategic partnership and joint VVol development efforts.
Powerful IBM XIV Storage Integration with VMware Virtual Volumes - Laura Guio
VMware vSphere Virtual Volumes and IBM XIV: A perfect fit
Additionally we have a Virtual Volume demo you should check out:
vSphere Virtual Volumes (VVOL) with IBM XIV Storage System
If you happen to onsite at the IBM Edge2015 event in Las Vegas the week of May 11th, be sure to attend the IBM-VMware session on Monday or Friday on this very topic:
Monday, 5/11 4:30 - 5:30 pm, San Polo 3503
Friday, 5/15 10:30 - 11:30 am, San Polo 3503
IBM Spectrum Control Base Edition: Orchestrate and Automate IBM Storage with VMware
Presenters: Yossi Siles, IBM and Rawlinson Rivera, VMware
Modified on by SandeepZende
IBM Storage at your service courtesy of IBM Spectrum Control and VMware vRealize Automation
In today's emerging or, I would say, stabilizing world of IT cloud, everything needs to be delivered in "as a service" fashion. Because of this there is a growing demand for any IT solution to be available as a service. Organizations are thinking creatively to come up with the new IT solution as a service and then there are organizations who are developing cloud platforms which help other organizations quickly deploy their cloud solution. And this is where the race has begun.
There are end users or the business opportunists who do not want to waste time in designing and implementing a solution and then selling it. They want to start their business in a very short time. They are ready to rely on the organizations who provide this type of platforms to make their solution available in a short time. The platforms would help in building either public cloud or private cloud or both i.e. hybrid cloud depending of their need.
There are various vendors in the market who themselves provide the whole platform and also integrate with other vendors to build a unique platform which will help build the cloud. In today's world, it is very necessary that if a product is being developed, it has to take into account of how it can be integrated into the cloud or how it can enable cloud platform.
There are various enablers of cloud ranging from top layer to the bottommost layer of the cloud solution. IBM Spectrum Control is one of those enablers which provide the efficient infrastructure management for the cloud, virtualized and software defined storage. It simplifies and automates storage provisioning, capacity management, availability monitoring and reporting.
IBM Spectrum Control will also be an important factor in the success of IBM Spectrum Storage offerings by providing a control plane that is capable of provisioning and monitoring of storage in cloud or on-premise with the control of defining it in the software. The software defined storage characteristics of IBM Spectrum Control allows itself of receiving the storage definitions from the top layer with the help of interface made available for cloud providers or enablers. The interfaces are made available in the form of plug-ins developed for a specific cloud vendor. For example, IBM Spectrum Control Base Edition provides plug-ins for VMware vRealize Orchestrator, VMware VASA, VMware vRealize Operations, VMware vRealize Automation, etc.
With the help of IBM Spectrum Control Base Edition and VMware vRealize suite, a cloud architect can design various cloud solutions and can deliver these various solutions in a "as a service" fashion. One such very useful solution for cloud environments is "Storage as a service" using IBM storage. In this solution, an architect can design a service using VMware vRealize Automation, VMware vRealize Orchestrator, IBM Spectrum Control Base Edition and IBM XIV storage system and make storage available as a service wherein an end user, if entitled to, can avail the service by requesting a storage space for its VMs.
VMware vRealize Automation with its Advanced Services can deliver almost anything as a service (XaaS). The Advanced Services of vRealize Automation allows a cloud architect/administrator to advertise vRealize Orchestrator workflows as a service. Whatever workflows that are designed in the vRealize Orchestrator can be exposed from vRealize Automation. The 'IBM Storage plug-in for vRealize Orchestrator' which is a component of IBM Spectrum Control allows vRealize Automation to define or provision the storage as per the administrator or user need.
For more details on how "storage as a service" can be implemented using IBM Storage, IBM Spectrum Control and VMware vRealize, refer the technical paper.
Also refer the recorded demos below :
Demo: IBM Spectrum Control & VMware vRealize Automation - Configuration
The video demonstrates the configuration flow of the integration of IBM Spectrum Control Base Edition, IBM XIV, VMware vRealize Orchestrator and VMware vRealize Automation to enable a 'Storage as a service' solution. This video also demonstrates the creation of volume, mapping a volume and creation of datastore from the vRealize Automation web console.
Demo: IBM Spectrum Control & VMware vRealize Automation - Datastore Creation
This short video demonstrates the creation of datastore upon a user request from VMware vRealize Automation and IBM Spectrum Control Base Edition playing a part to seamlessly create a volume in storage for datastore.
For more information: https://www.ibm.com/systems/storage
Disclaimer: Above are my personal thoughts and not necessarily of my employer.
Modified on by MandarVaidya
From the good old days of DOS everyone knew the benefits compression. Back in those days disk capacity was scarce.
In those days, PC's had 40MB HDD capacity and programs like Foxpro 2.6 and Windows 3.1 could not be accomodated on a single disk. One had to remove Windows 3.1 installation to make space for Foxpro. Soon came newer version of DOS with a program called “Stacker”. Stacker had the possibility to compress the disk space data and thus more space was available for applications.
Gone are the days of 40MB HDD's and soon the disk capacity increased.
In the current era of technology the data is growing tremendously. Organizations especially are facing the issues with structured and unstructured data.
IBM has a wide variety of storages available ranging from small and medium business to large enterprises with scalable capacity, and in order to give its clients more value from Storage a compression enabled storage was introduced. IBM first introduced Random Access Compression Engine (R.A.C.E) technology in the IBM Real-time Compression(RtC) appliances. IBM then integrated the same technology in IBM Storwize V7000 family in 2013.
RtC is seamlessly integrated with Storwize V7000 system software stack to compress data before writing it to disk resulting in up to 80% storage capacity savings depending on the type of data. It is effectively equal to five times more capacity out of the same physical capacity in your system. RtC allows you to compress data even before it is written to your disks and is completely transparent to the applications and at the same time maintaining data consistency. It is implemented without any changes to applications, hosts, fabric, network..etc..
Since its inception, many users started implementing RtC with their Storwize V7000. Even though RtC provided great disk space savings by compressing the data, implementation of RtC in first generation of Storwize V7000 came with performance penalty. When enabled, RtC used significant processing power of the system causing performance bottlenecks and thus the benefit offered by RtC was dwarfed by these performance issues.
IBM addressed this issue in the next generation of Storwize V7000 system by making use of hardware compression acceleration with Intel®QuickAssist Acceleration Technology, that provided dedicated processing power and greater throughput for compression.
With the new hardware compression acceleration and better hardware resources, the Storwize V7000 Gen 2 easily overcame the performance penalties seen with Storwize V7000 Gen 1 systems. The performance of Gen 2 compressed volumes exceeds the non compressed volumes of Storwize Gen 1 systems.
In order to showcase the benefits of Gen 2, benchmarking was performed with VMware's VMMark tool and Oracle databases with OLTP workloads.
Following benefits were observed on Storwize V7000 Gen2 over V7000 Gen1 for Oracle benchmarks -
- 70+% compression ratio for Oracle database files.
- Three times faster response time
- Five times faster in virtual disk (vdisk) read latency
- Four times faster managed disk (mdisk) response time
- Three time less managed disk (MDisk) write operations (compression reduces back-end I/O load, making the system more efficient, thus delivering better performance)
- With a higher number of processors, second generation of Storwize V7000 system is seamlessly able to support I/O activity with compression enabled
Following benefits were observed on Storwize V7000 Gen2 over V7000 Gen1 for VMMark benchmarks -
- Average 50% compression observed for Redhat and Windows virtual machines.
- e-Commerce workload shows 30% improvement in benchmarking scores
- e-Commerce workload shows 35 % less latency
- Mail server workload and Web application workload benchmarking scores were similar across both generations. However, lower processor utilization was observed on Gen2 even after running benchmarks over compressed volumes.
For more details, refer following ISV technical papers
Using IBM Storwize V7000 Real time compression feature with Oracle
Benefits of IBM Storwize V7000 Real-time Compression feature with VMware vSphere 5.5
Disclaimer : The thoughts expressed above are collective thoughts of Shashank Shingornikar and Mandar Vaidya. They do not necessary represent that of their employer.