This blog is for the open exchange of ideas relating to IBM Systems, storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
Tony Pearson is a Master Inventor, Senior IT Architect and Event Content Manager for [IBM Systems for IBM Systems Technical University] events. With over 30 years with IBM Systems, Tony is frequent traveler, speaking to clients at events throughout the world.
Lloyd Dean is an IBM Senior Certified Executive IT Architect in Infrastructure Architecture. Lloyd has held numerous senior technical roles at IBM during his 19 plus years at IBM. Lloyd most recently has been leading efforts across the Communication/CSI Market as a senior Storage Solution Architect/CTS covering the Kansas City territory. In prior years Lloyd supported the industry accounts as a Storage Solution architect and prior to that as a Storage Software Solutions specialist during his time in the ATS organization.
Lloyd currently supports North America storage sales teams in his Storage Software Solution Architecture SME role in the Washington Systems Center team. His current focus is with IBM Cloud Private and he will be delivering and supporting sessions at Think2019, and Storage Technical University on the Value of IBM storage in this high value IBM solution a part of the IBM Cloud strategy. Lloyd maintains a Subject Matter Expert status across the IBM Spectrum Storage Software solutions. You can follow Lloyd on Twitter @ldean0558 and LinkedIn Lloyd Dean.
Tony Pearson's books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
The developerWorks Connections platform will be sunset on December 31, 2019. On January 1, 2020, this blog will no longer be available. More details available on our FAQ.
Continuing my week in Tokyo, Japan, I was going to title this post "Chunks, Extents and Grains", but decidedinstead to use the fairy tale above.
Fellow blogger BarryB from EMC, on his The Storage Anarchist blog, once again shows off his [PhotoShop talents], in his post [the laurel and hardy of thin provisioning]. This time, BarryB depicts fellow blogger and IBM master inventor, Barry Whyte, as Stan Laurel and fellow blogger Hu Yoshida from HDS as Oliver Hardy.
At stake is the comparison in various implementations of thin provisioning among the major storage vendors.On the "thick end", Hu presents his case for 42MB chunks on his post [When is Thin Provisioning Too Thin]. On the "thin end", IBMer BarryW presents the "fine-grained" details of Space-efficient Volumes (SEV), made available with the IBM System Storage SAN Volume Controller (SVC) v4.3, in his series of posts:
BarryB paints both implementations as "extremes" in inefficiency. Some excerpts from his post:
"... Hitachi's "chubby" provisioning is probably more performance efficient with external storage than is the SVC's "thin" approach. But it is still horribly inefficient in context of capacity utilization.
... the "thin extent" size used by Symmetrix Virtual Provisioning is both larger than the largest that SVC uses, and (significantly) smaller than what Hitachi uses."
"free" may be the most expensive solution you can buy...
Before you rush off to put a bunch of SVCs running (free) SEV in front of your storage arrays, you might want to consider the performance implications of that choice. Likewise, for Hitachi's DP, you probably want to understand the impact on capacity utilization that DP will have. DP isn't free, and it isn't very space efficient, either."
BarryB would like you to think that since EMC has chosen an "extent" size between 257KB and 41MB it must therefore be the optimal setting, not too hot, and not too cold. As I mentioned last January in my post[DoesSize Really Matter for Performance?], EMC engineers had not yet decided what that extent size should be, andBarryB is noticeably vague on the current value.According to this [VMware whitepaper],the thin extent size is currently 768 KBin size. Future versions of the EMC Enginuity operating environment may change the thin extent size. (I am sure theEMC engineers are smarter and more decisive than BarryB would lead us to believe!)
BarryB is correct that any thin provisioning implementation is not "free", even though IBM's implementation is offeredat no additional charge. Some writes may be slowed downwaiting for additional storage to be allocated to satisfy the request, and some amount of storage must be set asideto hold the metadata directory to point to all these chunks, extents or grains. For the convenience of not havingto dynamically expand LUNs manually as more space is needed, you will pay both a performance and capacity "price".
However, as they say, the [proof of the pudding is in the eating], or perhaps I should say porridge in this case.Given that the DMX4 is slower than both HDS USP-V and IBM SVC, you won't see EMC publishing industry-standard[SPC benchmarks] using their"thin extent" implementation anytime soon. IBM allows a choice of grain size, from 32KB to 256KB, in an elegantdesign that keeps the metadata directory less than 0.1 to 0.5 percent overhead. I would be surprised if EMC canmake a case to be more efficient than that! The performance tests are stillbeing run, but what I have seen so far, people will be very pleased with the minimal impact from IBM SEV, an acceptable trade-off for improved utilization and reduced out-of-space conditions.
So if you are a client waiting for your EMC equipment to be fully depreciated so you can replace it for faster equipment from IBM or HDS, you can at least improveits performance and capacity utilization today by virtualizing it with IBM SAN Volume Controller.
Continuing my week's theme on how bad things can get following the "Do-it-yourself" plan, I start with James Rogers' piece in Byte and Switch, titled[Washington Gets E-Discovery Wakeup Call]. Here's an excerpt:
"A court filing today reveals there may be gaps in the backup tapes the White House IT shop used to store email. It appears that messages from the crucial early stages of the Iraq War, between March 1 and May 22, 2003, can't be found on tape. So, far from exonerating the White House staffers, the latest turn of events casts an even harsher light on their email policies.
Things are not exactly perfect elsewhere in the federal government, either. A recent [report from the Government Accountability Office (GAO)] identified glaring holes in agencies’ antiquated email preservation techniques. Case in point: printing out emails and storing them in physical files."
You might think that laws requiring email archives are fairly recent. For corporations, they began with laws like Sarbanes-Oxley that the second President Bush signed into law back in 2002. However, it appears that laws for US Presidents to keep their emails were in force since 1993, back when the first President Clinton was in office. (we might as all get used to saying this in case we have a "second" President Clinton next January!)
"The Federal Record Act requires the head of each federal agency to ensure that documents related to that agency's official business be preserved for federal archives. The Watergate-era Presidential Records Act augmented the FRA framework by specifically requiring the president to preserve documents related to the performance of his official duties. A [1993 court decision] held that these laws applied to electronic records, including e-mails, which means that the president has an obligation to ensure that the e-mails of senior executive branch officials are preserved.
In 1994, the Clinton administration reacted to the previous year's court decision by rolling out an automated e-mail-archiving system to work with the Lotus-Notes-based e-mail software that was in use at the time. The system automatically categorized e-mails based on the requirements of the FRA and PRA, and it included safeguards to ensure that e-mails were not deliberately or unintentionally altered or deleted.
When the Bush administration took office, it decided to replace the Lotus Notes-based e-mail system used under the Clinton Administration with Microsoft Outlook and Exchange. The transition broke compatibility with the old archiving system, and the White House IT shop did not immediately have a new one to put in its place.
Instead, the White House has instituted a comically primitive system called "journaling," in which (to quote from a [recent Congressional report]) "a White House staffer or contractor would collect from a 'journal' e-mail folder in the Microsoft Exchange system copies of e-mails sent and received by White House employees." These would be manually named and saved as ".pst" files on White House servers.
One of the more vocal critics of the White House's e-mail-retention policies is Steven McDevitt, who was a senior official in the White House IT shop from September 2002 until he left in disgust in October 2006. He points out what would be obvious to anyone with IT experience: the system wasn't especially reliable or tamper-proof."
So we have White House staffers manually creating PST files, and other government agencies printing out their emails and storing them in file cabinets. When I first started at IBM in 1986, before Notes or Exchange existed, we used PROFS on VM on the mainframe, and some of my colleagues printed out their emails and filed them in cabinets. I can understand how government employees, who might have grown up using mainframe systems like PROFS, might have just continued the practice when they switched to Personal Computers.
Perhaps the new incoming White House staff hired by George W. Bush were more familiar with Outlook and Exchange, and ratherthan learning to use IBM Lotus Notes and Domino, found it easier just to switch over. I am not going to debatethe pros and cons of "Lotus Notes/Domino" versus "Microsoft Outlook/Exchange" as IBM has automated email archiving systems that work great for both of these, as well as also for Novell Groupwise. So, taking the benefit of the doubt,when President Bush took over, he tossed out the previous administration's staff, and brought in his own people, andlet them choose the office productivity tools they were most comfortable with.Fair enough, happens every time a new President takes office. No big surprise there.
However, doing this without a clear plan on how to continue to comply with the email archive laws already on the books, and that it continues to be bad several years later, is appalling. I can understand why business are upset in deploying mandated archiving solutions when their own government doesn't have similar automation in place.
If you are looking for a reason to travel to Florida next month, IBM will be presenting at the [Storage Networking World conference], April 6-9, 2009 in Orlando. This conference is organized by ComputerWorld and the Storage Networking Industry Association [SNIA]. IBM is a platinum sponsor for this event, and will have various executives presenting IBM's leadership in storage:
Barry Rudolph, VP, Strategy and Stack Integration, Storage Platform
IBM will be demonstrating solutions throughout the conference, includingeight SNIA tutorial and breakout speaking sessions, a panel discussion, two new Summits (Cloud Computing, and Solid-State Storage), and four Hands-on-Labs:
Plus, IBM will have a huge 10 foot by 20 foot booth located in the Expo hall and a kiosk in the Platinum Galleria. The demonstrations highlighted in the IBM booth will showcase Information Infrastructure solutions, which will help simplify, reduce risk, increase efficiency and lower costs. I won't be there myself, but you can ask my IBM colleagues about:
The Next Generation of Storage: IBM XIV Storage System
Storage Virtualization with SAN Volume Controller (SVC)
Infrastructure Management with IBM Tivoli Storage Productivity Center (TSPC)
Data Deduplication using the IBM ProtecTIER solution
Storage and Data Services
As sponsor of this event, IBM has received a limited number of free conference passes. We will be assigning these upon request to IBM clients and prospective clients. If you would like to go, contact your IBM Business Partner or local storage rep.Act fast! First come, first served.
Continuing this week's theme on dealing with the global economic meltdown, recession and financial crisis, I found a great video that recaps IBM CEO Sam Palmisano's recommendations to being more competitive in thisenvironment.
In a recent speech to business leaders, Sam outlined what he sees as the four most importantsteps to thriving in the global economy. The highlights can be seen here in this [2-minute video]on IBM's "Forward View" eMagazine.
Soon, the U.S. is switching on-air television signals from analog to digital format. The switch-over happensFebruary 17, 2009. According to the [Federal Communications Commission], Americans haveuntil this Monday, March 31, to request up to two 40-dollar coupons towards the purchase of digital-to-analog converter boxesso that the on-air digital signals can be used with existing analog-only television equipment.
(For my readers outside the United States, a bit of background explanation may be necessary. Americans consider access to television a self-evident and unalienable right.According to a Pew Research report[Luxury or Necessity?] 64 percent of Americansconsider a television set a necessity, and 33 percent consider paid providers, like cable or satellite, a necessity.Even prisoners in U.S. jails are allowed to watch television!)
Taking advantage of the "Y2K crisis" like nature of this 2/17/2009 deadline, paid providers have been advertisingthat this deadline only applies to on-air customers. Those who have cable or satellite can continue to use theiranalog equipment. I have been a subscriber for Cox Cable for some time, and my parents recently made the switchas well. Two weeks ago, however, my parents called me in a panic. Cox Cable chose to move one channel, TurnerClassic Movies (TCM), over from their analog line-up over to their digital line-up. They thought this wasn't goingto happen until 2/17/2009! They asked me to investigate and provide them alternative options.
I spoke to a Cox Cable representative.
Did Turner force Cox Cable to do this? Did they digitize their entire collection of movies? No, Cox Cable is choosing to send the TCM signal over the digital bandwidth, and they are converted back to analog by their set-top box.
Do customers who now get one less channel get a discount? No, same price, less service.
Why move a single channel over? Eventually, everything is going digital, and this is a small "baby step" to getpeople to switch over.
But TCM is a collection of grainy, black-and-white movies from the 1950s and 1960s, it is probably the channelthat gets the least benefit to convert to digital. Why choose TCM specifically? TCM is "commercial-free" so providesno additional revenue opportunity. Moving this to digital frees up an analog channel to run a new "on demand" servicethat could generate additional revenue for Cox Cable.
What would it take in terms of additional cost and equipment to watch the TCM in digital?A set-top digital box from Cox Cable, which costs one-time 10 dollars to install by a professional technician, plus 11 dollars per month for the extra "service" provided.
Do I need a High-Def television set or other equipment? No, the digital signal for TCM is standard format, so no HD equipment required.
I currently split my cable signal, so that I can watch one channel and record another, or record two separate channels at the same time, using a standard format VCR and Tivo, can I continue to do this with the digital set-top box? Yes, absolutely.
I decided to give it a try, and a technician was scheduled to perform the installation last Sunday, which was Easter holiday for some people. The technician was able to connect the set-top box directly to my television set, but thesignal is converted to a single "Channel 3", forcing the use of a separate Cox Cable remote control unit to set the channel on the set-top box. He set the set-top box to TCM (channel 199) and showed that the TCM channel was now available again.
How would my VCR or Tivo record anything? You have to set the set-top box manually to the appropriate channel desired, then set the VCR or Tivo to record "Channel 3".
How would I record one channel while watching another? That does not appear possible with this set-top box. If we split before entering the set-top box, then that equipment would get the analog channels only, not TCM.
How about recording two different channels concurrently? No way.
I feel bad for the technician. He spent two hours on his Easter Sunday to install service that I was told by theirsales rep would work with my equipment, only to find out it won't and he ended up having to take it all back out andcancel the work order. He doesn't even get paid overtime for this.
So, I am back to where I was before, analog channels minus the TCM channel. However, the lesson is clear, eventuallyeverything is going to digital, and people may not realize what this means to them.
In case you haven't noticed, IBM System Storage makes most of their announcements on Tuesdays. IBM announced a lot today, so here is a quick run-down.
Cisco storage networking products
IBM continues to resell Cisco switches and directors, but now can offer these with a 1-year IBM warranty.
The entry-level Cisco 9124offers 8 to 24 ports. For IBM BladeCenter, IBM now offers the Cisco10-port and 20-port modules that slide into the back of the chassis, and are functionally equivalent to the 9124.The original BladeCenter came with a 16-port module with 14 internal, but only 2 external, which severely hamperedbandwidth connectivity to external storage. These new modules provide more external ports to relieve that constraint.
The midrange Cisco9200switches have two models, both with 16 fixed ports, with the option for a blade that can provide 12, 24 or 48 additional ports. The 9216A has 16 FCP ports, and the 9216i has 14 FCP ports, and 2 GbE ports to act as a router, such as toconnect to a remote location for business continuity using Metro Mirror or Global Mirror.
The enterprise-class Cisco 9500directors can support up to 528 ports.
TS3400 Tape Library
The new TS3400library is a small entry-level size library, supporting the enterprise-class TS1120 drive, providing interoperabilitywith the larger tape libraries, with all the support for tape encryption.
In addition to Linux, Unix, and WIndows, the TS1120 can now be connected to System i servers. In the past, the only IBMtape available to System i were the LTO models. There are a lot of businesses that need to comply with government regulations that are looking for tape encryption, and now IBM has made it accessible to more clients.
300GB drives at 15K RPM
The DS8000 can now support new drives with 300GB capacity at 15,000 RPM (15K). These can be up to 30 percent faster than the 10,000 RPM drives for typical workloads.
IBM continues its market leadership with these new set of features and offerings!
Continuing my coverage of the 27th annual[Data Center Conference], the weather here in Las Vegas has been partly cloudy,which leads me to discuss some of the "Cloud Computing" sessions thatI attended on Wednesday.
The x86 Server Virtualization Storm 2008-2012
Along with IBM, Microsoft is recognized as one of the "Big 5" of Cloud Computing. With theirrecent announcements of Hyper-V and Azure, the speaker presented pros-and-cons between thesenew technologies versus established offerings from VMware. For example, Microsoft's Hyper-Vis about three times cheaper than VMware and offers better management tools. That could beenough to justify some pilot projects. By contrast, VMware is more lightweight, only 32MB,versus Microsoft Hyper-V that takes up to 1.5GB. VMware has a 2-3 year lead ahead of Microsoft, and offers some features that Microsoft does not yet offer.
Electronic surveys of the audience offered some insight. Today, 69 percent were using VMware only, 8 percent had VMware plus other, including Xen-based offerings from Citrix,Virtual Iron and others. However, by 2010, the audience estimated that 39 percent would be VMware+Microsoft and another 23 percent VMware plus Xen, showing a shift away from VMware'scurrent dominance. Today, there are 11 VMware implementations to Microsoft Hyper-V, and thisis expected to drop to 3-to-1 by 2010.
Of the Xen-based offerings, Citrix was the most popular supplier. Others included Novell/PlateSpin,Red Hat, Oracle, Sun and Virtual Iron. Red Hat is also experimenting with kernel-based KVM.However, the analyst estimated that Xen-based virtualization schemes would never get past8 percent marketshare. The analyst felt that VMware and Microsoft would be the two dominant players with the bulk of the marketshare.
For cloud computing deployments, the speaker suggested separating "static" VMs from "dynamic" ones. Centralize your external storage first, and implement data deduplicationfor the OS load images. Which x86 workloads are best for server virtualization? The speaker offered this guidance:
The "good" are CPU-bound workloads, small/peaky in nature.
The "bad" are IO-intensive, those that exploit the features of native hardware
The "ugly" refers to workloads based on software with restrictive licenses and those not fully supported on VMs. If you have problems, the software vendor may not help resolve them.
Moving to the Cloud: Transforming the Traditional Data Center
IBM VP Willie Chiu presented the various levels of cloud computing.
Software-as-a-Service (SaaS) provides the software application, operating system and hardware infrastructure, such as SalesForce.com or Google Apps. Either the software meets your needs or it doesn't, but has the advantage that the SaaS provider takes care of all the maintenance.
Platform-as-a-Service (PaaS) provides operating system, perhaps some middleware like database or web application server, and the hardware infrastructure to run it on. The PaaS provider maintains the operating system patches, but you as the client must maintain your own applications. IBM has cloud computing centers deployed in nine different countries across the globe offering PaaS today.
Infrastructure-as-a-Service (IaaS) provides the hardware infrastructure only. The client must maintain and patch the operating system, middleware and software applications. This can be very useful if you have unique requirements.
In one case study, Willie indicated that moving a workload from a traditional data center to the cloud lowered the costs from $3.9 million to $0.6 million, an 84 percent savings!
We've Got a New World in Our View
Robert Rosier, CEO of iTricity, presented their "IaaS" offering. "iTricity" was coined from the concept of "IT as electricity". iTricity is the largest Cloud Computing company in continental Europe, hosting 2500 servers with 500TB of disk storage across three locations in the Netherlands and Germany.
Those attendees I talked to that were at this conference before commented that this year's focus on virtualization and cloud computing is noticeably more than in previous years. For more on this, read this 12-page whitepaper:[IBM Perspective on Cloud Computing]
IBM hired independent analyst Enterprise Strategy Group[ESG] to validate the box, and run workload-specific benchmarks. I agreewith Chris, the results are impressive! The report includes results from Microsoft Exchange JetStresstool to provide insight into email performance, and another benchmark to simulate Web server IOPS.
Also, the published SPC-1 benchmark for the DS5300 puts it at about 29 percent improvement over the DS4800.Chris argues the DS5300 is similar in class to NetApp FAS3170, which IBM sells as the IBM System Storage N6070.
If you are interesting in either the DS5300 or N6070, contact your local IBM Business Partner or sales rep.
This week, I am attending the [InterConnect Conference] in Las Vegas, Feb 21-25, 2016. This is IBM's premier Cloud & Mobile conference for the year.
Monday morning I attended the General session and a break-out session.
7030A General Session Day 1: Digital Business Transformation
The General Session was kicked off by severak clients:
Richard Holmes, Westpac Group, a 200-year-old bank with 21,000 branch locations across Australia and New Zealand. They have migrated 70 percent of their applications to the Hybrid Cloud. Provisioning server and storage resources went from 84 days to just minutes.
Matthias Rebellius, Siemens AG, Building Services. They use IBM Watson IoT to monitor the energy usage of their buildings. They have reduced energy consumption 20 to 30 percent, eliminating over 10 million cubic feet of CO2 greenhouse gas.
Robert LeBlanc, IBM Senior Vice President for Cloud, took the stage and welcomed the 23,000 attendees. Developers are turning to IBM Cloud to deliver timely, knowledgeable, and secure experiences for their customers and end-users. Business leaders are seeking new ways to enable their companies to securely implement hybrid cloud strategies that integrate mobile, IoT, and cognitive. He focused on five areas:
Choice, but with Consistency
Hybrid Cloud Integration
Powerful, Accessible data and analytics
Robert indicates that 100 percent of our strategic software products are now Hybrid-Cloud enabled. We get over 3.2 billion API calls per month, and 20,000 new IBM Bluemix users per month. More than 7,000 startups are now running on IBM SoftLayer. IBM was once again ranked #1 for Hybrid Cloud by industry analysts.
IBM predicts that 80 percent of Internet traffic will be video by year 2019. To that end, IBM offers Aspera, Ustream, and Cloudleap.
New IBM Watson APIs can analyze "tone", "emotion" and "vision".
IBM has partnered with Github to offer an Enterprise-class Github-as-a-Service offering suitable for business use.
IBM "Open for Data" has over 150 pre-populated public data sources for use with analytics. This allows applications to analyze their own data in context with public sources.
Carl Eschenbach, VMware, emphasized its partnership with IBM, announcing the ability to run VMware on IBM SoftLayer "bare metal" systems, enabling features like NSX networking and VSAN virtual storage.
Brian Cross, Apple Vice President of Product Marketing, presented the enormity of Apple's developer ecosystem:
1.5 million apps on Apple iOS application store
11 million developers making these apps
100 billion downloads of these apps
1 billion Apple devices
In the past, these developers used Xcode development environment. To take the most advantage of Apple hardware features, many developers use C or C++ programming languages to develop "Native Apps".
Apple developed a new programming language called Swift that has already made it to the top 20 development languages. He gave a demo of "Swift Playground" that allows developers see their apps running while they develop and edit the code.
Apple has made Swift open source, and extended its use across iOS, Mac OS X, Watch OS, tvOS and even Linux operating systems. This means you can write code for devices, client workstations and even servers in your datacenter or Cloud. Download it at [Swift.org].
John Ponzo, IBM Fellow, Vice President and CTO of MobileFirst, wrapped up the General session. He mentioned the "IBM Swift Sandbox" service that helps developers learn Swift programming:
Kitura -- This open source framework would allow developers to build end-to-end applications, deploy, and collaborate on web services and applications written in Swift. Kitura allows developers to build front-end and back-end code using Swift as the programming language to help simplify modern application development.
OpenWhisk -- A feature on IBM Bluemix that provides an event-driven computing service for dynamic applications. It competes against Amazon's Lambda service.
With new ways to deploy Hybrid Cloud, using new composable development tools, it is clear that "Cloud" is not merely a destination, but a new innovation platform.
1581A University of Chicago Taps into IBM Cloud Object Storage for More Effective Patient Treatments
This session was 30 minutes with Piers Nash, University of Chicago - Center for Data Intensive Science (CDIS), client testimonial, followed by Russell Kennedy, IBM, that covered an overview of Cleversafe used in the solution.
University of Chicago's Center for Data Intensive Science (CDIS) accelerates medical discoveries by democratizing access to data for scientific research. Utilizing an object storage solution, CDIS centrally stores and manages vast amounts of genomic and clinical data at web-scale, allowing researchers to collaborate via shared access to harmonized data sets, speeding discovery and enabling precision medicine.
Their initial focus is cancer research. Cancer costs over $100 Billion USD per year in healthcare costs. It is #1 killer among people under 85 years old, affecting half of all men, and a third of all women. There are 1.7 million new cancer cases in the USA every year, 15 million worldwide.
There is no "single cure" for cancer. Whereas all humans share nearly identical 3.2 billion base pairs of genetic material, there are over 15,000 different kinds of cancers, each with its own genome. Capturing RNA sequences of patients results in images 10-20 GB in size, and over the course of treatment could add up to 1 TB of image data per patient. A million patients with 1TB of data each would be an Exabyte of data (1,000 Petabytes).
To store all of this data, CDIS created the Bionimbus Protected Data Cloud, using Cleversafe as the underlying storage technology. This system goes live June 2016, and they plan to keep the data forever.
(We'll see how well that goes 10 years from now! It might be cheaper just to re-sequence a human's DNA as needed, rather than storing it forever, since an individual's DNA never changes.)
The data is "de-identified" meaning that researches using the data are unable to identify individual people associated with each case study or genomic result. They have already collected 1.66 PB of this data.
Most cancer treatments that have been effective have focused on specific genetics. The problem is targeting precise therapies to the right patients. For example, there are two very similar Lung cancers, and about 20 percent of the time, a Lung cancer is mis-identified, such that the patients has adverse reactions to the wrong treatment. By having more analytics-based medicine, the hope is to reduce this trial-and-error approach.
Russ Kennedy, IBM, wrapped up the session explaining Cleversafe, which was a Chicago-based company formed in 2004 that was acquired last year by IBM. Why did University of Chicago choose Cleversafe? Several reasons:
University of Chicago attempts to use open source projects like Gluster or Ceph failed around the 1-2 PB mark. They knew they would need much more than this!
Cleversafe was a Chicago-based company, offering local support
IDC ranked Cleversafe #1 marketshare leader of object storage in 2014 and 2015! It beats out competitors like Dell/EMC and Salty, as well as Cloud Service Providers like Amazon or Google.
Why object storage? IBM predicts a 332 percent growth in data generated from Mobile devices. As much as 90 percent of traffic on Mobile devices will be from Cloud apps rather than voice or text messages. There will be a 10-fold increase of data stored by year 2020, and at least 80 percent of this data will be unstructured content. Cleversafe estimates that managing object storage requires 15x fewer administrators than traditional storage.
Cleversafe consists of three components. The "Accessor" is software that runs bare metal, as Virtual Machine or Docker container. It offers the OpenStack Swift, HTTP/REST and Amazon S3 object-based interfaces to ingest the data. The data is encrypted, divided into pieces, then through a process called [Erasure Coding] is converted to slices. Those slices are stored on storage-rich servers called "storage nodes".
For example, five pieces of data converted to nine slices could be stored on nine machines, three machines at Site 1, three at Site 2, and three at Site 3. You only need to read back any five slices to reconstruct the data, so you could lose any four of the nine machines and still have full recoverability. If the 5/9 example above, you could lose any one site, and a machine in one of the two remaining surviving sites, and still retrieve all of your data.
There is now an "open beta" called the Transparent Cloud Storage Tiering that bridges GPFS and Spectrum Scale over to Cleversafe.
I wrapped up the morning with a lunch at Border Grill with storage clients and IBM Business Partners. This was the best steak I have had this week!
Are you covering the business impact of the internet failure across Asia, the Middle East and North Africa? The outage has brought business in those regions to a standstill. This disaster shines a direct spotlight on the vulnerability of technology and serves as a reminder of the ever increasing importance of protecting business critical information.
Disaster recovery needs to be a critical element of every technology plan. We don’t yet know the financial impact of this wide spread internet failure, but the companies with disaster recovery plans in place, were likely able to failover their entire systems to servers based in other regions of the world.
When I first heard of this outage, I am thinking, so a few million people don't have access to FaceBook and YouTube, what's the big deal? We in the U.S.A. are in the middle of a [Hollywood writer's strike] and don't have fresh new television sitcoms to watch! Yahoo News relays the typical government's response:[Egypt asks to stop film, MP3 downloads during Internet outage], presumably so that real business can take priority over what little bandwidth is still operational. Fellow IBM blogger "Turbo" Todd Watson pokes fun at this, in his post[Could Someone Please Get King Tutankhamun On The Phone?].Like us suffering here in America, perhaps our brothers and sisters in Egypt and India may getre-acquainted with the joys of reading books.
However, the [Internet Traffic Report-Asia] shows how this impacted various locations including: Shanghai, Mumbai, Tokyo, Tehran, and Singapore. In some cases, you have big delays in IP traffic, in other cases, complete packet loss, depending on where each country lies on the["axis of evil"].This is not something just affecting a few isolated areas, the impact is indeed worldwide. This would be a goodtime to talk about how computer signals are actually sent.
DWDM takes up to 80 independent signals, converts each to a different color of light, and sends all the colors down a single strand of glass fiber. At the receiving end, the colors are split off by a prism,and each color is converted back to its original electrical signal.
Similar DWDM, but only eight signals are sent over the glass fiber. This is generally cheaper, becauseyou don't need highly tuned lasers.
Wikipedia has a good article on [Submarine Communications Cable],including a discussion on how repairs are made when they get damaged or broken.It is important to remember that lost connectivity doesn't mean lost data, just lack of access to the data. Thedata is still there, you just can't get to it right now. For some businesses, that could be disruptive to actualoperations. In other cases, it means that backups or disk mirroring is suspended, so that you only have yourlocal copies of data until connectivity is resumed.
When two cables in the Mediterranean were severed last week, it was put down to a mishap with a stray anchor.
Now a third cable has been cut, this time near Dubai. That, along with new evidence that ships' anchors are not to blame, has sparked theories about more sinister forces that could be at work.
For all the power of modern computing and satellites, most of the world's communications still rely on submarine cables to cross oceans.
It gets weirder. In his blog Rough Type, Nick Carr's[Who Cut the Cables?] reportsnow a fourth cable has been cut, in a different location than the other two cable locations. If the people cuttingthe cables are looking to see how much impact this would have, they will probably be disappointed. Nick Carrrelates how resilient the whole infrastructure turned out to be:
Though India initially lost as much as half of its Internet capacity on Wednesday, traffic was quickly rerouted and by the weekend the country was reported to have regained 90% of its usual capacity. The outage also reveals that the effects of such outages are anything but neutral; they vary widely depending on the size and resources of the user.
Outsourcing firms, such as Infosys and Wipro, and US companies with significant back-office and research and development operations in India, such as IBM and Intel, said they were still trying to asses how their operations had been impacted, if at all.
Whether it is man-made or natural disaster, every business should have a business continuity plan. If you don't have one, or haven't evaluated it in a while, perhaps now is a good time to do that. IBM can help.
"Our survey data shows that over the past 12 months, more firms have bought their storage from a single vendor. While this might not be for everyone, it's worth serious consideration for your environment. Maybe you won't get the best price per gigabyte every time, but you'll probably save money in the long run because of simpler management, increased staff specialization, increased capacity utilization, and better customer service."
A Forrester survey of 170 companies ranging from SMBs to large enterprises in North America and Europe found that more than 80 percent bought their primary storage from one vendor over the last year. That includes 64 percent of the companies with more than 500 TB of raw storage.
The report, written by analyst Andrew Reichman, says using more than one primary storage vendor can make it more complex to manage, provision and support the storage environment. And while using multiple vendors can often bring better pricing, buying from one vendor can result in volume discounts.
“You may have tried to contain costs by forcing multiple incumbent vendors to continuously compete against each other, with price as the primary differentiator,” Reichman writes. “This strategy can reduce prices and limit vendor lock-in, but it can also lead to management complexity and poor capacity utilization.”
The report recommends keeping things simple by and using fewer vendors when possible. However, that advice comes with several caveats: buying all storage from one vendor means taking the bad with the good, and some vendors’ product families differ so much “they may as well come from different vendors.”
As if by coincidence, fellow blogger from EMC Chuck Hollis gives his reflections on this same topic. Here's an excerpt:
When it comes to buying storage (or any infrastructure technology, for that matter), there seem to be two camps:
Best-of-breed (i.e. multivendor): -- buy what's best, get the best price, keep all the vendors on their toes, etc. etc.
Single vendor: primarily use one vendor's offerings, and hold them accountable for the outcome.
If Chuck had said "multivendor" versus "single vendor", then that would have been a true dichotomy, but interestinglyhe equates best-of-breed with a multivendor approach. Let's consider two examples:
Disk from one vendor, Tape from another
Here is a multivendor strategy, and if you have a clear idea of what best-of-breed means to you, then you couldpick the best disk in the market, and the best tape in the market. However, I don't think this keeps either vendor"on their toes", or helps you negotiate lower prices by threatening to switch to the other vendor. In shops likethis, the staffing usually matches, so there are disk administration and tape operations, with little or no overlap, andlittle or no interest in retraining to use a new set of gear. It is true that disk-based VTL could be used where real tape libraries are used, but this may not be enough to threaten your existing vendors that you will switch all your disk to tape, or all your tape to disk.
One could argue that the vendor that sells the besttape could be the exact same vendor that sells the best disk. In this case, your multivendor strategy would actuallywork against you, forcing you away from one of your best-of-breed choices.
Disk and Tape from one vendor for some workloads, Disk and Tape from another vendor for other workloads
Here is a different multivendor strategy. Having disk and tape for the same vendor allows you to take advantageof possible synergies. The IT staff knows how to use the products from both vendors. This strategy does let you keep your vendors "on their toes". You can legitimately threaten to shift your budget from one vendor over another.However, whatever your definition of best-of-breed is, chances are the product from one vendor is, and the other vendor is not. Both meet some lowest common denominator, meeting some minimum set of requirements, which would allow you to swap out one for the other.
I guess I look at it differently. The equipment in your data center should be thought of as a team. Do your servers, storage and software work well together?
While Americans like to celebrate the accomplishments of individual musicians, athletes or executives, it is actually bands that compete against other bands, sports teams that compete against other sport teams, and companies that compete against other companies. Teamwork in the data center is not just for the people who work there, but also for the IT equipment. Just as a new incoming athlete may not get along well with teammates, shiny new equipment may not get along with your existing gear. Conversely, your existing infrastructure may not let the talents or features of your new equipment shine through.
Putting together the best parts from different teams might serve as a great diversion for those who enjoy["fantasy football"], it may not be the best approach for the data center. Instead, focus on managing your data center as a team, perhaps with theuse of IBM TotalStorage Productivity Center to minimize the heterogeneity of your different equipment. Pick an ITvendor that sells "team players" for your servers, storage and software, with broad support for interoperability and compatibility.
This week, I am attending the [InterConnect Conference] in Las Vegas, Feb 21-25, 2016. This is IBM's premier Cloud & Mobile conference for the year.
Wednesday morning I attended more break-out sessions.
1273: New IBM DS8880 Family: Always-On Data at Cloud Speed
Brian Sherman (with support from Eddie Lin) explored the business value that the IBM System Storage DS8000 series provides to organizations requiring ultimate performance and availability.
Brian reviewed the DS8000 advanced functions, including those that have recently become available, and explains what benefits they provide. While he focused on the latest DS8880 family, some of these were also available on the prior DS8870 models.
Cloud-related features include OpenStack Cinder drivers, REST interfaces, Mobile app monitoring, zKVM and PowerVC support, use of IBM Spectrum Control Base, VMware VAAI primitives, SRA and Web-admin plugin support.
3015A Open Doors with an OpenStack Approach
Mohammed "Mo" Abdula, IBM, presented this overview of IBM's involvement with OpenStack, including BlueBox, which provides a private on-premises OpenStack deployment.
Most enterprises know that a single approach to cloud adoption, whether public or private, will not optimize business results. Connecting one or more clouds to traditional systems, or other clouds, is a realistic and achievable strategy.
OpenStack, being an open technology, is making it easy for enterprises to customize the way they deploy mission-critical business applications.
Code, Community and Culture enable innovation - Cloud should hide the details so that people can focus on what is important. OpenStack is opening the doors for enterprises to quickly get on the Cloud journey.
The automotive industry heavily uses OpenStack. Mo gave an example of a successful promotion by a car dealer that resulted in great sales revenues through social media. The app was developed on IBM SoftLayer than moved on-premises. OpenStack interfaces made it possible.
7186A IBM Spectrum Storage Experiences
Douglas O'Flaherty, IBM, served as emcee for this exciting discussion. Three clients presented their success stories with various Spectrum Storage software. Each speaker had 20 minutes to present their story.
Paul Rafferty, IBM Silverpop
Silverpop was a started that provides Marketing automation, empowering marketers with cloud-based capabilities and cutting-edge big data analytics that deliver personalized customer engagements that scale for any sized business. It was IBM acquired in 2014, but Paul presented as a client of IBM Spectrum Accelerate.
To support clients, Silverpop does everything in the Cloud. With their acquisition by IBM, they have switched to using IBM SoftLayer. To that end, they needed robust storage that provides snapshots, consistency groups, and remote disk-to-disk replication, so they selected bare-metal servers running with IBM Spectrum Accelerate, which is the software-only implementation of XIV storage systems.
Silverpop deploys Spectrum Accelerate on either 7-node or 15-node clusters, with an additional spare-node pre-configured in case of failure. Each node is a 2U x86 server with dual 8-core Intel Xeon E5-2650 processors, 128 GB RAM, two 800GB Solid-State Drives (SSD) and 10 SATA drives 4TB capacity each. The 7-node provides about 120TB of usable capacity, and the 15-node about 255 TB.
Worldwide, Silverpop has 1,500 nodes deployed across 10 IBM SoftLayer datacenters, running 15,000 virtual machines. The virtual machines run on the same nodes as Spectrum Accelerate, including Oracle database, DB2 database, HDFS file system, and Spark analytics. They use Chef and UrbanCode for orchestration and code deployment.
If you ask 10 different Spectrum Protect architects how to design a system, you get a wide variety of answers. Blueprints reduce this complexity down to three "T-shirt" sizes: Small, Medium, and Large, based on the amount of backup traffic per day. Small for deployments less than 6 TB per day, Medium for 6-20 TB per day, and Large for over 20 TB per day.
The blueprints can be deployed on Windows, Linux-x86, Linux on POWER, and AIX. They are disk-based storage pools using either IBM Storwize family or Elastic Storage Server models. The blueprints include configuration scripts that can be customized, and Joe suggested tips for those who want to incorporate tape storage pools.
Bob Oesterlin, Nuance
Nuance creates their Nuance Dragon® voice-recognition dictation software. They process 7500 TB per day, 85% read, 15% write traffic. They have 6 PB of Spectrum Scale file system.
To free up space and reduce costs, Nuance stood up their own OpenStack Swift object-store on storage-rich servers. Files that have turned cold were moved out of Spectrum Scale and into this Object Store, which has now grown to over 4 PB of capacity. Unfortunately, there was no way for end-users who had files on Scale to find them after they were moved to Object Store.
IBM has solved this with Transparent Cloud Storage Tiering, which is currently in open beta. With this new approach, files are "migrated" from Spectrum Scale to Cleversafe object-store, but a stub is left behind in the file system directory so that they can be "recalled" back to Spectrum Scale. This is the same methodology IBM uses to migrate/recall data to tape.
I would vote this the best session I have seen all week! Each client solved real-world business problems with Spectrum Storage software.
To encourage traffic through the Solutions EXPO, foot traffic was re-directed through the booths to get to lunch. This reminds me of having to go through the "gift shop" when you leave amusement rides or museums.
Here we are again at Top Gun class.In between class topics, we often show short video clips.
This week, we saw IBM Executive Bob Hoey's wisdom on selling mainframe computers. Bob is the VP of Sales for our System z server line, but the lessons might also apply to high-end disk or enterprise tape libraries.
Shakespeare wrote "What's in a name? That which we call a rose by any other word would smell as sweet." This week my theme will be on names, naming convention, and how we access information on storage.
Take for example these two sentences:
The Bears beat New Orleans. Chicago clobbered the Saints.
Though they appear very different, football fans who might have watched either or both of the two conference title games yesterday would quickly recognize that they refer to the same two teams and the same end-result.
I'll be traveling to Asia next week. While most people call me "Tony", my legal given name is "Anthony" which is what appears on my passport and other legal documents. Most English-speaking countries handle this fine, but it can be confusing in Japan or China, where "A. Pearson" doesn't match "T. Pearson".
In the US, our given and family names are referred to as our "first name" and our "last name", relating to their positional sequence. In Asia, family names come first, followed by their given names last. To help avoid confusion, we have started adopting the practice of putting the family name in ALL CAPITAL LETTERS, so I would "Tony PEARSON" while my colleague may be "WONG Francis".
In Japanese, "Mr. JONES" would be "Jones-san". However, Pearson-san is such a toungue-twister, that most just say "Tony-san" which is fine with me. I have been called "Mr. Tony" in a variety of countries, perfectly acceptable.
You can call me anything you like, just don't call me late for dinner.
Well, it's Tuesday, and more IBM announcements were made today. Many of my colleagues are in Dallas, Texas for the[Storage Networking World conference], and hopefully I will get some feedback from them before the week is over.
Today, IBM made announcements for Storage Area Networking (SAN) gear and disk systems.
8 Gbps Longwave transceivers
IBM now offers 8 Gbps Longwave SFP transceivers on the[IBM System Storage SAN256B and SAN768B] directors, as well as the IBM System Storage SAN24B-4 Express, SAN40B-4, and SAN80B-4 switches (orderable as [machine type models] or [partnumbers] ).These transceivers support single mode fiber up to 10km in distance, comparedto the 50-75 meters supported by the Shortwave SFP transceivers.
Like theShortwave SFP transceivers we already have available, these Longwave transceivers have "N-2" support, which means they can support two generations back: auto-negotiate down to 4 Gbps and 2 Gbps speeds. If you still have 1 Gbps equipment, now is a good time to consider upgrading those, or keep a few 4 Gbps ports available that can auto-negotiate down to 1 Gbps speed.
Mainframe clients that sent data to a remote Business Continuity/Disaster Recovery (BC/DR) location often used "channel extenders", which were special boxes used to minimize performance delays when transmitting FICON across long distances. This was especially helpful for z/OS Global Mirror (what we used to call XRC) as well as electronic vaulting to tape.
Now, this functionality can be part of the directors and routers, eliminating the need for separate equipment.This is available for the SAN768B and SAN256B directors, as well as SAN18B-R and SAN04B-R routers.
Before the merger between Brocade and McDATA, IBM offered SAN18B-R routers from Brocade, and SAN04M-R routers from McDATA. The former had 16 Fibre Channel (FC) ports and two Ethernet ports, and the latter was less expensive with just four ports.Brocade came up with a clever replacement for both. The [IBMSystem Storage SAN04B-R] router comes by default withtwo active FC ports and two Ethernet ports, but also with 14 additional FC ports inactive. A "High Performance Extension" feature activates these additional ports, bringing the SAN04B-R up to the SAN18B-R level, and allows it to support the FICON Accelerator feature above.
So, instead of having specialized channel extenders at both primary and secondary sites, you can havea director with FICON Accelerator at the primary site, sending FICON over Ethernet to a 1U-high router (also running the FICON Accelerator) at the secondary site, whichcan greatly reduce costs. The FICON Accelerator can in some cases double the amount of data transfer throughput,but of course, your mileage may vary.
On the disk side, the [IBMSystem Storage DS3000 series] disk systems have been enhanced, withsupport for 450GB high-speed 15K RPM SAS drives, RAID-6 double-drive protection, more FlashCopy point-in-time copies,and more partitions.On the DS3000, "storage partitions" is what the rest of the industry calls "LUN masking". A storage partition allowsyou to isolate a set of LUNs to only be seen by a single host server, or host cluster that shares the same set ofLUNs. Some clients felt that the default of four partitions was too low, so now up to 32 partitions can be configured.(This is not to be confused with "Logical Partitions" that isolate processor and cache resources available on theIBM System Storage DS8000 and other high-end storage disk systems.)
IBM also extended the Operating System support.The DS3000 series now supports Solaris, either on x86 or SPARC-based servers. The DS3300 iSCSI support now supportsLinux on POWER. The DS3400 allows support of IBM i (the new name for i5/OS V6R1) through the VIOS feature.
The [IBMSystem Storage DCS9900] is a bigger, faster version of the DCS9550. Like the DCS9550, the DCS9900 is designedfor high performance computing (HPC) workloads. The DCS9550 supported up to 960TB in two frames, with 2.8 GB/sec throughput,and an optional disk spin-down capability.The new DCS9900 can support up to 1.2 PB in two frames, with 5.6 GB/sec throughput, but no spin-down capability.
So whether your data center is filled with System z mainframes, or other open systems, IBM has a solution for you.
IBM released its [2008 Annual Report]. IBM has improved in revenues, profits and earnings per share compared to recent past years. Part of the success comes from IBM's focus on [generating higher value].Here are some excerpts:
"Several years ago, we saw change coming.
Value was shifting in the IT industry, driven by the rising tide of global integration, a new computing paradigm and new client needs. These shifts meant the world was becoming not just smaller and “flatter,” but also smarter.
We remixed our businesses in order to move to the emerging higher-value spaces.
IBM has divested commoditizing businesses like personal computers, and strengthened its position through strategic investments and acquisitions in higher-value segments like business intelligence and analytics, virtualizationand green solutions.
From 2000 to 2008 we acquired more than100 companies to complement and scale our portfolioof products and offerings. This has changed ourbusiness mix toward higher-value, more profitable segments of the industry.
We became a globally integrated enterprise in order to capture the best growth opportunities and improve IBM’s profitability.
IBM operates in more than 170 countries and enjoys an increasingly broad-based geographic reach.Our non-U.S. operations generated approximately65 percent of IBM’s revenue in 2008. IBM’s Growth Markets unit, which was established in 2008,grew 10 percent last year, and made up 18 percentof our revenues. Revenue increased 18 percent(15 percent in local currency) in Brazil, Russia, India and China.
As a result, IBM is a higher-performing enterprise today than it was a decade ago.
Our business model is more aligned with our clients’ needsand generates better financial results.
We have therefore been able to invest in future sources of growth and provide record return to investors…
…while continuing to invest in R&D—more than $50 billion from 2000 to 2008.
This gives us confidence that we are entering the current economic environment from a position of strength…
In 2008 we made progress toward our 2010 objectivesby growing earnings per share 24 percent. And withthis strong 2008 performance, we are clearly ahead of pace on our road map to $10–$11 of earnings per share.
…and that we will emerge from it even stronger, thanks to our long-term fundamentals and our agenda for a smarter planet.
All around the world, businesses, governmentsand institutions are investing to reduce costs,drive innovation and transform their infrastructure. The economic downturn has intensified this trend,as leaders seek not simply to repair what isbroken, but to prepare for a 21st Century economy.
Many of their key priorities are in areas whereIBM has leading solutions—such as smarter utility grids, traffic, healthcare, financial systems,telecommunications and cities. We are aggressively pursuing this transformational, global opportunity."
It is good to see that IBM continues to proceed with long-term investments during these tough times!