This blog is for the open exchange of ideas relating to IBM Systems, storage and storage networking hardware, software and services.
(Short URL for this blog: ibm.co/Pearson )
Tony Pearson is a Master Inventor, Senior IT Architect and Event Content Manager for [IBM Systems for IBM Systems Technical University] events. With over 30 years with IBM Systems, Tony is frequent traveler, speaking to clients at events throughout the world.
Lloyd Dean is an IBM Senior Certified Executive IT Architect in Infrastructure Architecture. Lloyd has held numerous senior technical roles at IBM during his 19 plus years at IBM. Lloyd most recently has been leading efforts across the Communication/CSI Market as a senior Storage Solution Architect/CTS covering the Kansas City territory. In prior years Lloyd supported the industry accounts as a Storage Solution architect and prior to that as a Storage Software Solutions specialist during his time in the ATS organization.
Lloyd currently supports North America storage sales teams in his Storage Software Solution Architecture SME role in the Washington Systems Center team. His current focus is with IBM Cloud Private and he will be delivering and supporting sessions at Think2019, and Storage Technical University on the Value of IBM storage in this high value IBM solution a part of the IBM Cloud strategy. Lloyd maintains a Subject Matter Expert status across the IBM Spectrum Storage Software solutions. You can follow Lloyd on Twitter @ldean0558 and LinkedIn Lloyd Dean.
Tony Pearson's books are available on Lulu.com! Order your copies today!
Safe Harbor Statement: The information on IBM products is intended to outline IBM's general product direction and it should not be relied on in making a purchasing decision. The information on the new products is for informational purposes only and may not be incorporated into any contract. The information on IBM products is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for IBM products remains at IBM's sole discretion.
Tony Pearson is a an active participant in local, regional, and industry-specific interests, and does not receive any special payments to mention them on this blog.
Tony Pearson receives part of the revenue proceeds from sales of books he has authored listed in the side panel.
Tony Pearson is not a medical doctor, and this blog does not reference any IBM product or service that is intended for use in the diagnosis, treatment, cure, prevention or monitoring of a disease or medical condition, unless otherwise specified on individual posts.
The developerWorks Connections Platform is now in read-only mode and content is only available for viewing. No new wiki pages, posts, or messages may be added. Please see our FAQ for more information. The developerWorks Connections platform will officially shut down on March 31, 2020 and content will no longer be available. More details available on our FAQ. (Read in Japanese.)
Well, it's Tuesday again, and you know what that means! IBM Announcements!
Today, IBM announced its latest IBM Tivoli Key Lifecycle Manager (TKLM) 2.0 version. Here's a quick recap:
Centralized Key Management
Centralized and simplified encryption key management through Tivoli Key Lifecycle Manager's lifecycle of creation, storage, rotation, and protection of encryption keys and key serving through industry standards. TKLM is available to manage the encryption keys for LTO-4, LTO-5, TS1120 and TS1130 tape drives enabled for encryption, as well as DS8000 and DS5000 disk systems using Full Disk Encryption (FDE) disk drives.
Partitioning of Access Control for Multitenancy
Access control and partitioning of the key serving functions, including end-to-end authentication of encryption clients and security of exchange of encryption keys, such that groups of devices have different sets of encryption keys with different administrators. This enables [multitenancy] or multilayer security of a shared infrastructure using encryption as an enforcement mechanism for access control. As Information Technology shifts from on-premises to the cloud, multitenancy will become growingly more important.
Support for KMIP 1.0 Standard
Support for the new key management standard, Key Management Interoperability Protocol (KMIP), released through the Organization for the Advancement of Structured Information Standards [OASIS]. This new standard enables encryption key management for a wide variety of devices and endpoints. See the
[22-page KMIP whitepaper] for more information.
As much as I like to poke fun at Oracle, with hundreds of their Sun/StorageTek clients switching over to IBM tape solutions every quarter, I have to give them kudos for working cooperatively with IBM to come up with this KMIP standard that we can both support.
Support for non-IBM devices from Emulex, Brocade and LSI
Support for IBM self-encrypting storage offerings as well as suppliers of IT components which support KMIP, including a number of supported non-IBM devices announced by business partners such as Emulex, Brocade, and LSI. KMIP support permits you to deploy Tivoli Key Lifecycle Manager without having to worry about being locked into a proprietary key management solution. If you are a client with multiple "Encryption Key Management" software packages, now is a good time to consolidate onto IBM TKLM.
Role-based access control for administrators that allows multiple administrators with different roles and permissions to be defined, helping increase the security of sensitive key management operations and better separation of duties. For example, that new-hire college kid might get a read-only authorization level, so that he can generate reports, and pack the right tapes into cardboard boxes. Meanwhile, for that storage admin who has been running the tape operations for the past ten years, she might get full access. The advantage of role-based authorization is that for large organizations, you can assign people to their appropriate roles, and you can designate primary and secondary roles in case one has to provide backup while the other is out of town, for example.
Wrapping up my week's theme of storage optimization, I thought I would help clarify the confusion between data reduction and storage efficiency. I have seen many articles and blog posts that either use these two terms interchangeably, as if they were synonyms for each other, or as if one is merely a subset of the other.
Data Reduction is LOSSY
By "Lossy", I mean that reducing data is an irreversible process. Details are lost, but insight is gained. In his paper, [Data Reduction Techniques", Rajana Agarwal defines this simply:
"Data reduction techniques are applied where the goal is to aggregate or amalgamate the information contained in large data sets into manageable (smaller) information nuggets."
Data reduction has been around since the 18th century.
Take for example this histogram from [SearchSoftwareQuality.com]. We have reduced ninety individual student scores, and reduced them down to just five numbers, the counts in each range. This can provide for easier comprehension and comparison with other distributions.
The process is lossy. I cannot determine or re-create an individual student's score from these five histogram values.
This next example, complements of [Michael Hardy], represents another form of data reduction known as ["linear regression analysis"]. The idea is to take a large set of data points between two variables, the x axis along the horizontal and the y axis along the vertical, and find the best line that fits. Thus the data is reduced from many points to just two, slope(a) and intercept(b), resulting in an equation of y=ax+b.
The process is lossy. I cannot determine or re-create any original data point from this slope and intercept equation.
In this last example, from [Yahoo Finance], reduces millions of stock trades to a single point per day, typically closing price, to show the overall growth trend over the course of the past year.
The process is lossy. Even if I knew the low, high and closing price of a particular stock on a particular day, I would not be able to determine or re-create the actual price paid for individual trades that occurred.
Storage Efficiency is LOSSLESS
By contrast, there are many IT methods that can be used to store data in ways that are more efficient, without losing any of the fine detail. Here are some examples:
Thin Provisioning: Instead of storing 30GB of data on 100GB of disk capacity, you store it on 30GB of capacity. All of the data is still there, just none of the wasteful empty space.
Space-efficient Copy: Instead of copying every block of data from source to destination, you copy over only those blocks that have changed since the copy began. The blocks not copied are still available on the source volume, so there is no need to duplicate this data.
Archiving and Space Management: Data can be moved out of production databases and stored elsewhere on disk or tape. Enough XML metadata is carried along so that there is no loss in the fine detail of what each row and column represent.
Data Deduplication: The idea is simple. Find large chunks of data that contain the same exact information as an existing chunk already stored, and merely set a pointer to avoid storing the duplicate copy. This can be done in-line as data is written, or as a post-process task when things are otherwise slow and idle.
When data deduplication first came out, some lawyers were concerned that this was a "lossy" approach, that somehow documents were coming back without some of their original contents. How else can you explain storing 25PB of data on only 1PB of disk?
(In some countries, companies must retain data in their original file formats, as there is concern that converting business documents to PDF or HTML would lose some critical "metadata" information such as modificatoin dates, authorship information, underlying formulae, and so on.)
Well, the concern applies only to those data deduplication methods that calculate a hash code or fingerprint, such as EMC Centera or EMC Data Domain. If the hash code of new incoming data matches the hash code of existing data, then the new data is discarded and assumed to be identical. This is rare, and I have only read of a few occurrences of unique data being discarded in the past five years. To ensure full integrity, IBM ProtecTIER data deduplication solution and IBM N series disk systems chose instead to do full byte-for-byte comparisons.
Compression: There are both lossy and lossless compression techniques. The lossless Lempel-Ziv algorithm is the basis for LTO-DC algorithm used in IBM's Linear Tape Open [LTO] tape drives, the Streaming Lossless Data Compression (SLDC) algorithm used in IBM's [Enterprise-class TS1130] tape drives, and the Adaptive Lossless Data Compression (ALDC) used by the IBM Information Archive for its disk pool collections.
Last month, IBM announced that it was [acquiring Storwize. It's Random Access Compression Engine (RACE) is also a lossless compression algorithm based on Lempel-Ziv. As servers write files, Storwize compresses those files and passes them on to the destination NAS device. When files are read back, Storwize retrieves and decompresses the data back to its original form.
As with tape, the savings from compression can vary, typically from 20 to 80 percent. In other words, 10TB of primary data could take up from 2TB to 8TB of physical space. To estimate what savings you might achieve for your mix of data types, try out the free [Storwize Predictive Modeling Tool].
So why am I making a distinction on terminology here?
Data reduction is already a well-known concept among specific industries, like High-Performance Computing (HPC) and Business Analytics. IBM has the largest marketshare in supercomputers that do data reduction for all kinds of use cases, for scientific research, weather prediction, financial projections, and decision support systems. IBM has also recently acquired a lot of companies related to Business Analytics, such as Cognos, SPSS, CoreMetrics and Unica Corp. These use data reduction on large amounts of business and marketing data to help drive new sources of revenues, provide insight for new products and services, create more focused advertising campaigns, and help understand the marketplace better.
There are certainly enough methods of reducing the quantity of storage capacity consumed, like thin provisioning, data deduplication and compression, to warrant an "umbrella term" that refers to all of them generically. I would prefer we do not "overload" the existing phrase "data reduction" but rather come up with a new phrase, such as "storage efficiency" or "capacity optimization" to refer to this category of features.
IBM is certainly quite involved in both data reduction as well as storage efficiency. If any of my readers can suggest a better phrase, please comment below.
In preparation for my [upcoming trip to Australia and New Zealand], I decided to upgrade my smartphone. My service provider T-Mobile offered me the chance to try out any new phone for 14 days for only ten dollar re-stocking fee. For the past 16 months, I have used the Google G1 phone. This is based on a storage-optimized Android operating system, based on open source Linux, with applications processed in a storage-optimized virtual machine called Dalvik, based on open source Java. According to Wikipedia, Android-based phones have #1 market share [outselling both BlackBerry OS and Apple iOS phones]. There are over 70 different companies using Android, driven away from the proprietary interfaces from Apple, BlackBerry and Microsoft.
Since I was already familiar with the Android operating system, I chose the Samsung Galaxy S Vibrant. I liked my G1, but it had only a small amount of internal memory to store applications. The G1 supported an external Micro SDHC card, but this only was used for music and photos. There was no way to install applications on the memory card, so I found myself having to uninstall applications to make room for new ones. By contrast, the Vibrant has 16GB internal memory, plenty of room for all applications, and supports Micro SDHC up to 32GB in size. My model can pre-installed with a 2GB card, of which 1.4GB is consumed by James Cameron's full-length movie Avatar. On the G1, swapping out memory cards was relatively easy. On the Vibrant, you have to take the phone apart to swap out cards, so I won't be doing that very often. I will probably just get a 32GB card and leave it in there permanently.
(FTC disclosure: I work for IBM. IBM has working relationships with Oracle, Google, and lots of other companies. IBM offers its own commercial version of Java related tools. I own stock in IBM, Apple, Google. I have friends and family who work at Microsoft. My review below is based entirely on my own experience of my new Samsung Galaxy S Vibrant phone. Samsung has created different models for different service providers. The T-Mobile Vibrant is an external USB storage device with telephony capabilities, comparable to the AT&T Captivate, Verizon Fascinate, or Sprint Epic 4G. The majority of mobile phones in the world contain IBM technology. This post is not necessarily an endorsement for Samsung over other smartphone manufacturers, nor T-Mobile over other service providers. I provide this information in context of storage optimization, state-of-the-art for smartphones in general, and disputes related to software patents between companies. I hold 19 patents, most of which are software patents.)
When Oracle acquired Sun Microsystems, it inherited stewardship of Java. Java is offered in two flavors. Java Standard Edition (SE) for machines that are planted firmly on or below your desk, and Java Micro Edition (ME) for machines that are carried around. Most Java-based phones limit themselves to Java ME, but Google decided to base its smartphones on the more powerful Java SE, but then optimize for the limited storage and computing resources. These two levels of Java have radically different licensing terms and conditions, so Larry Ellison of Oracle cried foul. On The Register, Gavin Clarke has an excellent article with details of the Oracle-vs-Google complaint. Daniel Dilger opines that Oracle [might kill Google’s Android and software patents all at once]. Fellow blogger Mark Twomey (EMC) on his StorageZilla blog, argues that [it's not about Android phones, but Android everything].
My Vibrant is roughly the size of a half-inch stack of 3x5 index cards in my hand. In my humble opinion, the problem is the grey area between mobile phone and the desktop personal computer. Laptops, netbooks, iPads, tablet computers, eBook readers, and smartphones fall somewhere in between. At what point do you stop licensing Java SE and start licensing Java ME instead?
Let's take a look at all the stuff my new Samsung Vibrant can do, and let you decide for yourself. I have 140 applications installed, which I can access alphabetically. I also have up to seven screens which I can fill with application icons and widgets to simplify access. The screen measures about 4 inches diagonally. Click on each image below to see the full 480x800 resolution.
Each screen has five rows. On my first screen, I have the first two rows related to photography. This includes a camera, camcorder, bar-code scanner and visual search engine (Google Goggles). I am not happy with Flickr Droid app in uploading photos, so I might need to find another app for that. Other reviews I read complain that the Vibrant's camera does not have am LED flash for night time shots, and that there is no forward facing camera to do Skype or FaceTime-style videoconferencing. I think it is fine the way it is. An interesting feature of the camera app is that it uses the volume up/down buttons to zoom in and out.
The next two rows related to books and documents. In addition to both Amazon's Kindle and Barnes and Noble's Nook eBook readers, I have Dropbox to make it easy to transfer files between all my machines, a camera-scanner that generates PDFs, and ThinkFree, which appears to be based on OpenOffice open source software to create, view and edit WORD documents, EXCEL spreadsheets and PowerPoint presentations.
My second screen is for music and video entertainment.
The top row is consumed by a single widget for [Pandora], an internet radio station, not to be confused with the Pandora moon that the movie Avatar is based on. I-heart-radio, Slacker, and Last.fm are other internet radio stations. Be careful when roaming in another country, as the $15-per-MB transfer fees can really add up. While the Galaxy S has a built-in FM radio, T-Mobile has decided to disable this feature in its Vibrant model, in favor of internet-based radio stations.
I am glad the Samsung Vibrant uses the same 3.5mm combo audio jack that I mentioned in my blog post about my
[New ThinkPad T410]. This allows me to use the same headset for both my laptop and my cell phone.
For those who use Microsoft Windows Media Player v10 or above, this phone lets you transfer over your songs, playlists and videos via the USB cable in PMC mode. The TED application shows 18-minute videos of lectures at conferences that focus on Technology, Entertainment and Design. MobiTV offers live streaming of popular Television shows, normally ten dollars monthly, but I got a free 30-day trial in the deal.
Screen 3 is focused on travel. I have a 30-day free trial of GoGo, the new Wi-Fi networks on various airlines. Hopefully, I will get to try this out on my upcoming flights. When GoGo is not available, the Extended Controls widget allows me to turn the phone into "Airplane mode", which would allow me to read eBooks and listen to pre-recorded music and videos stored on my phone. Most of the apps on Android are free, but Extended Controls, shown here in the top row, cost me money but well worth it. With this you can customize different size widgets with all the appropriate setting toggles you want. On this one, I can toggle Wi-Fi, Data transfer, GPS positioning, and Airplane mode.
Google Maps, Google Places and Google Sky Map are all well represented here. I also like TripIt, which is a free Software-as-a-Service for managing your trip itenerary, and syncs up with their online website. Currency and Language translation can help on international travel. The standard Alarm Clock also includes Time Zone conversion as well.
My screen 4 is my central home page. There are four buttons on the bottom of the phone: Menu, Home, Back, and Search. Hit the "Home" button on any screen, and it jumps immediately to Screen 4. From here, I can get to any of the other screens with just swiping my finger across the surface. Therefore, I chose to keep this screen simple.
For meetings, I have a big clock, and an Extended Controls widget to set my phone on silent/vibrate mode, and show my battery status. I put icons here for apps that I might need in a hurry, like Camera, Evernote, or Shazam. For those not familiar with Shazam, it will listen to the microphone for whatever song is playing in the background where you are, and it will identify the song's title and artist.
The "Starred" folder lists those five or so contacts that I have marked with a "star" to be on this short list. From here, I can call or send them an SMS text message.
Screen 5 is for office productivity. I have a 2x2 widget from Astrid to list my to-do items. I have a 1x2 widget showing my last call. My calendar syncs up with my Google calendar online.
The Locale widget allows me to change which on-screen keyboard to use. There is the standard Android keyboard which allows voice-to-text input, the Samsung keyboard that offers [XT9 mode], and the new ["Swype"] keyboard that allows you to write words quickly with squiggles swiped across the keyboard. The Swype is incredible accurate when I am typing in English. When I am communicating in Spanish, it gets in the way, spell-checking when it shouldn't.
Screen 6 is for my social media, news and search facilities. I have HootSuite Lite for managing my Twitter and Facebook posts. For news junkies, NPR, USA Today and CNN all offer mobile versions.
I have a selection of browsers, including Opera Mini 5, and Dolphin Browser HD. The latter offers a variety of special add-ons similar to Firefox on a desktop system. I also have specialty search sites, including the Internet Movie Database (IMDB), Fandango for local movie times, and Dex for local phone listings.
Screen 7 is for system administration. The top row is another "Extended Controls" widget, this time to change between 2G and 3G networks, brightness setting, set the the time-out interval for when the screen should automatically shut off, and a "stay awake" to turn off the screen saver altogether.
I can do some really powerful things here. For example, I have an application to let me use secure shell (ssh) to access our systems at work. I also can "tether" my laptop to my Vibrant, for those few times when Wi-Fi is not available, to let my laptop use the phone's signal as a dial-up modem. It is slower than Wi-Fi, but might be just what I need in a pinch.
The bottom row is the same across all seven screens, which you can customize. I left the bottom row in its original default, with options to make phone calls, look up contacts, and send text messages. The bottom right corner launches a list of all applications alphabetically, to access those not on my seven main screens.
Just in case I switch to a local SIM card while abroad in another country, I asked T-mobile to unlock my phone, which they happily did at no additional charge. For example, while I am in Australia, I can either leave my T-Mobile USA chip in the phone, and pay roaming charges per minute, or I can purchase a SIM chip from a local phone company with pre-paid minutes. This often includes unlimited free incoming calls to a local Australian phone number, and voicemail.
Unlocking the phone to use different SIM cards is different than "jailbreaking", a term that refers to Apple's products. For Android phones, jailbreaking is called "rooting", as the process involves getting "root" user access that you normally don't have. The only reason I have found to have my phone "rooted" was to take these lovely screen shots, using the "Screen Shot It" application. This is another application that I paid for. I used the free trial for a few screenshots first to check it out, liked the results, and bought the application.
So, this new smartphone looks like a keeper. I got a screen protector to avoid scratching, and a two-piece case that snaps around the phone to give it more heft. All my chargers are "Mini USB" for my old G1 phone, and this new Vibrant phone is "Micro USB" instead, so I had to order new ones for my car, my office, and for my iGo (tip A97).
This review is more to focus on the fact that the IT industry is changing, and what was traditionally performed on personal computers are now being done on new handheld devices. Android provides a platform for innovation and healthy competition. Let's all hope Oracle and Google can work out their differences amicably.
The "Storage Resource Agent" introduced for Linux, AIX and Windows in v4.1 is a lightweight agent, written in native "C" language instead of Java, to avoid all the resources that Java consumes. In this release, it is now supported for HP-UX and Solaris, and adds file level and database level storage resource management (SRM) reporting for all five platforms.
For new customer deployments, this will eliminate all the pain setting up a "Common Agent Manager". The Productivity Center server will send out the agent, the agent collects the data, and can then optionally uninstall itself. In this manner, you always have the latest version of the code collecting the data. For those with Common Agent Manager already installed, you can continue running as is, or slowly transition over to the new lightweight agent methodology.
Full support for IBM XIV Storage System
IBM XIV® Storage System support updated to include provisioning, data path explorer and performance management reporting. Before this release, Productivity Center could only discover and provide rudimentary capacity information for XIV systems. Now you can carve LUNs and monitor XIV disk performance just like you can with most other disk systems.
Storage Area Network (SAN) configuration planning
For those who have both Productivity Center Standard Edition (SE) and Productivity Center for Replication, the SAN Config Planner is now "replication-aware" and will add LUNs to existing copy sessions, or create new copy sessions, and ensure that the devices chosen meet the appropriate criteria.
HyperSwap™ for the IBM AIX® environment
On z/OS mainframes, if you experience an outage on a storage system, Productivity Center for Replication (TPC-R) can automatically swap to the synchronous mirror copy without disruption to the operating system or application. Now, IBM has extended this awesome feature to the AIX platform for high availability in POWER-based server environments.
Detailed Session Reporting for Global Mirror
Before, TPC-R enforced the notion of only one Global Mirror master per storage system. Now, TPC-R v4.2 is capable of supporting multiple Global MIrror sessions, and provide more detailed session reporting for these environments. This can be useful if for some unknown reason the bits are not being shoveled from point A to point B, and you need to do some "problem determination".
SVC Incremental FlashCopy
Productivity Center for Replication now adds support for the "Incremental" feature of SVC FlashCopy. While FlashCopy requests are processed instantaneously, there is background processing required that can consume cycles. Incremental processing keeps track of what changed since the last FlashCopy, and minimizes this behind-the-scenes overhead.
Integrated Distributed Disaster Recovery manager
IBM Tivoli System Automation Application Manager [TSA-AM] can now integrate with TPC-R to provide application-aware disaster recovery capability. This can coordinate between IBM Tivoli System Automation for Multiplatforms [TSA-for-MP], IBM HACMP/PowerHA, as well as other clustering products like Microsoft Cluster Services (MSCS) and Veritas Cluster Services on Solaris. When TSA-AM detects an outage, it can notify Globally Dispersed Parallet Sysplex Distributed Cluster Management (GDPS-DCM) to take action. This integration was actually completed with TPC v4.1 back in April, but got buried deep inside our big storage launch, so I bring it up again as a gentle reminder that IBM offers the best end-to-end management on the planet.
At last month's Storage University, I presented an overview of [Tivoli Storage Productivity Center v4.1]. Many of the questions were along the lines of "When will TPC do xyz?" and all I could answer was "Soon" since I knew they would be delivered with this TPC v4.2 release, but I couldn't provide any more details than that at the time.
I am just one of the speakers. We will have at each location the local IBM team and IBM clients giving testimonials. All the speakers will be available afterward for Q&A. It's shaping up to be an exciting series of events!