Modified by TonyPearson
Back in 1986, when I first started with IBM, my first job was working on a software product called Data Facility Hierarchical Storage Manager (DFHSM). This did "Information Lifecycle Management" (ILM) by moving data sets from one storage tier to another. (The phrase "Information Lifecycle Management" was coined by StorageTek in 1991, and later resurrected by EMC a few years ago. As is often typical, things that appear new to the distributed systems crowd, are often well-established concepts in the mainframe arena).
To help explain DFHSM and its sister product Data Facility Data Set Services (DFDSS), an enterprising sales rep in Los Angeles named C.D. Larsen made a video called "Re-arranging the sock drawer". He explained that sometimes you want the socks you wear the most on the top drawer, and socks that you only wear now and again in lower drawers. DFHSM can re-arrange your sock drawer based on policy-based automation, determining which ones you wear most often, and moving the others down the "hierarchy" accordingly.
To explain DFDSS, he pulled out an entire drawer of socks, and move it to another level. DFDSS was able to do volume-level backups and dumps to tape very quickly, since it did not process individual data sets, but rather the entire volume image as a whole. These two products are now DFSMShsm and DFSMSdss components of the DFSMS element of the z/OS operating system.
Mainframes use an interesting naming convention for its data sets. 44 characters, divided up into qualifiers that are 1-8 characters long, separated by periods. For example:
The first qualifier indicated it belonged to me, that it was for my Project A, that it was a testcase, and specifically TEST1 job control language. Arranging them in this order meant that I could easily find all the data needed for project A, but if I wanted to keep all the testcase data together, I might have put that as the second qualifer instead.
On Linux, UNIX and Windows, most people are more familiar with hierarchical file systems, so the same file might be stored as:
Same concept. You set up a taxonomy of they way you want to organize your data, so that related data can be grouped together and easier to manage. Whereas we used to tell customers that "Qualifiers are your friend", we now tell people "sub-directories are your friend". This is true when organizing the files on your laptop, in your Lotus Notes, and in Second Life.
Since starting Second Life last November, I have picked up all kinds of free things along the way, and now have thousands of objects in my "inventory". Basically, its like keeping things in your pocket, when you want it, you just take it out of your pocket, and *poof* it appears magically on the ground. I was having a hard time finding things in my inventory, so I decided to re-arrange with sub-folders. This is done in-world, and I found it best to do this away from other avatars asking "what are you doing?" which can get quite annoying. Find a remote island or the rooftop of some building when doing "house cleaning".
I've arranged my main folders as follows. These all appear on a single screen, and makes it easy to find exactly what I am looking for.
- Body Parts
- Calling Cards
- Lost and Found
- Photo Album
In Second Life, you can make complete "outfits" which include your body shape, skin, eyes, hair, and clothes. However, saving away many outfits means duplicating a lot of items. Therefore, I separated them out. I keep body shape, skin, eyes and hair in the folder "Body Parts" and all of the clothing items under "Clothing". Under clothing, I separated everything out into the major categories:
I could have a separate folder for "socks", but I keep those in the "shoes" folder.
technorati tags: IBM, DFHSM, DFDSS, DFSMS, z/OS, qualifiers, taxonomy, Second Life, inventory
Well, I'm back from Mexico.
The flight back was uneventful, except for the leg from Houston to Tucson. The lady in the window seat had "overallocated storage" and required a "distance extension" on her safety belt. To accomodate her, her husband and I flipped up the "logical partitions" between the seats, and "compressed" to take up less space to accomodate. Luckily, it was only for two hours.
On the flight to Houston, I was asked what kind of drink I wanted, in Spanish, as the crew were all from Mexico. Here's a quick Spanish lesson:
- this stands for drink in general, and can include liquor and soft drinks
- this stands generically for soft drink. They will often use "Coke" to refer to any cola beverage, regardless of brand.
It is interesting that Spanish language in each country is slightly different. The Mexicans I met with and spoke Spanish to immediately recognized I was from South America, and not from Central America. Likewise, folks in Puerto Rico knew I was from somewhere from South America, and not from Mexico or Central America. In Columbia, Argentina, and even Brazil, my speech is more recognizable as being from Bolivia.
Before IBM got into an OEM agreement with Network Appliance, I used to indicate that EMC and NetApp were the "Coke and Pepsi" of the NAS marketplace. IBM had a presence, but it was in the single digits, whereas these two major players had roughly equal marketshare, just as Coke and Pepsi dominate equally the US marketplace. That analogy doesn't work in other countries, as in some cases the country might be more heavily in favor of one or the other.
On my flight over from Houston to Tucson, however, I was asked what kind of "pop" I wanted. I always say "soda" to refer generically to soft drinks, but realize that others say "pop" instead. Not only can Americans be able to detect what part of the country people are from by accent, but also by the words they use.
Now I see a blog that explores in great detail the issue of Pop vs Soda vs Coke.
So, it looks like I'll need to "retire" my Coke vs. Pepsi analogy, not because their marketshare has changed, but because IBM's parntering with NetApp greatly skews the advantage over EMC.
technorati tags: IBM, Mexico, NAS, OEM, NetApp, EMC, Coke, Pepsi, Bolivia, Pop, Soda
Today, I went looking for reading-glasses. Unfamiliar with my surroundings, I asked several people where I might be able to find and purchase these, and was sent in various directions. My first stop was a bookstore. It would make sense that since many people need reading glasses to read the books, that they would sell them there, but no. The staff didn't know where I could go, but pointed me in the direction of a mall. At the mall, I found a pharmacy. Many pharmacies sell reading glasses, so I stopped in, but no, not this one. The pharmacists suggested the super-store nearby. I walked in to the super-store, and asked the first employee where they keep their reading glasses, and they said the other corner. The other corner was the electronics department. It made sense that they sold CDs and DVDs in the same section as the equipment that plays them, but reading glasses? Skeptical, I went to the pharmacy department, and the young and beautiful lady (everyone is young, thin and beautiful here) had me follow her, and she led me back to the electronics department, whereupon she pointed to a rack of sunglasses. I indicated that I need reading glasses, not sunglasses. She pulled one out, and it was indeed reading glasses, 1.25, just what I was looking for. Others were tinted, so you can read the newspaper out in the sunlight. The pair I chose cost only $97 in the local currency.
After reading the last sentence, you might be thinking I am describing my "avatar" in Second Life, but no, I am talking about my search for reading glasses on the streets of Mexico. I am here this week in meetings with IBM Business Partners and sales reps to discuss IBM's latest System Storage products and offerings.
We used to tell people they should "clothe" servers with storage. IBM offers both, so yes it makes sense to offer both as part of a complete solution. However, when you look through a dictionary definition "to clothe" you learn it is to dress, wrap or cover with clothing, an implication that it is external, and perhaps temporary, easily changed, like switching from sunglasses to reading glasses. In Second Life, objects can be "worn", simply by attaching or detaching them to your "avatar". Sometimes clothing serves a purpose, like reading glasses, provides protection, like raincoats, and other times, more decorative, like"icing on the cake" or "gold plating".
This concept was fine 50 years ago, when we were in a server-centric world, and dumb storage devices were attached to very intelligent servers. Back then, we used the derogatory term "subsystems" to emphasize that storage was just part of the server, not a system of its own.
Today, we live in an information-centric world. The information outlives the media, and the media outlives the servers that access it. It is not unreasonable to attach dozens or hundreds of servers to a single storage system, or collection of storage systems. Over 20 percent of IBM System Storage DS8000 series, for example, are attached to Windows rack-optimized or blade servers. Imagine a refrigerator surrounded by dozens or hundreds of pizza boxes. Storage is no longer a subsystem, but a system on its own right, dressed, wrapped or covered by servers that deliver the right information, to the right people, at the right time.
So perhaps we should reverse it, telling people they should "clothe" their storage with servers!
technorati tags: IBM, disk, storage, servers, Second Life, clothing, DS8000, Turbo, subsystem, system
I didn't really have a theme this week, still recovering from jet-lag from my travels through Japan, Australia, China.
Gary Diskman has an amusing blog entry about a Funny disaster recovery job posting. It is not clear if he is being completely tongue-in-cheek, or a bit cynical. However, it rings true that you get what you measure, and some managers look for easy metrics, even if there are unintended consequences.
Western medicine works this way. Rather than paying your doctor to keep you healthy, you pay him per visit, to get refills on prescriptions, check-ups on medical conditions, surgeries and so on. While Eastern medicine is focused on keeping people healthy, Western medicine profits more from resolving "situations".
I have seen similar situations on the "health" of the data center. In one case, the admins were measured on how quickly they bring back up their web-servers after a crash. They had this process down to a science, because they were measured on how quickly they resolved the situation. I suggested switching from Windows to Linux, a much more reliable operating system for web-serving, and showed examples of web-servers running Linux that have been up for 1000 days or more. Management changed the metrics to "average up-time in days" and magically the re-boots all but disappeared, thanks to Linux, but also thanks in part to shifting the incentive structure. Perhaps some of those earlier situations were "artificially created"?
Back in the 1980s, I was working on a small software project that was about 5000 lines of code. In those days, testers were measured by the number of "successful" testcases that ran without incident. Testcases that uncovered an error were labeled as "failures" to be re-run after the developers fixed the code. When I declared my code ready for test, the test team ran 110 testcases, all successfully, and they were all rewarded for meeting their schedule. I, on the other hand, did not accept these results, met with them and told them I would give them $100 each if they could find a bug in my code in the next week. Nobody writes 5000 lines of code without some error along the way, not even me. (As one author put it, more people have left earth's gravity to orbit the planet than have written perfect code that did not require subsequent review or testing. It's so true. Good software is difficult to write.)
The test team accepted the challenge, and found 6 problems, more than I expected, but at least I felt more confident of the code quality after fixing these. As I suspected, the unintended consequence of counting "successful" testcases was that testers would write the most simple, basic, least-likely-to-challenge-boundaries testcases to ensure they meet their numbers. My experiment was costly to me, but more importantly was a wake-up call for the test management, and they realized they needed to re-evaluate their test procedures, metrics and terminology. This was a long time ago, and I am glad to see that the overall "software engineering" practice has matured much over the past 20 years.
So, my advice is to determine metrics that have the intended consequences you want, while avoiding any negative unintented consequences that might undermine your eventual success. People will quickly figure out how to maximize the results, and if you can align their goals to company goals, then everybody benefits.
Well, I'll be blogging from Mexico next week (yes, it is a business trip!). Enjoy the weekend.
technorati tags: IBM, Business Continuity, Disaster Recovery, Gary Diskman, indentives, metrics, testcases, unintented consequences
I am still wiping the coffee off my computer screen, inadvertently sprayed when I took a sip while reading HDS' uber-blogger Hu Yoshida's post on storage virtualization andvendor lock-in
. This blog appears to be the text version of theirfunny video
While most of the post is accurate and well-stated, two opinions particular caught my eye. I'll be nice and call them opinions, since these are blogs, and always subject to interpretation. I'll put quotes around them so that people will correctly relate these to Hu, and not me.
"Storage virtualization can only be done in a storage controller. Currently Hitachi is the only vendor to provide this."
-- Hu Yoshida
Hu, I enjoy all of your blog entries, but you should know better. HDS is fairly new-comer to the storage virtualization arena, so since IBM has been doing this for decades, I will bring you and the rest of the readers up to speed. I am not starting a blog-fight, just want to provide some additional information for clients to consider when making choices in the marketplace.
First, let's clarify the terminology. I will use 'storage' in the broad sense, including anything that can hold 1's and 0's, including memory, spinning disk media, and plastic tape media. These all have different mechanisms and access methods, based on their physical geometry and characteristics. The concept of 'virtualization' is any technology that makes one set of resources look like another set of resources with more preferable characteristics, and this applies to storage as well as servers and networks. Finally, 'storage controller' is any device with the intelligence to talk to a server and handle its read and write requests.
Second, let's take a look at all the different flavors of storage virtualization that IBM has developed over the past 30 years.
IBM introduces the S/370 with the OS/VS1 operating system. "VS" here refers to virtual storage, and in this case internal server memory was swapped out to physical disk. Using a table mapping, disk was made to look like an extension of main memory.
IBM introduces the IBM 3850 Mass Storage System (MSS). Until this time, programs that ran on mainframes had to be acutely aware of the device types being written, as each device type had different block, track and cylinder sizes, so a program written for one device type would have to be modified to work with a different device type. The MSS was able to take four 3350 disks, and a lot of tapes, and make them look like older 3330 disks, since most programs were still written for the 3330 format. The MSS was a way to deliver new 3350 disk to a 3330-oriented ecosystem, and greatly reduce the cost by handling tape on the back end. The table mapping was one virtual 3330 disk (100 MB) to two physical tapes (50 MB each). Back then, all of the mainframe disk systems had separate controllers. The 3850 used a 3831 controller that talked to the servers.
IBM invents Redundant Array of Independent Disk (RAID) technology. The table mapping is one or more virtual "Logical Units" (or "LUNs") to two or more physical disks. Data is striped, mirrored and paritied across the physical drives, making the LUNs look and feel like disks, but with faster performance and higher reliability than the physical drives they were mapped to. RAID could be implemented in the server as software, on top or embedded into the operating system, in the host bus adapter, or on the controller itself. The vendor that provided the RAID software or HBA did not have to be the same as the vendor that provided the disk, so in a sense, this avoided "vendor lock-in".Today, RAID is almost always done in the external storage controller.
IBM introduces the Personal Computer. One of the features of DOS is the ability to make a "RAM drive". This is technology that runs in the operating system to make internal memory look and feel like an external drive letter. Applications that already knew how to read and write to drive letters could work unmodified with these new RAM drives. This had the advantage that the files would be erased when the system was turned off, so it was perfect for temporary files. Of course, other operating systems today have this feature, UNIX has a /tmp directory in memory, and z/OS uses VIO storage pools.
This is important, as memory would be made to look like disk externally, as "cache", in the 1990s.
IBM AIX v3 introduces Logical Volume Manager (LVM). LVM maps the LUNs from external RAID controllers into virtual disks inside the UNIX server. The mapping can combine the capacity of multiple physical LUNs into a large internal volume. This was all done by software within the server, completely independent of the storage vendor, so again no lock-in.
IBM introduces the Virtual Tape Server (VTS). This was a disk array that emulated a tape library. A mapping of virtual tapes to physical tapes was done to allow full utilization of larger and larger tape cartridges. While many people today mistakenly equate "storage virtualization" with "disk virtualization", in reality it can be implemented on other forms of storage. The disk array was referred to as the "Tape Volume Cache". By using disk, the VTS could mount an empty "scratch" tape instantaneously, since no physical tape had to be mounted for this purpose.
Contradicting its "tape is dead" mantra, EMC later developed its CLARiiON disk library that emulates a virtual tape library (VTL).
IBM introduces the SAN Volume Controller. It involves mapping virtual disks to manage disks that could be from different frames from different vendors. Like other controllers, the SVC has multiple processors and cache memory, with the intelligence to talk to servers, and is similar in functionality to the controller components you might find inside monolithic "controller+disk" configurations like the IBM DS8300, EMC Symmetrix, or HDS TagmaStore USP. SVC can map the virtual disk to physical disk one-for-one in "image mode", as HDS does, or can also map virtual disks across physical managed disks, using a similar mapping table, to provide advantages like performance improvement through striping. You can take any virtual disk out of the SVC system simply by migrating it back to "image mode" and disconnecting the LUN from management. Again, no vendor lock-in.
The HDS USP and NSC can run as regular disk systems without virtualization, or the virtualization can be enabled to allow external disks from other vendors. HDS usually counts all USP and NSC sold, but never mention what percentage these have external disks attached in virtualization mode. Either they don't track this, or too embarrassed to publish the number. (My guess: single digit percentage).
Few people remember that IBM also introduced virtualization in both controller+disk and SAN switch form factors. The controller+disk version was called "SAN Integration Server", but people didn't like the "vendor lock-in" having to buy the internal disk from IBM. They preferred having it all external disk, with plenty of vendor choices. This is perhaps why Hitachi now offers a disk-less version of the NSC 55, in an attempt to be more like IBM's SVC.
IBM also had introduced the IBM SVC for Cisco 9000 blade. Our clients didn't want to upgrade their SAN switch networking gear just to get the benefits of disk virtualization. Perhaps this is the same reason EMC has done so poorly with its "Invista" offering.
So, bottom line, storage virtualization can, and has, been delivered in the operating system software, in the server's host bus adapter, inside SAN switches, and in storage controllers. It can be delivered anywhere in the path between application and physical media. Today, the two major vendors that provide disk virtualization "in the storage controller" are IBM and HDS, and the three major vendors that provide tape virtualization "in the storage controller" are IBM, Sun/STK, and EMC. All of these involve a mapping of logical to physical resources. Hitachi uses a one-for-one mapping, whereas IBM additionally offers more sophisticated mappings as well.
technorati tags: IBM, disk, tape, storage, virtualization, Hu Yoshia, HDS, Hitachi, TagmaStore, USP, NSC, disk-less, SAN Volume Controller, LVM, AIX, RAID, SAN, blade, Sun, STK, Cisco, EMC, Invista,