Sometimes, it's difficult to explain the products I manage to people outside the IT storage industry. How do you explain FCP vs. FICON, Giant Magnetoresistive (GMR) heads, the SMI-S interface, etc. enough to then explain how your job relates to those technologies. At least my friends and family read this blog, so they can somewhat understand some of the things I am working on. When I visit my folks on Sundays, we sometimes discuss items they read in my blog that week.
In addition to a "take your children to work day", we have discussed within IBM a "take your parents to work day", especially for the young new hires who have a hard time explaining what their new job is to the rest of their family.
Seth Godin points to a video ad to fill a job position and the confusion therein with the "recruiter" who just doesn't understand the job involved.
The problem is not just your parents, but any of your co-workers old enough to be parents who haven't bothered to keep up with the latest advancements in Web 2.0 technology. Here are some examples:
- A project leader working with a technology partner asked if me if there was a difference between a "blog" and a "wiki" and which should his team use. This was not a simple yes/no answer, and involved some explanation, conversation and understanding of what he was trying to accomplish.
- For one of my meetings, someone instant-messaged me asking where it was, was it "face-to-face" (F2F) or Conference call (CC). I replied back, "A2A w/CC" (avatar-to-avatar with voice over conference call). When you are meeting other avatars in-world in Second Life, it gets quite distracting having everyone typing away, with their hands and fingers moving furiously, so we use a conference call to complement our 3D interaction.
That's why I was very excited to seeLinden Lab announces voice beta in Second Life. It won't be fully ready until later this year, but adding voice to Second Life will greatly reduce the hurdles we now have trying to coordinate conference calls with in-world activity.
I realize not everyone can keep up with all the new and different technologies, but the social networking aspects of some of these new developments are worth looking into.
technorati tags: IBM, blog, wiki, social networking, technology, avatar, voice, Second Life, GMR, FICON, FCP, SMI-S
Tonight I had dinner with Henry Daboub (an SVC expert from Houston, TX) and some clients, who asked what I would blog about tonight, and I figured it made sense to blog about the SVC.
Hu Yoshida clarifies his position about storage virtualization, including the statement: "As a result they can not provide the availability, scalability, and performance of a DS8300. If they could, there would be no need for a DS8300."
Of course, if humans descended from apes, why are there still apes? Now that we have cars, why are there still trains? But perhaps a better question is: now that there are supercomputers, why are there still mainframe servers?
The issue is the difference between scale-up versus scale-out. Scale-up is making a single box as big and beefy as possible. When the SVC was introduced, the major vendors all had scale-up designs: IBM ESS 800, HDS Lightning, EMC Symmetrix. Like the mainframe, they were for customers that wanted everything in a single monolithic container.
SAN Volume Controller was the result of IBM Research asking the question, if you could put anyone's software (feature and functionality) on anyone's hardware (monolithic scale-up design), what combination would you choose? What if the brains inside today's monolithic systems could be snapped into the another vendor's frame? What if you could run SRDF on an HDS box, or ShadowImage on an IBM box? The surprising response was that most customers would want a single software for consistency, but wanted the option to choose from different vendors hardware, to negotiate the best price of the commodity iron. Based on this feedback, the SVC was born.
The idea was simple, put all the brains in a separate appliance. The appliance would do the non-disruptive migrations, the caching, the striping, and all the copy services. This lets the customer chose then the hardware they want, any mix of FC and ATA disk, from any vendor.
The SVC design was based on IBM's long history in supercomputers. Using the same "scale-out" technology, the power comes not from having it all in one monolithic box, but rather in a design that combines small nodes together. While the cache is not globally shared, the data is shared between node-pairs, and the logical-to-physical mapping is routed around to all nodes in a cluster. Each SVC node talks to each other SVC node through the FCP ports, eliminating the need for additional wiring. For the most part, each node does its own separate work, but when it needs to, they can communicate across, just like nodes in a supercomputer.
Both the SVC and the DS8300 Turbo have better than 99.999 percent availability, based on redundant components designed for no single point of failure (SPOF). IBM has sold thousands of each, and they have been in the field enough time that we can make that claim. There is nothing between scale-up versus scale-out that makes on inherently more available than the other.
Both the SVC and the DS8300 Turbo can scale from as little as a few TB of disk, to hundreds of TB of disk. We have yet to meet a customer that is too big for the SVC. The DS8300 Turbo is able to scale by adding up to four extension frames, but is still considered a single box from a scale-up perspective. From a processor perspective, an 8-node SVC cluster has 16 Intel Xeon processors, and the DS8300 has 8 POWER5+ processors (dual 4-way). The key advantage of scale-out is that you can add capacity to the SVC in smaller increments. Jumping from a DS8100 (dual 2-way) to a DS8300 (dual 4-way) is a big jump.
SVC remains the fastest disk system in the industry, based on both the SPC-1 and SPC-2 benchmarks. The latest model now supports 8GB per node, for a total of 64GB for an 8-node cluster. This can be used for both read and write non-volatile storage. By comparison, DS8300 Turbo has 32GB write non-volatile storage, and up to 256 GB of read-only cache. The SVC is able to do 155,519 IOPS, faster than the 123,030 IOPS for the DS8300, and of course faster than anything from EMC, HDS, HP or Texas Memory Systems. Of course, workloads vary, and there might be some workloads where the 256GB of read-only cache of the monolithic DS8300 is the better choice.
- copy services
Both SVC and DS8300 Turbo offer FlashCopy (point-in-time copy), Metro Mirror (synchronous) and Global Mirror (asynchronous). SVC provides the additional benefit that it can perform a FlashCopy from one frame to another, and the ability to migrate data seemlessly from one box to another.
Interestingly, IBM has seen a resurgence in both mainframe sales, as well as interest in supercomputers. Both have their place, based on the workload characteristics, and so IBM will continue to offer both modular scale-out designs, as well as monolithic scale-up designs, to meet the different needs of the marketplace.
technorati tags: IBM, disk, SAN, Volume, Controller, DS8300, Turbo, Hu Yoshida, FlashCopy, Metro Mirror, Global Mirror, SPC, benchmarks, HDS, HP, EMC, mainframe, supercomputer
Modified by TonyPearson
Back in 1986, when I first started with IBM, my first job was working on a software product called Data Facility Hierarchical Storage Manager (DFHSM). This did "Information Lifecycle Management" (ILM) by moving data sets from one storage tier to another. (The phrase "Information Lifecycle Management" was coined by StorageTek in 1991, and later resurrected by EMC a few years ago. As is often typical, things that appear new to the distributed systems crowd, are often well-established concepts in the mainframe arena).
To help explain DFHSM and its sister product Data Facility Data Set Services (DFDSS), an enterprising sales rep in Los Angeles named C.D. Larsen made a video called "Re-arranging the sock drawer". He explained that sometimes you want the socks you wear the most on the top drawer, and socks that you only wear now and again in lower drawers. DFHSM can re-arrange your sock drawer based on policy-based automation, determining which ones you wear most often, and moving the others down the "hierarchy" accordingly.
To explain DFDSS, he pulled out an entire drawer of socks, and move it to another level. DFDSS was able to do volume-level backups and dumps to tape very quickly, since it did not process individual data sets, but rather the entire volume image as a whole. These two products are now DFSMShsm and DFSMSdss components of the DFSMS element of the z/OS operating system.
Mainframes use an interesting naming convention for its data sets. 44 characters, divided up into qualifiers that are 1-8 characters long, separated by periods. For example:
The first qualifier indicated it belonged to me, that it was for my Project A, that it was a testcase, and specifically TEST1 job control language. Arranging them in this order meant that I could easily find all the data needed for project A, but if I wanted to keep all the testcase data together, I might have put that as the second qualifer instead.
On Linux, UNIX and Windows, most people are more familiar with hierarchical file systems, so the same file might be stored as:
Same concept. You set up a taxonomy of they way you want to organize your data, so that related data can be grouped together and easier to manage. Whereas we used to tell customers that "Qualifiers are your friend", we now tell people "sub-directories are your friend". This is true when organizing the files on your laptop, in your Lotus Notes, and in Second Life.
Since starting Second Life last November, I have picked up all kinds of free things along the way, and now have thousands of objects in my "inventory". Basically, its like keeping things in your pocket, when you want it, you just take it out of your pocket, and *poof* it appears magically on the ground. I was having a hard time finding things in my inventory, so I decided to re-arrange with sub-folders. This is done in-world, and I found it best to do this away from other avatars asking "what are you doing?" which can get quite annoying. Find a remote island or the rooftop of some building when doing "house cleaning".
I've arranged my main folders as follows. These all appear on a single screen, and makes it easy to find exactly what I am looking for.
- Body Parts
- Calling Cards
- Lost and Found
- Photo Album
In Second Life, you can make complete "outfits" which include your body shape, skin, eyes, hair, and clothes. However, saving away many outfits means duplicating a lot of items. Therefore, I separated them out. I keep body shape, skin, eyes and hair in the folder "Body Parts" and all of the clothing items under "Clothing". Under clothing, I separated everything out into the major categories:
I could have a separate folder for "socks", but I keep those in the "shoes" folder.
technorati tags: IBM, DFHSM, DFDSS, DFSMS, z/OS, qualifiers, taxonomy, Second Life, inventory
Well, I'm back from Mexico.
The flight back was uneventful, except for the leg from Houston to Tucson. The lady in the window seat had "overallocated storage" and required a "distance extension" on her safety belt. To accomodate her, her husband and I flipped up the "logical partitions" between the seats, and "compressed" to take up less space to accomodate. Luckily, it was only for two hours.
On the flight to Houston, I was asked what kind of drink I wanted, in Spanish, as the crew were all from Mexico. Here's a quick Spanish lesson:
- this stands for drink in general, and can include liquor and soft drinks
- this stands generically for soft drink. They will often use "Coke" to refer to any cola beverage, regardless of brand.
It is interesting that Spanish language in each country is slightly different. The Mexicans I met with and spoke Spanish to immediately recognized I was from South America, and not from Central America. Likewise, folks in Puerto Rico knew I was from somewhere from South America, and not from Mexico or Central America. In Columbia, Argentina, and even Brazil, my speech is more recognizable as being from Bolivia.
Before IBM got into an OEM agreement with Network Appliance, I used to indicate that EMC and NetApp were the "Coke and Pepsi" of the NAS marketplace. IBM had a presence, but it was in the single digits, whereas these two major players had roughly equal marketshare, just as Coke and Pepsi dominate equally the US marketplace. That analogy doesn't work in other countries, as in some cases the country might be more heavily in favor of one or the other.
On my flight over from Houston to Tucson, however, I was asked what kind of "pop" I wanted. I always say "soda" to refer generically to soft drinks, but realize that others say "pop" instead. Not only can Americans be able to detect what part of the country people are from by accent, but also by the words they use.
Now I see a blog that explores in great detail the issue of Pop vs Soda vs Coke.
So, it looks like I'll need to "retire" my Coke vs. Pepsi analogy, not because their marketshare has changed, but because IBM's parntering with NetApp greatly skews the advantage over EMC.
technorati tags: IBM, Mexico, NAS, OEM, NetApp, EMC, Coke, Pepsi, Bolivia, Pop, Soda
Today, I went looking for reading-glasses. Unfamiliar with my surroundings, I asked several people where I might be able to find and purchase these, and was sent in various directions. My first stop was a bookstore. It would make sense that since many people need reading glasses to read the books, that they would sell them there, but no. The staff didn't know where I could go, but pointed me in the direction of a mall. At the mall, I found a pharmacy. Many pharmacies sell reading glasses, so I stopped in, but no, not this one. The pharmacists suggested the super-store nearby. I walked in to the super-store, and asked the first employee where they keep their reading glasses, and they said the other corner. The other corner was the electronics department. It made sense that they sold CDs and DVDs in the same section as the equipment that plays them, but reading glasses? Skeptical, I went to the pharmacy department, and the young and beautiful lady (everyone is young, thin and beautiful here) had me follow her, and she led me back to the electronics department, whereupon she pointed to a rack of sunglasses. I indicated that I need reading glasses, not sunglasses. She pulled one out, and it was indeed reading glasses, 1.25, just what I was looking for. Others were tinted, so you can read the newspaper out in the sunlight. The pair I chose cost only $97 in the local currency.
After reading the last sentence, you might be thinking I am describing my "avatar" in Second Life, but no, I am talking about my search for reading glasses on the streets of Mexico. I am here this week in meetings with IBM Business Partners and sales reps to discuss IBM's latest System Storage products and offerings.
We used to tell people they should "clothe" servers with storage. IBM offers both, so yes it makes sense to offer both as part of a complete solution. However, when you look through a dictionary definition "to clothe" you learn it is to dress, wrap or cover with clothing, an implication that it is external, and perhaps temporary, easily changed, like switching from sunglasses to reading glasses. In Second Life, objects can be "worn", simply by attaching or detaching them to your "avatar". Sometimes clothing serves a purpose, like reading glasses, provides protection, like raincoats, and other times, more decorative, like"icing on the cake" or "gold plating".
This concept was fine 50 years ago, when we were in a server-centric world, and dumb storage devices were attached to very intelligent servers. Back then, we used the derogatory term "subsystems" to emphasize that storage was just part of the server, not a system of its own.
Today, we live in an information-centric world. The information outlives the media, and the media outlives the servers that access it. It is not unreasonable to attach dozens or hundreds of servers to a single storage system, or collection of storage systems. Over 20 percent of IBM System Storage DS8000 series, for example, are attached to Windows rack-optimized or blade servers. Imagine a refrigerator surrounded by dozens or hundreds of pizza boxes. Storage is no longer a subsystem, but a system on its own right, dressed, wrapped or covered by servers that deliver the right information, to the right people, at the right time.
So perhaps we should reverse it, telling people they should "clothe" their storage with servers!
technorati tags: IBM, disk, storage, servers, Second Life, clothing, DS8000, Turbo, subsystem, system
I didn't really have a theme this week, still recovering from jet-lag from my travels through Japan, Australia, China.
Gary Diskman has an amusing blog entry about a Funny disaster recovery job posting. It is not clear if he is being completely tongue-in-cheek, or a bit cynical. However, it rings true that you get what you measure, and some managers look for easy metrics, even if there are unintended consequences.
Western medicine works this way. Rather than paying your doctor to keep you healthy, you pay him per visit, to get refills on prescriptions, check-ups on medical conditions, surgeries and so on. While Eastern medicine is focused on keeping people healthy, Western medicine profits more from resolving "situations".
I have seen similar situations on the "health" of the data center. In one case, the admins were measured on how quickly they bring back up their web-servers after a crash. They had this process down to a science, because they were measured on how quickly they resolved the situation. I suggested switching from Windows to Linux, a much more reliable operating system for web-serving, and showed examples of web-servers running Linux that have been up for 1000 days or more. Management changed the metrics to "average up-time in days" and magically the re-boots all but disappeared, thanks to Linux, but also thanks in part to shifting the incentive structure. Perhaps some of those earlier situations were "artificially created"?
Back in the 1980s, I was working on a small software project that was about 5000 lines of code. In those days, testers were measured by the number of "successful" testcases that ran without incident. Testcases that uncovered an error were labeled as "failures" to be re-run after the developers fixed the code. When I declared my code ready for test, the test team ran 110 testcases, all successfully, and they were all rewarded for meeting their schedule. I, on the other hand, did not accept these results, met with them and told them I would give them $100 each if they could find a bug in my code in the next week. Nobody writes 5000 lines of code without some error along the way, not even me. (As one author put it, more people have left earth's gravity to orbit the planet than have written perfect code that did not require subsequent review or testing. It's so true. Good software is difficult to write.)
The test team accepted the challenge, and found 6 problems, more than I expected, but at least I felt more confident of the code quality after fixing these. As I suspected, the unintended consequence of counting "successful" testcases was that testers would write the most simple, basic, least-likely-to-challenge-boundaries testcases to ensure they meet their numbers. My experiment was costly to me, but more importantly was a wake-up call for the test management, and they realized they needed to re-evaluate their test procedures, metrics and terminology. This was a long time ago, and I am glad to see that the overall "software engineering" practice has matured much over the past 20 years.
So, my advice is to determine metrics that have the intended consequences you want, while avoiding any negative unintented consequences that might undermine your eventual success. People will quickly figure out how to maximize the results, and if you can align their goals to company goals, then everybody benefits.
Well, I'll be blogging from Mexico next week (yes, it is a business trip!). Enjoy the weekend.
technorati tags: IBM, Business Continuity, Disaster Recovery, Gary Diskman, indentives, metrics, testcases, unintented consequences
I am still wiping the coffee off my computer screen, inadvertently sprayed when I took a sip while reading HDS' uber-blogger Hu Yoshida's post on storage virtualization andvendor lock-in
. This blog appears to be the text version of theirfunny video
While most of the post is accurate and well-stated, two opinions particular caught my eye. I'll be nice and call them opinions, since these are blogs, and always subject to interpretation. I'll put quotes around them so that people will correctly relate these to Hu, and not me.
"Storage virtualization can only be done in a storage controller. Currently Hitachi is the only vendor to provide this."
-- Hu Yoshida
Hu, I enjoy all of your blog entries, but you should know better. HDS is fairly new-comer to the storage virtualization arena, so since IBM has been doing this for decades, I will bring you and the rest of the readers up to speed. I am not starting a blog-fight, just want to provide some additional information for clients to consider when making choices in the marketplace.
First, let's clarify the terminology. I will use 'storage' in the broad sense, including anything that can hold 1's and 0's, including memory, spinning disk media, and plastic tape media. These all have different mechanisms and access methods, based on their physical geometry and characteristics. The concept of 'virtualization' is any technology that makes one set of resources look like another set of resources with more preferable characteristics, and this applies to storage as well as servers and networks. Finally, 'storage controller' is any device with the intelligence to talk to a server and handle its read and write requests.
Second, let's take a look at all the different flavors of storage virtualization that IBM has developed over the past 30 years.
IBM introduces the S/370 with the OS/VS1 operating system. "VS" here refers to virtual storage, and in this case internal server memory was swapped out to physical disk. Using a table mapping, disk was made to look like an extension of main memory.
IBM introduces the IBM 3850 Mass Storage System (MSS). Until this time, programs that ran on mainframes had to be acutely aware of the device types being written, as each device type had different block, track and cylinder sizes, so a program written for one device type would have to be modified to work with a different device type. The MSS was able to take four 3350 disks, and a lot of tapes, and make them look like older 3330 disks, since most programs were still written for the 3330 format. The MSS was a way to deliver new 3350 disk to a 3330-oriented ecosystem, and greatly reduce the cost by handling tape on the back end. The table mapping was one virtual 3330 disk (100 MB) to two physical tapes (50 MB each). Back then, all of the mainframe disk systems had separate controllers. The 3850 used a 3831 controller that talked to the servers.
IBM invents Redundant Array of Independent Disk (RAID) technology. The table mapping is one or more virtual "Logical Units" (or "LUNs") to two or more physical disks. Data is striped, mirrored and paritied across the physical drives, making the LUNs look and feel like disks, but with faster performance and higher reliability than the physical drives they were mapped to. RAID could be implemented in the server as software, on top or embedded into the operating system, in the host bus adapter, or on the controller itself. The vendor that provided the RAID software or HBA did not have to be the same as the vendor that provided the disk, so in a sense, this avoided "vendor lock-in".Today, RAID is almost always done in the external storage controller.
IBM introduces the Personal Computer. One of the features of DOS is the ability to make a "RAM drive". This is technology that runs in the operating system to make internal memory look and feel like an external drive letter. Applications that already knew how to read and write to drive letters could work unmodified with these new RAM drives. This had the advantage that the files would be erased when the system was turned off, so it was perfect for temporary files. Of course, other operating systems today have this feature, UNIX has a /tmp directory in memory, and z/OS uses VIO storage pools.
This is important, as memory would be made to look like disk externally, as "cache", in the 1990s.
IBM AIX v3 introduces Logical Volume Manager (LVM). LVM maps the LUNs from external RAID controllers into virtual disks inside the UNIX server. The mapping can combine the capacity of multiple physical LUNs into a large internal volume. This was all done by software within the server, completely independent of the storage vendor, so again no lock-in.
IBM introduces the Virtual Tape Server (VTS). This was a disk array that emulated a tape library. A mapping of virtual tapes to physical tapes was done to allow full utilization of larger and larger tape cartridges. While many people today mistakenly equate "storage virtualization" with "disk virtualization", in reality it can be implemented on other forms of storage. The disk array was referred to as the "Tape Volume Cache". By using disk, the VTS could mount an empty "scratch" tape instantaneously, since no physical tape had to be mounted for this purpose.
Contradicting its "tape is dead" mantra, EMC later developed its CLARiiON disk library that emulates a virtual tape library (VTL).
IBM introduces the SAN Volume Controller. It involves mapping virtual disks to manage disks that could be from different frames from different vendors. Like other controllers, the SVC has multiple processors and cache memory, with the intelligence to talk to servers, and is similar in functionality to the controller components you might find inside monolithic "controller+disk" configurations like the IBM DS8300, EMC Symmetrix, or HDS TagmaStore USP. SVC can map the virtual disk to physical disk one-for-one in "image mode", as HDS does, or can also map virtual disks across physical managed disks, using a similar mapping table, to provide advantages like performance improvement through striping. You can take any virtual disk out of the SVC system simply by migrating it back to "image mode" and disconnecting the LUN from management. Again, no vendor lock-in.
The HDS USP and NSC can run as regular disk systems without virtualization, or the virtualization can be enabled to allow external disks from other vendors. HDS usually counts all USP and NSC sold, but never mention what percentage these have external disks attached in virtualization mode. Either they don't track this, or too embarrassed to publish the number. (My guess: single digit percentage).
Few people remember that IBM also introduced virtualization in both controller+disk and SAN switch form factors. The controller+disk version was called "SAN Integration Server", but people didn't like the "vendor lock-in" having to buy the internal disk from IBM. They preferred having it all external disk, with plenty of vendor choices. This is perhaps why Hitachi now offers a disk-less version of the NSC 55, in an attempt to be more like IBM's SVC.
IBM also had introduced the IBM SVC for Cisco 9000 blade. Our clients didn't want to upgrade their SAN switch networking gear just to get the benefits of disk virtualization. Perhaps this is the same reason EMC has done so poorly with its "Invista" offering.
So, bottom line, storage virtualization can, and has, been delivered in the operating system software, in the server's host bus adapter, inside SAN switches, and in storage controllers. It can be delivered anywhere in the path between application and physical media. Today, the two major vendors that provide disk virtualization "in the storage controller" are IBM and HDS, and the three major vendors that provide tape virtualization "in the storage controller" are IBM, Sun/STK, and EMC. All of these involve a mapping of logical to physical resources. Hitachi uses a one-for-one mapping, whereas IBM additionally offers more sophisticated mappings as well.
technorati tags: IBM, disk, tape, storage, virtualization, Hu Yoshia, HDS, Hitachi, TagmaStore, USP, NSC, disk-less, SAN Volume Controller, LVM, AIX, RAID, SAN, blade, Sun, STK, Cisco, EMC, Invista,
In case you haven't noticed, IBM System Storage makes most of their announcements on Tuesdays. IBM announced a lot today, so here is a quick run-down.
- Cisco storage networking products
IBM continues to resell Cisco switches and directors, but now can offer these with a 1-year IBM warranty.
The entry-level Cisco 9124offers 8 to 24 ports. For IBM BladeCenter, IBM now offers the Cisco10-port and 20-port modules that slide into the back of the chassis, and are functionally equivalent to the 9124.The original BladeCenter came with a 16-port module with 14 internal, but only 2 external, which severely hamperedbandwidth connectivity to external storage. These new modules provide more external ports to relieve that constraint.
The midrange Cisco9200switches have two models, both with 16 fixed ports, with the option for a blade that can provide 12, 24 or 48 additional ports. The 9216A has 16 FCP ports, and the 9216i has 14 FCP ports, and 2 GbE ports to act as a router, such as toconnect to a remote location for business continuity using Metro Mirror or Global Mirror.
The enterprise-class Cisco 9500directors can support up to 528 ports.
- TS3400 Tape Library
The new TS3400library is a small entry-level size library, supporting the enterprise-class TS1120 drive, providing interoperabilitywith the larger tape libraries, with all the support for tape encryption.
In addition to Linux, Unix, and WIndows, the TS1120 can now be connected to System i servers. In the past, the only IBMtape available to System i were the LTO models. There are a lot of businesses that need to comply with government regulations that are looking for tape encryption, and now IBM has made it accessible to more clients.
- 300GB drives at 15K RPM
The DS8000 can now support new drives with 300GB capacity at 15,000 RPM (15K). These can be up to 30 percent faster than the 10,000 RPM drives for typical workloads.
IBM continues its market leadership with these new set of features and offerings!
technorati tags: IBM, SAN, Cisco, 9124, BladeCenter, warranty, 9200, 9216i, 9216A, 9500, TS3400, TS1120, LTO, DS8000, disk,tape,15K, RPM
I am back from China, and now glad to be back in the old USA. Last week, someone asked me what would it take to add a specific feature to the IBM System Storage DS8300. The what-would-it-take question is well-known among development circles informally as a "sizing" effort, or more formally as "Development Expense" estimate.
For software engineering projects, the process was simply that an architect would estimate the number of "Lines of Code" (LOC) typically represented in thousands of lines of code (KLOC). This single number would convert to another single number, "person-months", which would then translate to another single number "dollars". Once you had KLOC, the rest followed directly from a formula, average or rule-of-thumb.
More amazing is that this single number could then determine a variety of other numbers, the number of total months for the schedule, the number of developers, testers, publication writers and quality assurance team members needed, and so on. Again, these were developed using a formula, developed and based on past experience of similar projects.
Earlier in my career, I was the lead architect for DFSMS for the z/OS operating system, and later for IBM TotalStorage Productivity Center, performing these sizing efforts. A famous IBM architect, Frederick P. Brooks, wrote a now-classic book that was requiredreading when I started at IBM, which just was re-released as Mythical Man-Month: Essays in Software Engineering, 20th Anniversary Edition. In addition to sound advice, he alsooffered a formula or two that helps with these estimating tasks.
Hardware design introduces a different set of challenges. When I was getting my Masters Degree in Electrical Engineering, it took myself and four other grad students a full semester just to design a six-layer, 900 transistor silicon chip, which could only perform a single function, multiply two numbers together.At IBM, another book that I was given to read was Soul of a New Machine, documenting six hardware engineers, and six software engineers, working long hours on a tight schedule to produce a new computer for Data General.
So why do I bring this up now? IBM architects William Goddard and John Lynott are being inducted posthumously this year into the prestigious National Inventors Hall of Fame for their disk system innovation.
Under the leadership of Reynold Johnson, the team developed an air-bearing head to “float” above the disk without crashing into the disk. Imagine a fighter airplane flying full speed across the country-side at 50 feet off the ground. If you every heard the term "my disk crashed", it was originally referring to the read/write head touching the disk surface, causing terrible damage.
A uniformly flat disk surface was created by spinning the coating onto the rapidly rotating disk, leaving many wearing lab coats covered with disk liquid at waist level. Developing disk-to-disk and track-to-track access mechanisms proved more challenging, and nearly halted the project. The team, however, was adamant that this problem could be solved, and customers were increasingly asking for random access technology. The result was the "350 Disk Storage Unit" designed for the "305 RAMAC computer", which I have talked about a lot last year as part of our "50 years of disk systems innovation" celebration.
Neither Goddard nor Lynott had computing experience prior to joining IBM. Goddard was a former science teacher who briefly worked in aerospace. Lynott had been a mechanic in the Navy and later a mechanical engineer. They didn't have a nice formula based on past experience, they didn't have the benefit of Fred Brooks' advice, or the rules-of-thumb or averages now used to estimate the size of projects. They had to break new ground.
Now that's innovation!
technorati tags: IBM, DS8300, disk, KLOC, sizing, estimate, DFSMS, z/OS, TotalStorage Productivity Center, Frederick Brooks, William Goddard, John Lynott, Mythical Man-Month, Reynold Johnson, RAMAC, 305, 350,
Wrapping up my week in China, I read an article by Li Xing in the local "China Daily" about energy efficiency in buildings. She argues that it is not enough for a building to be energy-efficient on its own, but you have to consider the impact of the other buildings around. Does it reflect the sun so harshly into neighboring windows that people are forced to put up blinds and use artificial light? Does it block the sun, so that rooms that previously could be used with natural sunlight must now be artificially lit?
A similar effect happens with power and cooling in the data center. Servers and storage systems generate heat, and that heat affects all the other equipment in the data center. IBM has the most power-efficient and heat-efficient servers and storage, but that is not enough. You have to consider the heat generated by all systems that might raise overall temperature.
This is what motivated IBM to deliver the IBM Rear Door Heat eXchanger, a member of IBM's CoolBlue(tm) portfolio.
According to a press release:
Research has indicated that water can remove far more heat per volume unit than air. For example, in order to disperse 1,000 watts, with 10 degree temperature difference, only 24 gallons of water per hour is needed, while the same space would require nearly 11,475 cubic feet of air. IBM's Rear Door Heat eXchanger helps keep growing datacenters at safe temperatures, without adding AC units. The unobtrusive solution brings more cooling capacity to areas where heat is the greatest -- around racks of servers with more powerful and multiple processors.
The CoolBlue portfolio of IBM innovations includes comprehensive hardware and systems-management tools for computing environments, enabling clients to better optimize the power consumption, management and cooling of infrastructure at the system, rack and datacenter levels. The CoolBlue portfolio includes IBM PowerConfigurator, PowerExecutive, and Rear Door Heat eXchanger.
The eXchanger works on standard 42U racks, and can help clients deal with the rapid growth of rack-mounted servers and storage on their raised floor. How cool is that!
technorati tags: IBM, Rear Door Heat eXchanger, CoolBlue, power, cooling, efficiency, server, storage
In Storage Technology News, Marc Staimer makes hisSeven network storage predictions for 2007
. Let's take a closer look at each one.
- Federal Rules for Civil Procedures (FRCP) will increase adoption of unstructured data classification, email archive systems and CAS.
CAS continues to flounder, but the rest I can agree with. Regulations are being adopted world wide. Japan has its own Sarbanes-Oxley (SOX) style legislation go into effect in 2008.IBM TotalStorage Productivity Center for Data is a great tool to help classify unstructured file systems. IBM CommonStore for email supports both Microsoft Exchange and Lotus Domino, and can be connected to IBM System Storage DR550 for compliance storage.
- Unified storage systems (combined file and block storage target systems) will become increasingly attractive in 2007, because of their ease of use and simplicity.
I agree with this one also. Our sales of IBM N series in 2006 was great, and looking to continue its strong growth in 2007. The IBM N series brings together FCP, iSCSI and NAS protocols into one disk system. With the SnapLock(tm) feature, N series can store both re-writable data, as well as non-erasable, non-rewriteable data, on the same box. Combine the N series gateway on the front-end with SAN Volume Controller on the back-end, and you have an even more powerful combination.
- Distributed ROBO backup to disk will emerge as the fastest growing data protection solution in 2007.
IDC had a similar prediction for 2006. ROBO refers to "Remote Office/Branch Office", and so ROBO backup deals with how to back up data that is out in the various remote locations. Do you back it up locally? or send it to a central location?Fortunately, IBM Tivoli Storage Manager (TSM) supports both ways, and IBM has introduced small disk and tape drives and auto-loaders that can be used in smaller environments like this. I don't know whether "backup to disk" will be the fastest growing, but I certainly agree that a variety of ROBO-related issues will be of interest this year.
- 2007 will be remembered as the year iSCSI SAN took off because of the much reduced pricing for 10 Gbit iSCSI and the continued deployment of 10 Gbit iSCSI targets.
While I agree that iSCSI is important, I can't say 2007 will be remembered for anything.We have terrible memory in these things. Ask someone what year did Personal Computers (PC) take off, and they will tell you about Apple's famous 1984 commercial. Ask someone when the Internet took off, cell phones took off, etc, and I suspect most will provide widely different answers, but most likely based on their own experience.
For the longest time, I resisted getting a cell phone. I had a roll of quarters in my car, and when I needed to make a call, I stopped at the nearby pay-phone, and made the call. In 1998, pay phones disappeared. You can't find them anymore. That was the year of the cell phones took off, at least for me.
Back to iSCSI, now that you can intermix iSCSI and SAN on the same infrastructure, either through intelligent multi-protocol switches available from your local IBM rep, or through an N series gateway, you can bring iSCSI technology in slowly and gradually. Low-cost copper wiring for 10 Gbps Ethernet makes all this very practical.
Another up-and-coming technology is AoE, or ATA-over-Ethernet. Same idea as iSCSI, but taken down to the ATA level.
- CDP will emerge as an important feature on comprehensive data protection products instead of a separate managed product.
Here, CDP stands for Continuous Data Protection. While normal backups work like a point-and-shoot camera, taking a picture of the data once every midnight for example. CDP can record all the little changes like a video camera, with the option to rewind or fast-forward to a specific point in the day. IBM Tivoli CDP for Files, for example, is an excellent complement to IBM Tivoli Storage Manager.
The technology is not really new, as it has been implemented as "logs" or "journals" on databases like DB2 and Oracle, as well as business applications like SAP R/3.
The prediction here, however, relates to packaging. Will vendors "package" CDP into existing backup products, possibly as a separately priced feature, or will they leave it as a separate product that perhaps, like in IBM's case, already is well integrated.
- The VTL market growth will continue at a much reduced rate as backup products provide equivalent features directly to disk. Deduplication will extend the VTL market temporarily in 2007.
VTL here refers to Virtual Tape Library, such as IBM TS7700 or TS7510 Virtualization Engine. IBM introduced the first one in 1997, the IBM 3494 Virtual Tape Server, and we have remained number one in marketshare for virtual tape ever since. I find it amusing that people are now just looking at VTL technology to help with their Disk-to-Disk-to-Tape (D2D2T) efforts, when IBM Tivoli Storage Manager has already had the capability to backup to disk, then move to tape, since 1993.
As for deduplication, if you need the end-target box to deduplicate your backups, then perhaps you should investigatewhy you are doing this in the first place? People take full-volume backups, and keep to many copies of it, when a more sophisticated backup software like Tivoli Storage Manager can implement backup policies to avoid this with a progressive backup scheme. Or maybe you need to investigate why you store multiple copies of the same data on disk, perhaps NAS or a clustered file system like IBM General Parallel File System (GPFS) could provide you a single copy accessible to many servers instead.
The reason you don't see deduplication on the mainframe, is that DFSMS for z/OS already allows multiple servers to share a single instance of data, and has been doing so since the early 1980s. I often joke with clients at the Tucson Executive Briefing Center that you can run a business with a million data sets on the mainframe, but that there wereprobably a million files on just the laptops in the room, but few would attempt to run their business that way.
- Optical storage that looks, feels and acts like NAS and puts archive data online, will make dramatic inroads in 2007.
Marc says he's going out on a limb here, and that's good to make at least one risky prediction. IBM used to have anoptical library emulate disk, called the IBM 3995. Lack of interest and advancement in technology encouraged IBM to withdraw it. A small backlash ensued, so IBM now offers the IBM 3996 for the System p and System i clients that really, really want optical.
As for optical making data available "online", it takes about 20 seconds to load an optical cartridge, so I would consider this more "nearline" than online. Tape is still in the 40-60 second range to load and position to data, so optical is still at an advantage.
Optical eliminates the "hassles of tape"? Tape data is good for 20 years, and optical for 100 years, but nobody keeps drives around that long anyways. In general, our clients change drives every 6-8 years, and migrate the data from old to new. This is only a hassle if you didn't plan for this inevitable movement. IBM Tivoli Storage Manager, IBM System Storage Archive Manager, and the IBM System Storage DR550 all make this migration very simple and easy, and can do it with either optical or tape.
The Blue-ray vs. DVD debate will continue through 2007 in the consumer world. I don't see this being a major player in more conservative data centers where a big investment in the wrong choice could be costly, even if the price-per-TB is temporarily in-line with current tape technologies. IBM and others are investing a lot of Research and Development funding to continue the downward price curve for tape, and I'm not sure that optical can keep up that pace.
Well, that's my take. It is a sunny day here in China, and have more meetings to attend.
technorati tags: IBM, FRCP, SOX, TotalStorage, Productivity Center, Microsoft, Exchange, Lotus, Domino, DR550, SnapLock, unified storage, NAS, iSCSI, FCP, ROBO, Tivoli, Storage Manager, TSM, Ethernet, AoE, CDP, DB2, Oracle, SAP, VTL, TS7700, TS7510, GPFS, DFSMS, Optical, 3995, 3996, Blue-Ray, D2D2T,DVD
It's official! IBM System Storage TS1120 tape drive takes home the gold award, the product of the year, announced by Storage magazine.
I spent 18 hours traveling from Australia to China yesterday, and we were partially delayed due to weather, but felt that it was necessary to discuss the innovative use of encryption on this drive.
While most consider the TS1120 an "Enterprise-class" tape technology for the mainframe, it is also attachable to the smallest distributed systems running Windows, Linux, or various flavors of UNIX. Rather than limit users with an Encryption Key Manager that only ran on z/OS, IBM instead chose to implement it in Java, that can be run on anything from z/OS to Linux, Unix and Windows platforms, giving clients choice and flexibility in their deployment.
The design is quite clever and elegant. In the encryption world, there are two ways to encrypt.
- Symmetric Key
This is very fast, because it uses a single key for both encryption and decryption, and can be incorporated on a chip. The problem is that anyone with the key can read the sensitive data.
- Asymmetric Key
This is slower, but more secure, using two separate keys. The public "encryption" key takes clear data and encrypts it. Anyone can be freely given this key, as they cannot use it to decrypt any other data. The private "decryption" key is able to decrypt the data, so that one is kept secret. If two business plan to exchange lots of tapes, they can exchange their "encryption" keys to each other.
So, let's say that Green, Inc. wants to send a tape to Blue, Co. Blue has already provided its public "encryption" key to Green, so Green does the following:
- Generate a unique data key, will call it the "red key", and there is one for each tape. It is a standard AES 256-bit symmetric key that can be processed with less than one percent overhead on the tape drive. All the data is encrypted with this key.
- Store the red key on the tape. How does Green give Blue the red key? Green encrypts it with Blue's RSA 2048-bit public "encryption" key. This is stored on three places on the tape cartridge, one in memory, and the other two on the media itself.
- Sends the tape over to Blue Co.
When it arrives on the dock at Blue Co., they do the following:
- Mount the tape and decrypt the "red key" using Blue's super-secret private decryption key.
- Pass the "red key" to the tape drive, and have it read, append or re-write the tape.
If the super-secret private key is ever compromised, all you have to do is mount the tape, unlock the red key with the old private key, and re-lock the red key with a new public key. Since the red key doesn't change, the rest of the data can be left in tact. The whole process takes less than 5 minutes, compared to Sun Microsystems method, which could take 1-2 hours per cartridge, having to decrypt and re-encrypt the entire data stream.
technorati tags: IBM, tape, TS1120, encryption, gold, award, storage, magazine, Sun, AES,RSA
Well, I have left Japan, and while everyone else is enjoying the Super Bowl, I am now in Australia, at another conference.Today I had the pleasure to hear filmmakers talk about their successes, and how IBM helps the movie industry.
- Khoa Do
At one extreme was Khoa Do, independent filmmaker. After acting in movies asideMichael Caine and Billy Zane, he decided to become his own director. He started a project to help seven disadvantaged youths from a poor drug-ridden section of Sydney, by having them act in his first full-length film.Armed with only an IBM laptop and small budget, he made the film called "The Finished People" that had critical acclaim.
The film was a success, and many of the disadvantaged youths have gone on to act in other movies. In 2005, Khoa Do was named "Young Australian of the Year".
Thanks to IBM technology, filmmaking is now accessible to a wider number of aspiring wanna-be directors. It is no longer necessary to be part of a large film studio with a multi-million dollar budget to tell your story.
- Xavier Desdoigts
At the other extreme, was Xavier Desdoigts, director of technical operations at Animal Logic, the Computer Graphics (CG) arthouse that produced special effects of movies like "The Matrix", "House of Flying Dragons" and "World Trade Center". They started with producing digital effects for TV commercials, like this one forCarlton Draught Beer.
With the support of a large film studio and multi-million dollar budget, Animal Logic now boasts the 86th most powerful "Supercomputer" based on IBM BladeCenter technology, with over 4000 servers connected into a cluster, for making the movie "Happy Feet". The movie took four years to make, with over 500 people, of 27 different nationalities. It was the first CG movie made in Australia, and has been well-received by audiences worldwide.
Mr. Desdoigts gave out some interesting facts and figures about the movie:
- While visually stunning on the big screen, each frame is only 1.4 Megapixel, about the same resolution as most camera phones.
- In one scene, there are 427,086 penguins all appearing on frame.
- Mumble, the lovable lead character, is made up of over 6 million feathers.
- As many as 17 dancers were "motion-captured" to choreograph the tap-dancing and character interaction segments.
- Only one system admin was needed to manage this entire server farm. (IBM Systems Director technology makes this possible)
- The movie consumed 103 TB of disk space, backed up to 595 LTO tape cartridges.
- An estimated 17 million CPU-hours were needed for all the processing and rendering.
Rather than talking about technology for technology sake, these filmmakers showed how technology couldbe put to use, in a practical sense, to provide the world something of value.
technorati tags: IBM, filmmaker, film industry, Khoa Do, Michael Caine, Billy Zane, Xavier Desdoigts, Animal Logic, Happy Feet, Systems Director, LTO, BladeCenter
I will wrap up this week's theme on travel, conferences and Japan discussingGroundhog day
, celebratedtoday (Feb. 2) in the US.
I thought of this because there was a 2003 movie called"Lost in Translation", the title of yesterday's post. This movie is about an American actor, played by Bill Murray, coming to Tokyoto film a whisky commercial. I first saw it with my sister and father, and we musthave been the only three who have actually been to Japan, as we were laughing hysterically,while the rest of the audience was utterly confused. If you have never been to Japan, see the movie before you go, then see it again after you get back home.
Ten years earlier, Bill Murray also played the lead role in another movie called"Groundhog day".In the movie, Bill Murray's character is TV newsman "Phil Connors" who travels to a small townwhere they bring out a small groundhog. If the groundhog can see his shadow, it predictsat least six more weeks of winter. If it does not, winter will end sooner. The nextday, Phil wakes up to realize that he is re-living the same day, over and over, like a modern-day Sisyphus or Promethius. Howhe handles himself in this situation, is what makes the movie so memorable.
When I explain what I do for IBM, to people I meet at home and abroad, I get asked the same set of questions.
- Don't you get bored presenting the same presentations?
The fact is, I never give the same presentation twice. Since I focus mostly on visual informationand what I say, versus the words of text on the page, I am able to customize my presentation toeach unique audience. In much the same way that Bill Murray's character managed to do somethingfun and different each day in the movie, despite his situation.
I do pity those presenters who focus entirely on text, turning their back to the audience, and then reading verbatim what is on each page.They should read Seth Godin's Really Bad PowerPointwith advice like "Bullets are for the NRA".
Another problem are presenters who apologize because they did not develop the materials they are presenting. Sorry, bub, you present it, you own it. The only person held accountable fora bad presentation at a conference is the speaker. When I make charts for others, I expect themthem to adjust it to their own speaking styles.
As a speaker, if you inherit materials fromsomeone else, have the courage to change it, or accept the parts you can't change, and have thewisdom to know the difference.
- Don't you get tired of traveling?
At first I thought this was odd. It's like asking "Don't you get tired of doing different things and eating different foodswith different people in a different country every week?" How can anyone grow tired of variety?
As with any question, you have to go inside the mind of the person asking the question.For most people, travel is an ordeal, outside their comfort zone. They are travelingto attend a funeral, family reunion, or a theme park with spouse and kids in tow.If that is the only kind of traveling a person knows, then it is understandablewhy they might ask this question.
- Don't you get annoyed answering the same questions at conferences?
As if this only happens at conferences!
Seriously, it might be the 17th time I've heard the question asked, but might be onlythe first time the person is asking it, and my response may be the crucial "first impression"that sets the stage for later engagements.
In this case, I focus on continuous improvement. What is the best way to answer thisquestion? How could I have answered that better? How could I have phrased the answerso it will be well-remembered? Again, like Bill Murray's character in "Groundhog Day",have fun with it, take advantage of the opportunity for improvement.
Enjoy the weekend!
technorati tags: IBM, Japan, travel, conferences, really, bad, PowerPoint, Lost In Translation, Groundhog Day, Bill Murray, Seth Godin
Continuing my week's theme on travel, conferences, and Japan, I will discuss translation and interpretation.
By now, you realize that I speak some Japanese, but not enough to give a full presentation. In addition to English, I can present Spanish and Brazilian Portuguese, but am not yet comfortable doing a full hour talk in Japanese, especially when technical terminology is required.
This brings us to the differences between translation and interpretation. The former is more literal, but the latter is needed to get the spirit or essence of what is being communicated. Sometimes, the differences in languages and culture need to be taken into account to get the right meaning across.
- One phrase, different interpretation
The conference attire was listed as "Business Casual" which they use the foreign words, as it is a very foreign concept to the Japanese. In the US, Business Casual could be polo shirt and kahki pants, perhaps. In Japan, where everyone wears a dark suit, white shirt and conservative tie, "business casual" means your shirt can be blue, or have stripes. Few dressed down for the occasion; I saw mostly white shirts underneath those dark suit coats.
- One interpretation, different connotations
Working with my interpreter team, I went page by page to explain what I would say. On one page, I mentioned having "free space" to run applications. They asked if "free space" was good or bad? I was caught off-guard by this question. Americans enjoy wide open spaces, and the comforts afforded by having enough "leg room", "head room" or "elbow room".The Japanese word for this is "yoyu", which roughly translates to "leeway". However, "yoyu" also is used in the negative sense, tailored-to-fit clothing, for example, is preferred over loose-fitting off-the-rack clothing, because it has no "yoyu". Having too much "free space" can be just as bad as not enough, much like an hour presentation that ends 20 minutes too early is just as bad as one that goes 20 minutes over.
- One word, two different interpretations
In explaining the word "archive" we came up with two separate Japanese words. One was "katazukeru", and the other was "shimau".If you are clearing the dinner plates from the table after your meal, for example, it could be done for two reasons.Both words mean "to put away", but the motivation that drives this activity changes the word usage. The first reason, katazukeru, is because the table is important, you need the table to be empty or less cluttered to use it for something else, perhaps play some card game, work on arts and craft, or pay your bills. The second reason, shimau, is because the plates are important, perhaps they are your best tableware, used only for holidays or special occasions only, and you don't want to risk having them broken. As it turns out, IBM supports both senses of the word archive. We offer "space management" when the space on the table, (or disk or database), is more important, so older low-access data can be moved off to less expensive disk or tape. We also offer "data retention" where the data itself is valuable, and must be kept on WORM or non-erasable, non-rewriteable storage to meet business or government regulatory compliance.
- Sames words, different order
On many of my charts, we show on the left the entry-level models, in the center the midrange offerings, and on the right the enterprise class high-end devices. In English, I would say "Small, Medium, and Large". However, in Japan, they read from right to left, and their words "Dai, Chu, Sho" represent "Large, Medium, Small". So, the chart had the offerings on the page correctly sequenced, I just had to start on the right, and work my way to the left, from largest to smallest.
Understanding the differences in both language and culture greatly helps in communications.
technorati tags: IBM, Japanese, Business Casual, free space, yoyu, archive, katazukeru, shimau, WORM, non-erasable, non-rewriteable, entry-level, midrange, enterprise-class