For those of us in the northern hemisphere, yesterday was this year's Winter Solstice
, representingthe shortest amount of daylight between sunrise and sunset. So today, I thought I would blog on my thoughtsof managing scarcity.
Earlier in my career, I had the pleasure to serve as "administrative assistant" to Nora Denzel for the week at a storage conference. My job was to make her look good at the conference, which if you know Nora, doesn't take much. Later, she left IBM to work at HP, and I gotto hear her speak at a conference, and the one thing that I remember most was her statement that thewhole point of "management" was to manage scarcity, as in not enough money in the budget,not enough people to implement change, or not enough resources to accomplish a task.(Nora, I have no idea where you are today, so if you are reading this, send me a note).
Of course, the flip-side to this is that resources that are in abundance are generallytaken for granted. Priorities are focused on what is most scarce. Let's examine some of theresources involved in an IT storage environment:
- Capacity - while everyone complains that they are "running out of space", the truth is that most external disk attached to Linux, UNIX, or Windows systems contain only 20-40% data. Many years ago, I visitedan insurance company to talk about a new product called IBM Tivoli Storage Manager. This company had 7TB of disk on their mainframe,and another 7TB of disk scattered on various UNIX and Windows machines. In the room were TWO storage admins for
the mainframe, and 45 storage admins for the distributed systems. My first question was "why so many people forthe mainframe, certainly one of you could manage all of it yourself, perhaps on Wednesday afternoons?" Their response was that they acted as eachother's backup, in case one goes on vacation for two weeks. My follow-up question to the rest of the audience was:"When was the last time you took two weeks vacation?" Mainframes fill their disk and tape storage comfortablyat over 80-90% full of data, primarily because they have a more mature, robust set of management software, likeDFSMS.
- Labor - by this I mean skilled labor able to manage storage for a corporation. Some companies I have visitedkeep their new-hires off production systems for the first two years, working only on test or development systemsonly until then. Of course, labor is more expensive in some countries than others. Last year, I was doing a whiteboard session on-site for a client in China, and the last dry-erase pen ran out of ink. I asked for another pen, and they instead sent someone to go re-fill it. I asked wouldn't it be cheaper just to buy another pen, and they said "No, labor is cheap, but ink is expensive." Despite this, China does complain that there is a shortage of askilled IT labor force, so if you are looking for a job, start learning Mandarin.
- Power and Cooling - Most data centers are located on raised floors, with large trunks of electrical power and hugeair conditioning systems to deal with all the heat generated from each machine. I have visited the data centers ofclients that are forced now to make decisions on storage based on power and cooling consumption, because the coststo upgrade their aging buildings are too high. Leading the charge is IBM, with technology advancements in chips, cards, and complete systems that use less power, and generate less heat. While energy is still fairly cheap in the grand scheme of things, fears ofGlobal Warmingand declining oil supplies, the costs ofpower and cooling have gotten some news lately. In 1956, Hubbert predicted US would reach peak oil supplies by1965-1970 (it happened in 1971), and this year Simmonsestimated that world-wide oil production began its decline already in 2005. Smart companies like Google have movedtheir server farms to places like Oregon in the Pacific Northwest for cheaper hydroelectric power.
- Bandwidth - Last year IBM introduced 4Gbps Fibre Channel and FICON SAN networking gear, along with the servers and storage needed to complete the solution. 4Gbps equates to about 400 MB/sec in data throughput. By comparison, iSCSI is typically run on 1Gbps Ethernet, but has so much overheads that you only get abour 80 MB/sec. Next year, we may see both 8 Gbps SAN, and 10 GbE iSCSI, to provide 800 MB/sec throughputs. My experience is that the SAN is not the bottleneck, instead people run out of bandwidth at the server or storage end first. They may not have a million dollars to buy the fastest IBM System p5 servers, or may not have enough host adapters at the storage system end.
- Floorspace - I end with floorspace because it reminds me that many "shortages" are temporary or artificially created. Floorspace is only in short supply because you don't want to knock down a wall, or build a new building, to handle your additional storage requirements.In 1997, Tihamer Toth-Fejel wrote an article for the National Space Society newsletter that estimated that ...Everybody on Earth could live comfortably in the USA on only 15% of our land area, with a population density between that of Chicago and San Francisco. Using agricultural yields attained widely now, the rest of the U.S. would be sufficient to grow enough food for everyone. The rest of the planet, 93.7% of it, would be completely empty.Of course, back in 1997 the world population was only 5.9 billion, and this year it is over 6.5 billion.
This last point brings me back to the concept of food, and I am not talking about doughnuts in the conference room, or pizza while making year-end storage upgrades. I'm talking aboutthe food you work so hard to provide for yourself and your family. The folks at Oxfam came up with a simpleanalogy. If 20 people sit down at your table, representing the world’s population:
- 3 would be served a gourmet, multi-course meal, while sitting at decorated table and a cushioned chair.
- 5 would eat rice and beans with a fork and sit on a simple cushion
- 12 would wait in line to receive a small portion of rice that they would eat with their hands while sitting on the floor.
So for those of you planning a special meal next Monday, be thankful you are one of the lucky three, and hopefulthat IBM will continue to lead the IT industry to help out the other seventeen.
Happy Winter Solstice!
technorati tags: IBM, Northern, Hemisphere, Winter, Solstice, Nora+Denzel, Oxfam, scarcity, Linux, UNIX, Windows, TSM, Tivoli+Storage+Manager, storage, admins, global+warming, climate+change, peak+oil, National+Space+Society, special, meal
Well, there's little to no chance we'll get snow in Tucson the rest of this year, so I built a snowman out in Second Life. That's my avatar on the right, andI am an eightbar specialist. Eightbar refers to our logo.
This was part of an IBM "Holiday Party" where dozens of IBMers met "in the virtual world" to participate in 3D competitions,I entered the "Build a Snowman" competition, since I am still a beginner at this. This was whatI was able to come up with in 20 minutes that we had to get it done. Why I made mine out of woodwith different colors was so that I could stand out from the crowd. Everyone else used traditionalwhite snowy textures.
Others had a more challenging "Build a Snow Globe" where you have to write scripts to get thelittle snow flakes to move around. This for the advanced builders of our group.
This is still new, emerging technology, but eventually, Second Life and other MMOs could be used to market products,that people can view from all three dimensions, talk to a technical specialist, and get all questions answered.It could be used for education, shopping around, and collaborating with others.
Anyways, I haven't heard the results, but I had fun anyways.
technorati tags: IBM, snowman, competition, Second Life, holiday party
Last week, in my posting on Toshiba's latest 1.8" drive
, Robert Pearson asks:
You may not be the right person to ask but I am asking everyone so "How do you see hybrid disk drives?"
(For the record, I am not immediately related to Robert. At onepoint, "Pearson" was the 12th most common surname in the USA, but now doesn't even make the Top 100.)
Robert, I would like to encourage you and everyone else to ask questions, don't worry if I am the wrong person to ask, asprobably I know the right person within IBM. Some people have called me the "Kevin Bacon" of Storage,as I am often less than six degrees away from the right person, having worked in IBM Storage for over 20 years.
For those not familiar with hybrid drives, there is a good write-up in Wikipedia.
Unfortunately, most of the people I would consult on this question, such as those from Market Intelligence or Research, are on vacation for the holidays, so, Robert, I will have to rely on my trusted 78-card Tarot deck and answer you with a five-card throw.
- Your first card, Robert, is the Hermit. This card represents "introspection". The best I/O is no I/O, which means that if applications can keep the information they need inside server memory, you can avoid the bus bandwidth limitations to going to external storage devices. Where external storage makes sense is when data is shared between servers, or when the single server is limited to a set amount of internal memory. So, consider maxing out the memory in your server first (IBM would be glad to sell you more internal memory!!!), then consider outside solid-state or hybrid devices. Windows for example has an architectural limit of 4GB.
- Your second card, Robert, is the Four of Cups, representing "apathy".On the card, you see three cups together, with the fourth cup being delivered from a cloud. This reminds me thatwe have three storage tiers already (memory,disk,tape), and introducing a fourth tier into the mix may not garnermuch excitement. For the mainframe, IBM introduced a Solid-State Device, call the Coupling Facility, which can be accessed from multipleSystem z servers. It is used heavily by DFSMS and DB2 to hold shared information. However, given some customer's apathytowards Information Lifecycle Management which includes "tiered storage", introducing yet another tier that forcespeople to decide what data goes where may be another challenge.
- Your third card, Robert, is the Chariot, which represents "Speed, Determination,and Will". In some cases, solid state disk are faster for reading, but can be slower for writing. In the case of ahybrid drive, where the memory acts as a front-end cache, read-hits would be faster, but read-misses might be slower.While the idea of stopping the drives during inactivity will reduce power consumption, spinning up and slowing downthe disk may incur additional performance penalties. At the time of this post, the fastest disk system remains the IBM SAN Volume Controller, based on SPC-1 and SPC-2 benchmarks in excess of those published for other devices.
- Your fourth card, Robert, is the Eight of Pentacles, which represents"Diligence, Hard work". The pentacles are coins with five-sided stars on them, and this often represents money.Our research team has projected that spinning disk will continue to be a viable and profitable storage media for at least anothereight years.
- Your fifth and last card, Robert, is the World, which normallyrepresents "Accomplishment", but since it is turned upside down, the meaning is reversed to "Limitation". Some Hybriddisks, and some types of solid state memory in general, do have limitations in the number of write cycles they can handle. For thoseunhappy with the frequency and slowness for rebuilds on SATA disk may find similar problems with hybrid drives.For that reason, businesses may not trust using hybrid drives for their busiest, mission-critical applications, but certainlymight use it for archive data with lower write-cycle requirements.
The tarot cards are never wrong, but certainly interpretations of the cards can be.
technorati tags: Robert Pearson, Kevin Bacon, IBM, storage, Tarot, card, deck, Hermit, Four-of-Cups, Coupling Facility, Chariot, SAN Volume Controller, SVC, SPC-1, SPC-2, benchmarks, Texas Memory Systems, Eight-of-Pentacles, World, Hybrid, SATA
I got an interesting email from a new blogger asking me for advice on how frequently to post entries.I am probably not the right person to ask, as I blog whenever a thought comes to mind that I think otherswould enjoy reading, and sometimes that means several times a day, and other times only a few per month.I actually have a day job, busy doing other things, and blogging is just now part of my general set of activities.My focus is quality not quantity.
With that in mind, I was delightfully surprised that this blog was ranked among theTop 10 Storage Blogs by Network World, which explains my recent spike in traffic.
I shared the news with my 72-year-old father, and he exclaimed "There are actually 10 or more blogsto cover the IT storage industry?" He couldn't understand why the world would read more than two or three. I personally track thirty-five of them, and I suspect there are hundredsothers out there. Of these, some blog quite regularly, while others do not, so I am in good company. Deni Connor, the author who selected these top 10, gave a nice general complement tothe entire list of blogs:
The blogs written by storage company executives can be surprisingly vendor-agnostic, though the analysts and consultants still tend to pull fewer punches.
And this was my goal as well, to enlighten and entertain, in a fair and balanced manner, that adds value to the blogosphere, rather than just repeat the IBM press releases of each day. If you are just looking for "announcements" there is an RSS feed for IBM System Storage you cansubscribe to.
Not surprisingly, two of the blog entries that Deni mentions are the ones I get the most comments on:
- ILM for my iPod tried to explain Information Lifecycle Management (ILM) into laymen terms that everyone could understand. As an engineer-turned-marketeer, explaining technology and concepts into laymen terms is something I find myself doing a lot to help others grasp what is otherwise rather complex industry we are in. Not surprisingly, many IBMers were not aware they were eligible for discounts on Apple products like the iPod, and thanked me for pointing them to this.
- Aperi is "Viagra" for SMI-S which has now become my infamous blog entry within the halls of IBM. I chose this term over "steroids" given the various scandals involving famous athletes that were going on at the time. To this day,if you search Google for "Tony Pearson" AND "Viagra" you get this blog entry at the top of the list. Oneco-worker overheard that I had "used Viagra" only to later find out they were referring to the fact that I "used Viagra as a metaphor in the title of a blog entry". And that was the real issue, not that I used the term in a popular vernacular that might not translate well into other languages, or that I failed to attribute this as a trademark that belonged to its respective manufacturer, but that it was in the title itself, and thus the URL became "aperi_is_viagra_for_smi" when published in newspapers and press releases. I have since learned to be more careful when phrasing the titles of my blog entries.
I began my year-end vacation today, but like exercising at the gym, I will try to keep up with my blogging over these next two weeks. Especially for those readers out there doing end-of-year storage infrastructure changes. This blog is for you.[Read More
On his "Data Storage - Dullness becomes Mainstream" blog, Chris Evans is
amazed athow low they can go!
.He compares the latest 100GB Toshiba 1.8" drive designed for portable music players, to the size andweight of older technology, like the IBM 3380 Direct Access Storage Device (DASD).
Chris couldn't find the dimensions of the 3380, so I thought I would provide the missing detail.The IBM 3380 History Archivesprovides a nice summary:
- The CJ2 model that Chris mentions was announced September 1, 1987 and shipped in 1988. Earlier models of the 3380 were announced 1980-1986.
- Capacity and performance were measured in 7-bit "characters", since we were not yet storing full 8-bit bytes.
- By today's standards, having such a large box to hold a few GB might seem amusing, but at the time, this unit was four times the capacity as its predecessor, the IBM 3350 DASD. Compare that with our first disk system, the IBM 350 Disk Storage Unit, introduced in 1956, that stored only 5 million characters (5MB) and was the size of two refrigerators.
- The term "DASD", pronounced daz-dee, was used as some earlier devices were based on magnetic drums or strips of magnetic tape. Today, DASD is still a common term for disk systems among mainframe administrators.
- The 3380 was also twice as fast as the IBM 3350, at 3 million characters per second (3 MB/sec). The irony was thatthe mainframe servers could not keep up, so a Speed Matching Buffer feature was invented to slow it down to half-speed, when used with certain models of mainframe.
As for the dimensions, I too had a hard time finding a publicly available resource that listed 3380 dimensions,so I searched internal IBM resources, and finally, asked someone over in the next building just to measure one ofthe 3380K models we still have in the Tucson test lab floor. The dimensions are ... (drumroll please)
- 70 inches (1778mm) tall
- 44 inches (1117mm) wide
- 32 inches (812mm) deep
The result is that the box could actually hold a much more impressive 52,500 of the new Toshiba drives, twicethe original, albeit conservative, estimate. Before anyone"tries this at home", however, keep in mind that around each Toshiba drive,as with any ATA drive, you need to have all the electronics to communicate to the outside world, and provide cooling. Running tens of thousands of these little guys in the spaceof 60 square feet would probably melt the floor or set off your smoke alarm system.
At least take a backup first.
technorati tags: Chris Evans, Toshiba, IBM, 3380, DASD, CJ2, 3350, ATA