For those in the US, last friday, the day after Thanksgiving, marks the official start of the Holiday shopping season. This has been called [Black Friday
] as some stores open as early as 4am in the morning, when it is still dark outside, to offer special discount prices. Some shoppers camp out in sleeping bags and lawn chairs in front of stores overnight to be the first to get in.
Not surprisingly, some folks don't care for this approach to shopping, and prefer instead shopping online. Since 2005, the Monday after Thanksgiving (yesterday) has been called [Cyber Monday].USA Today newspaper reports [Cyber Monday really clicks with customers]. Many of the major online shopping websites indicated a 37 percent increase in sales yesterday over last year's Cyber Monday.
On Deadline dispels the hype on both counts:[Cyber Monday: Don't Believe the Hype?"], indicating that Black Friday is not the peak shopping for bricks-and-mortar shops, andthat Cyber Monday is not the busiest online shopping day of the year, either.
Despite the controversy, all of this increased use of the internet could lead to what is now being termed an "Internet Brown-out" in the next few years.Magaret Rouse of [IT Knowledge Exchange] points to this MacWorld article by Grant Gross titled [Study: Internet could run out of capacity in two years]. Here's an excerpt:
A flood of new video and other Web content could overwhelm the Internet by 2010 unless backbone providers invest up to US$137 billion in new capacity, more than double what service providers plan to invest, according to the study, by Nemertes Research Group, an independent analysis firm. In North America alone, backbone investments of $42 billion to $55 billion will be needed in the next three to five years to keep up with demand, Nemertes said.
Internet users will create 161 exabytes of new data this year, and this exaflood is a positive development for Internet users and businesses, IIA says.
If the "161 Exabytes" figure sounds familiar, it is probably from the IDC Whitepaper [The Expanding Digital Universe] that estimated the 161 Exabytes created, captured or replicated in 2006 will increase six-fold to 988 Exabytes by the year 2010. This is not just video captured for YouTube by internet users, but also corporate data captured by employees, and all of the many replicated copies. The IDC whitepaper was based on an earlier University of California Berkeley's often-cited 2003[How Much Info?] study, which not only looked at magnetic storage (disk and tape), but also optical, film, print, and transmissions over the air like TV and Radio.
A key difference was that while UC Berkeley focused on newly created information, the IDC study focused on digitized versions of this information, and included theadded impact of replication.It is not unusual for a large corporate databases to be replicated many times over. This is done for business continuity, disaster recovery, decision support systems, data mining, application testing, and IT administrator training. Companies often also make two or three copies of backups or archives on tape or optical media, to storethem in separate locations.
Likewise, it should be no surprise that internet companies maintain multiple copies of data to improve performance.How fast a search engine can deliver a list of matches can be a competitive advantage. Content providers may offer the same information translated into several languages.Many people replicate their personal and corporate email onto their local hard drives, to improve access performance, as well as to work offline.
The big question is whether we can assume that an increased amount of information created, captured and replicated will have a direct linear relation to the growth of what is transmitted over the internet. Three fourths of the U.S. internet users watched an average of 158 minutes of online video in May 2007, is this also expected to grow six-fold by 2010? That would be fifteen hours a month, at current video densities, or more likely it would be the same 158 minutes but of much higher quality video.
On the other hand, much of what is transmitted is never stored, or stored for only very short periods of time.Some of these transmissions are live broadcasts, you are either their to watch and listen to them when they happen, or you are not. Online video games are a good example. The internet can be used to allow multiple players to participate in real time, but much of this is never stored long-term. An interesting feature of the Xbox 360 is to allow you to replay "highlight" videos of the game just played, but I do not know if these can be stored away or transferred to longer term storage.
Of course, there will always be people who will save whatever they can get their hands on. Wired Magazine has anarticle [Downloading Is a Packrat's Dream], explaining that many [traditional packrats] are now also "digital packrats", and this might account for some of this growth. If you think you might be a digital packrat,Zen Habits offers a [3-step Cure].
In any case, the trends for both increased storage demand, and increased transmission bandwidth requirements, are definitely being felt. Hopefully, the infrastructure required will be there when needed.
technorati tags: Thanksgiving, Christmas, Black Friday, Cyber Monday, MacWorld, Nemertes, IDC, whitepaper, UC Berkeley, How Much Info, study, Xbox 360, video, YouTube
Continuing this week's theme on the z10 EC mainframe being able to perform the workloadof hundreds or thousands of small 2-way x86 servers, I offer a simple analogy.
- One car, one driver
If you wonder why so many companies subscribe to the notion that you should only runa single application per server, blame Sun, who I think helped promote this idea.Not to be out-done, Microsoft, HP and Dell think that it is a great idea too. Imaginethe convenience for operators to be able to switch off a single machine and impactonly a single application. Imagine how much this simplifies new application development,knowing that you are the only workload on a set of dedicated resources.
This is analogous to a single car, single driver, where the car helps get the personfrom "point A" to "point B" and the single driver represents the driver and solepassenger of the vehicle. If this were a single driver on a energy-efficient motorcycleor scooter, than would be reasonable, but people often drive alone much bigger vehicles,what Jeff Savit would call "over-provisioning". Chips have increased in processingpower much faster than individual applications have increased their requirements, so as a result,you have over-provisioning.
- Carpooling - one bus, one driver, and many other passengers riding along
This is how z/OS operates. Yes, you could have up to 60 LPARs that you could individuallyturn on and off, but where z/OS gets most of its advantages is that you can run many applicationsin a single OS instance, through the use of "Address Spaces" which act as application containers.Of course, it is more difficult to write for this environment, because you have to be a good"z/OS citizen", share resources nicely, and be WLM-compliant to allow your application to beswapped out for others.
While you get efficiencies with this approach, when you bring the OS down, all the apps on that OS image haveto stop with it. For those who have "Parallel Sysplex" that is not an issue. For example, let's say youhave three mainframes, each running several LPARs of z/OS, and your various z/OS images all are able toprocess incoming transactions for a common shared DB2 database. Thanks to DB2 sharing technology, youcould take down an individual LPAR or z/OS image, and not disrupt transaction processing, because theIP spreader just sends them to the remaining LPARs. A "Coupling Facility" allows for smooth operationsif any of the OS images are lost from an unexpected disaster or disruption.
Needless to say, IBM does not give each z/OS developer his or her own mainframe. Instead, we get to run z/OS guest images under z/VM. It was even possible to emulate the next generation S/390 chipsetto allow us to test software on hardware that hasn't been created yet. With HiperSockets, we canhave virtual TCP/IP LAN connections between images, have virtual coupling facilities, have virtualdisk and virtual tape, and so on. It made development and test that much more efficient, which iswhy z/OS is recognized as one of the most rock-solid bullet-proof operating systems in existence.
The negatives of carpooling or taking the bus applies here as well. I have been on buses that havestopped working, and 50 people are stranded. And you don't need more than two people to make thelogistics of most carpools complicated. This feeds the fear that people want to have separatemanageable units one-car-one-driver than putting all of their eggs into one basket, having to scheduleoutages together, and so on.
(Disclaimer: From 1986 to 2001 I helped the development of z/OS and Linux on System z. Mostof my 17 patents are from that time of my career!)
- Bicycle races and Marathons
The third computing model is the Supercomputer. Here we take a lot of one-way and two-way machines,and lash them together to form an incredible machine able to perform mathematical computations fasterthan any mainframe. The supercomputer that IBM built for Los Alamos National Laboratory just clockedin at 1,000,000,000,000,000 floating point operations per second. This is not a single operating system,but rather each machine runs its own OS, is given its primary objective, and tries to get it done.NetworkWorld has a nice article on this titled:[IBM, Los Alamos smash petaflop barrier, triple supercomputer speed record].If every person in the world was armed with a handheld calculator and performed one calculation per second, it would take us 46 years collectively to do everything this supercomputer can do in one day.
I originally thought of bicycle races as an analogy for this, but having listened to Lance Armstrong at the[IBM Pulse 2008] conference, I learned thatbiking is a team sport, and I wanted something that had the "every-man-for-himself" approach to computing.So, I changed this to marathons.
The marathon was named after a fabled greek soldier was sent as messenger from the [Battle of Marathon to the City of Athens],a distance that is now standardized to 26 miles and 385 yards, or 42.195 kilometers for my readersoutside the United States.
If you were given the task to get thousands of people from "point A" to "point B" 26 plus milesaway, would you chose thousands of cars, each with a lone driver? Conferences with a lot of people in a few hotels useshuttle buses instead. A few drivers, a few buses, and you can get thousands of people from a fewplaces to a few places. But the workloads that are sent to supercomputers have a single end point,so a dispatcher node gives a message to each "greek soldier" compute node, and has them run it on their own. Somemake it, some don't, but for a supercomputer that is OK. When the message is delivered, the calculation for thatlittle piece is done, and the compute node gives it another message to process. All of the computations areassembled to come up with the final result. Applications must be coded very speciallyto be able to handle this approach, but for the ones that are, amazing things happen.
So, how does "server virtualization" come into play?
- Logical Partitions
IBM has had Logical Partitions for quite some time. A logical partition, or LPAR, can run its own OSimage, and can be turned on and off without impacting other LPARs. LPARs can have dedicated resources,or shared resources with other LPARs. The IBM z10 EC can have up to 60 LPARs. System p and System i,now merged into the new "POWER Systems" product line, also support LPARs in this manner. Depending onthe size of your LPAR, this could be for a single OS and application, or a single OS with lots of applications.
- Address Spaces/Application Containers
This is the bus approach. You have a single OS, and that is shared by a set of application containers. z/OS does this with address spaces, all running under a single z/OS image, and for x86there are products like [Parallels Virtuozzo Containers] that can run hundred of Windows instances under a single Windows OS image, or a hundred Linux imagesunder a single Linux OS image. However, you cannot mix and match Windows with Linux, just as all theaddress spaces on z/OS all have to be coded for the same z/OS level on the LPAR they run in.
- Virtual Guests
The term "guests" were chosen to model this after the way hotels are organized. Each guest has a roomwith its own lockable entrance and privacy, but shared lobby, and in some countries, shared bathroomson every hall. This approach is used by z/VM, VMware and others. The z/VM operating system can handle any S/390-chip operating system guest, so you could have a mix ofz/OS, TPF, z/VSE, Linux and OpenSolaris, and even other z/VM levels running as guests. Many z/VM developers runin this "second level" mode to develop new versions of the z/VM operating system!
As part of the One Laptop Per Child [OLPC] development team (yes, I ama member of their open source community, and now have developer keys to provide contributions), I havebeen experimenting with Linux KVM. This was [folded into the base Linux 2.6.20 kernel and availableto run Linux and Windows guest images. This is a nice write-up on[Wikipedia].
The key advantage of this approach is that you are back to one-car-one-driver simplistic mode of thinking. Each guest can be turned on and off without impacting otherapplications. Each guest has its own OS image, so you can mix different OS on the same server hardware.You can have your own customized kernel modules, levels of Java, etc.Externally, it looks like you are running dozens of applications on a single server, but internally,each application thinks it is the only one running on its own OS. This gives you simpler codingmodel to base your test and development with.
Jeff is correct that running less than 10 percent utilization average across your servers is a cryingshame, and that it could be managed in a manner that raises the utilization of the servers so that fewer areneeeded. Just as people could carpool, or could take the bus to work, it just doesn't happen, and data centersare full of single-application servers.
VMware has an architectural limit of 128 guests per machine, and IBM is able to reach this withits beefiest System x3850 M2 servers, but most of the x86 machines from HP, Dell and Sun are less powerful,and only run a dozen or so guests. In all cases, fewer servers means it is simpler to manage, so moreapplications per server is always the goal in mind.
VMware can soak up 30 to 40 percent of the cycles, meaning the most you can get from a VMware-basedsolution is 60 to 70 percent CPU utilization (which is still much better than the typical 5 to 10 percent average utilization we see today!) z/VM has been finely tuned to incur as little as 7 percent overhead,so IBM can achieve up to 93 percent utilization.
Jeff argues that since many of the z/OS technologies that allow customers to get over90 percent utilization don't apply to Linux guests under z/VM, then all of the numbers are wrong.My point is that there are two ways to achieve 90 percent utilization on the mainframe, one is throughz/OS running many applications on a single LPAR (the application container approach), and the other through z/VM supporting many Linux OS images, each with one (or a few) applications (the virtual guest approach).
I am still gathering more research on this topic, so I will try to have it ready later this week.
technorati tags: IBM, z10, EC, mainframe, Jeff Savit, Sun, Microsoft, HP, Dell, car, driver, bus, carpool, marathon, z/OS, LPAR, z/VM, guest, address space, application containers, ibmpulse, pulse2008, VMware, x3850
Yesterday (September 7, 2006) the Eclipse Foundation
announced that it has approved the creation of the Aperi Storage Management Framework Project.
There's been a lot of confusion out there about Aperi, so I thought I would post some facts and opinions about this exciting new project. A few years ago, I was thelead architect for IBM TotalStorage Productivity Center, IBM's infrastructure management product that helped launch the creation of Aperi.
From the latin word for "open", Aperi is an open source project that aims to simplify the management of storage environments, using the Storage Management Initiative - Specification (SMI-S) open standardto promote interoperability and eliminate complexity in today’s storage environments.
Aperi should provide immediate value upon install with basic storage management capabilities, rather than just simply a collection of components that require costly integration. We've discussed requirements for functions such as:
- Resource discovery, monitoring, and reporting
- Fabric Topology mapping
- Event management
- Disk / Tape management
- Device configuration & LUN assignment
- SAN fabric management
- Basic asset management
- User Interface
The big confusion most people have is Aperi's relation to SMI-S and the Storage Networking Industry Association (SNIA)open standards group. The best way to explain this is to go backto your High School SAT college-entrance exams. Remember questions like this?
CRUMB : BREAD ::
(A) splinter : wood
(B) water : bucket
(C) cream : butter
(The answer: a crumb is to bread like a splinter is to wood.)
Aperi is an implementation of SMI-S standard, similar to MySQL or PostgreSQL areopen standard relational database implementations of Structured Query Language (SQL).These compete with proprietary database implementations such as IBM DB2 Universal Database,Oracle Database, Microsoft SQL Server, or Sybase.
Aperi: SMI-S ::
PostgreSQL : Structured Query Language (SQL)
It is often the case that the folks writing the code are different than the folks defining the standards. This is the case between the members of Aperi writing code, and the members of the SNIA writing standards. IBM happens tohave employees writing Aperi code, and other employees helping define SMI-S standards.What can I say, IBM is a big company and a leader in many areas.
A good analogyis how the Apache community has developed an awesome web server, and the Firefox Mozillacommunity have developed an awesome web browser, both of which are implementations of the HTTP/HTML standards adopted by the World Wide Web Consortium. Apache and Firefoxcompete with proprietary implementations, such as Microsoft Internet Information Services(IIS) web server and Internet Explorer web browser.
Aperi: SNIA ::
Apache : World Wide Web (WWW) Consortium
With this arrangement, Aperi and the SNIA will have very complementary roles in defining and driving standards across the entire storage market. To that end, Aperi will make extensive use of the SNIA’s Technology Center and SNIA’s “plugfests” to test the interoperability of the Aperi framework with the variety of 3rd-party storage offerings available. By providing a tested implementation of SMI-S, Aperi will drive broader industry availability of SMI-S, as well as offer the many benefits of an industry-backed open source community.
Check out this vote of confidence:
"Eclipse's Aperi Project will further advance the adoption of SNIA's SMI-S, benefiting the entire storage industry and IT community. Furthermore, the SNIA and Aperi will define plans to collaborate on new storage standards, standards testing programs, and storage interoperability programs."
--- Wayne M. Adams, chair, SNIA Board of directors
So, both proprietary and open source implementations have their place in the world.Proprietary products are needed for advanced, unique value-add, and opensource projects are for basic support focused on interoperability and flexibility.These can be combined, for example, proprietary "plug-ins" built on an open source base. The more choices the client has, the better.
Storage vendors benefit too. Vendors are tired of being in the "Y.A.C." business, building "Yet Another Configurator" for each new device developed, with basic functionsto carve LUNs, read performance stats, and so on. By shipping Aperi instead, storagevendors like IBM can invest their development dollars in real innovations, things thatmatter for the customer.
If you are unfamiliar withopen source, I highly recommend Eric Raymond's bookThe Cathedral and the Bazaar. It might just change the way you think aboutopen source usage in the workplace.
technorati tags: Eclipse, Aperi, IBM, TotalStorage, Productivity Center, open, storage management, Eric Raymond, open+source, SMI-S, SNIA, Cathedral, Bazaar
Chris Evans over at Storage Architect posts aboutHardware Replacement Lifecycle Update
, on how storage virtualization can helpwith storage hardware replacemement. He makes two points that I would like to comment on.
... indeed products such as USP, SVC and Invista can help in this regard. However at some stage even the virtualisation tools need replacing and the problem remains, although in a different place.
Knowing that replacement of technologies at all levels are inevitable, IBM System Storage SAN Volume Controlleris actually designed to allow cluster non-disruptive upgrade, which we announcedMay 2006.
The process is quite elegant. The SVC consists of one or more node-pairs, and can be upgraded while the systemis up and running by replacing nodes one at a time in a sequence of suspend and resume. All of the mapping tablesare loaded onto the new nodes from the rest of the still active nodes.
I was hoping as part of the USP-V announcement HDS would indicate how they intend to help customers migrate from an existing USP which is virtualising storage, but alas it didn't happen.
Unlike the SVC, once cannot just upgrade the USP in place and make it into a USP-V. While it might be possible tounplug external disk from the old USP, and re-plug into the new USP-V, what do you do about the internal disk data?I doubt you can just move drawers and trays of disk from the old to the new. The data has to be moved some other way.
Some have asked why not just put an SVC in front of both the old USP and the new USP-V and transfer the data that way.While SVC does support virtualizing the old USP device, IBM is still testing the new USP-V as a managed device, and so this solution is not yet available, and would only apply to the LUNs in the USP-V, not the volumes specifically formatted for System i or System z.
An alternative is to take advantage of IBM's Data Mobility Services, the result of our recentacquisition of SofTek. IBM can help you both mainframe and distributed systems data from any device, to any device.
In a typical four year lifecycle of storage arrays, it might take six months or so to fill up the box, and might takeas much as a year at the end to move the data out to other equipment. SVC can greatly reduce both of these, so that you can take immediate advantage of new equipment as soon as possible, and keep using it for close to the full four years,migrating weeks or days before your lease expires.
technorati tags: IBM, SVC, Chris Evans, storage, architect, disk, virtualization, Invista, USP, USP-V, data mobility services, SofTek
Web Worker Daily has an article[Bill Gates on Collaboration and Continued Learning
] discussing Bill Gates' essay [The skills you need to succeed
].Bill talks about the importance of software, and how it has turned nearly everyone into information workers.
While Bill Gates is personally benefiting from code he wrote 30 years ago, most software engineers don't getroyalties for their creative efforts. Robin Harris on StorageMojo has a great piece on [Why are the writers striking?]The writers in this case are those who write scripts for television programs. They get 4 cents for every$19.99 DVD sold today, and want this bumped up to 8 cents. More importantly, they want the same deal forcontent shown over the internet. Currently, they get nothing when content they wrote for is shown on the internet, and they would like that fixed also.
Paying royalties to creative writers encourages them to write good stuff. The best stuff will result in moreroyalties, and we want to encourage this. What about software engineers? Don't we want them to write the beststuff also? Shouldn't they get royalties too, not just a flat salary and continued employment?
Something to think about...
technorati tags: Web Worker Daily, Bill Gates, collaboration, continued learning, skills, succeed, software, information workers, Robin Harris, StorageMojo, writers, strike, royalties, software engineers
Continuing this week's theme on "best of breed", some questions arise: How is this calculated or determined?How is one storage solution "better" than another? Which attributes weigh more heavily in the decision?
Some attributes are directly measurable, like storage performance. For this, gather up a list ofall the storage products you are interested in, go to the [Storage Performance Council website],determine whether SPC-1 or SPC-2more closely matches your application workload, and then choose the best product fromthe benchmarks, discarding any vendors that don't bother to have benchmarks posted.The new SPC-2 benchmark was created, in part, to address new workloads for the Media and Entertainmentindustry. (For a comparison of the two, see my post [SPC benchmarks for Disk System Performance])
However, other attributes, like "easy to manage", are not as straightforward to measure.One client compared the complexity of different solutions by counting the number of cables involved to connect the various parts of each solution. Only external cables were considered. All of the cables inside an IBM SystemStorage DS8000 would not be counted. By this measure, a single IBM System z10 EC mainframe connected to a single IBM DS8000 disk system over a few FICON cables would therefore be "less complicated" than a thousand x86 servers connectedvia FCP SAN switches to dozens of disk systems.
I thought of this when fellow IBM blogger Alan Lepofsky posted[Lenovo x300 versusMacbook Air commercial], pointing to this:
But counting cables only handles the hardware part of the interconnections. You have to also considerthe interconnections between the software, between users, and between IT administrators. It is not alwaysobvious where those connections are, and how to count them into consideration.
This month, IBM introduced the first "Management Complexity Factor" (MCF) for the Media andEntertainment industry. IBM MCF a result of IBM's acquisition of NovusCG, and is an essential part of"Storage Optimization Services" being offered by IBM. Here is an excerpt from the[IBM Press Release]:
"Media companies are facing a double-edged sword with the exponential rise in digital media storage needs, coupled with concerns about optimizing storage to be more efficient," said Steve Canepa, vice president of Media and Entertainment, IBM. "By quickly and cost-effectively analyzing the interconnected IT and storage environments that increasingly comprise media operations, MCF for Media helps our clients identify opportunities for improvement and align their IT and business strategies."
Since 1995, IBM has invested more than $18 billion on public acquisitions, making it the most acquisitive company in the technology industry, based on volume of transactions.
IBM has a strong global focus on the media and entertainment industry across all of its services and products, serving all the major industry segments -- entertainment, publishing, information providers, media networks and advertising.
For more information on IBM, please visit [www.ibm.com/media].
technorati tags: SPC, storage, performance, benchmarks, SPC-1, SPC-2, complexity, Steve Canepa, IBM, MCF, acquisitions
As the popular idiom goes, ["Every cloud has a silver lining"
], and perhaps there are some things that we gain from the recent financial collapse of the global world markets.
Let's examine some of these:
- Consolidation and the [Economies of Scale]
As financial firms focus on costs, the IT departments will have an opportunity to consolidate their servers, networks and storage equipment. Consolidating disk and tape resources, implementing storage virtualization, and reducingenergy costs might get a boost from this crisis. Consolidating disparate storage resources to a big SoFS, XIV,DS8000 disk system, or TS3500 tape library might greatly help reduce costs.
- Establishing enterprise-wide Storage Resource Management (SRM)
Having mixed vendor environments that result from such mergers and acquisitions can be complicated to manage. Thankfully, IBM TotalStorage Productivity Centermanages both IBM and non-IBM equipment, based on open industry standards like SMI-S and WBEM.Merged companies might let go IT people with limited vendor-specific knowledge, but keep the ones familiar withcross-vendor infrastructure management skills and ITIL certification.
- Comparing different vendor equipment
It seems that often times when there is a merger or acquisition, the two companies were using different storage gear from different vendors. IBM has made some incredible improvements over the past three years, in both performance enhancements and energy efficiency, but many companies with non-IBM equipment may not be aware of them.If there was ever a time to perform a side-by-side comparison between IBM and non-IBM equipment, here isyour chance.
For more on the impact of the financial meltdown on IT, see this InfoWorld[Special Report].
technorati tags: IBM, cloud, silver lining, financial, collapse, crisis, meltdown, global, world, markets, consolidation, XIV, DS8000, SoFS, TS3500, performance, energy efficiency, SMI-S, WBEM, ITIL, TotalStorage Productivity Center, SRM, InfoWorld
In Monday's post, [IBM Information Infrastructure launches today
], I explained how this strategic initiative fit into IBM's New EnterpriseData Center vision. For you podcast fans, IBM Vice Presidents Bob Cancilla (Disk Systems), Craig Smelser (Storage and Security Software), and Mike Riegel (Information Protection Services), highlight some of the new products and offerings in this 12-minute recording:
This post will focus on Information Security, the second of the four-part series this week.
Security protects information against both internal and external threats.
For internal threats, most focus on whether person A has a "need-to-know" about information B. Most of the time, thisis fairly straightforward. However, sometimes production data is copied to support test and development efforts. Here is the typical scenario: the storage admin copies production data that contains sensitive or personal informationto a new copy and authorizes software engineers or testers full read/write access to this data.In some cases, the engineers or testers may be employees, other times they might be hired contractors from an outside firm.In any case, they may not be authorized to read this sensitive information. To solve this IBM announced the[IBM Optim Data Privacy Solution] for a variety of environments, including Siebel and SAP enterprise resource planning (ERP)applications.
I found this solution quite clever. The challenge is that production data is interrelated and typically liveinside [relational databases].For example, one record in one database might have a name and serial number, and then that serial number is used to reference a corresponding record in another database. The IBM Optim Data Privacy Solution applies a range of"masks" to transform complex data elements such as credit card numbers, email addresses and national identifiers, while retaining their contextual meaning. The masked results are fictitious, but consistent and realistic, creating a “safe sandbox” for application testing. This method can mask data from multiple interrelated applications to create a “production-like” test environment that accurately reflects end-to-end business processes.The testers get data they can use to validate their changes, and the storage admins can rest assured theyhave not exposed anyone's sensitive information.
Beyond just who has the "need-to-know", we might also be concerned with who is "qualified-to-act".Most systems today have both authentication and authorization support. Authentication determines that youare who you say you are, through the knowledge of unique userid/passwords combinations, or other credentials. Fingerprint, eye retinal scans or other biometrics look great in spy movies, but they are not yetwidely used. Instead, storage admins have to worry about dozens of different passwords on differentsystems. One of the many preview announcements made by Andy Monshaw on Monday's launch was that IBM isgoing to integrate the features of [Tivoli Access Manager for Enterprise Single Sign-On] into IBM's Productivity Center software, and be renamed "IBM Tivoli Storage Productivity Center".You enter one userid/password, and you will not have to enter the individual userid/password of all the managedstorage devices.
Once a storage admin is authenticated,they may or may not be authorized to read or act on certain information.Productivity Center offers role-based authorization, so that people can be identifiedby their roles (tape operator, storage administrator, DBA) and that would then determine what they areauthorized to see, read, or act upon.
For external threats, you need to protect data both in-flight and at-rest. In-flight deals with data thattravels over a wire, or wirelessly through the air, from source to destination. When companies have multiplebuildings, the transmissions can be encrypted at the source, and decrypted on arrival.The bigger threat is data at-rest. Hackers and cyber-thieves looking to download specific content, like personal identifiable information, financial information, and other sensitive data.
IBM was the first to deliver an encrypting tape drive, the TS1120. The encryption process is handled right at the driveitself, eliminating the burden of encryption from the host processing cycles, and eliminating the need forspecialized hardware sitting between server and storage system. Since then, we have delivered encryption onthe LTO-4 and TS1130 drives as well.
When disk drives break or are decommissioned, the data on them may still be accessible. Customers have a tough decision to make when a disk drive module (DDM) stops working:
- Send it back to the vendor or manufacturer to have it replaced, repaired or investigated, exposing potentialsensitive information.
- Keep the broken drive, forfeit any refund or free replacement, and then physically destroy the drive. Thereare dozens of videos on [YouTube.com] on different ways to do this!
The launch previewed the [IBM partnership with LSI and Seagate] to deliver encryption technology for disk drives, known as "Full Drive Encryption" or FDE.Having all data encrypted on all drives, without impacting performance, eliminates having to decide which data gets encryptedand which doesn't. With data safely encrypted, companies can now send in their broken drives for problemdetermination and replacement.Anytime you can apply a consistent solution across everything, without human intervention anddecision making, the less impact it will have. This was the driving motivation in both disk and tape driveencryption.
(Early in my IBM career, some lawyers decided we need to add a standard 'paragraph' to our copyright text in the upper comment section of our software modules, and so we had a team meeting on this. The lawyer that presented to us that perhaps only20 to 35 percent of the modules needed to be updated with this paragraph, and taught us what to look for to decidewhether or not the module needed to be changed. Myteam argued how tedious this was going to be, that this will take time to open up each module, evaluate it, and make the decision. With thousands of modules involved the process could take weeks. The fact that this was going to take us weeks did not seem to concern our lawyer one bit, it was just thecost of doing business.Finally, I asked if it would be legal to just add the standard paragraph to ALL the modules without any analysis whatsoever. The lawyer was stunned. There was no harm adding this paragraph to all the modules, he said, but that would be 3-5x more work and why would I even suggest that. Our team laughed, recognizing immediately that it was the fastest way to get it done. One quick program updated all modules that afternoon.)
To manage these keys, IBM previewed the Tivoli Key Lifecycle Manager (TKLM).This software helps automate the management of encryption keys throughout their lifecycle to help ensure that encrypted data on storage devices cannot be compromised if lost or stolen. It will apply to both disk and tapeencryption, so that one system will manage all of the encryption keys in your data center.
For those who only read the first and last paragraphs of each post, here is my recap:Information Security is intended as an end-to-end capability to protect against both internal and external threats, restricting access only to those who have a "need-to-know" or are "qualified-to-act". Security approacheslike "single sign-on" and encryption that applies to all tapes and all disks in the data center greatly simplify the deployment.
technorati tags: IBM, Robert Cancilla, Craig Smelser, Mike Riegel, Optim, Data Privacy, Siebel, SAP, ERP, need-to-know, Andy Monshaw, credentials, biometrics, role-based, authorization, authentication, Tivoli, Access Manager, Enterprise, Single Sign-On, Productivity Center, encryption, in-flight, at-rest, TS1120, LTO-4, TS1130
Yesterday, IBM announced a variety of new storage offerings. Our theme this time around was "Policies and Performance". Here's a quick recap.
- N series
IBM offers new appliance and gateway models of its popular "unified storage" IBM System Storage N series disk systems.The N5300 appliance has two models. A10 for the single-controller, and A20 for the dual-controller model. The N5600 gateway also has two models. G10 for the single-controller, and G20 for the dual-controller model.A new EXN4000 disk expansion drawer is 3U high, and can hold up to 14 disks. It can support 1Gbps, 2Gpbs and 4Gpbs speeds.In addition to all this new "performance", we offer a new "policy" called the Advanced Single Instance Storage feature for the N5000 and N7000 series, which provides de-duplication at the block level. This can be particularly useful if you are using your N series for e-mail, document publishing, databases, backups or archives.
- SAN Volume Controller
A technology refresh with the new 8G4 model. Like its predecessor, the 8F4, this new model has 8GB of cache per node, and is fitted with 4Gpbs SAN attachment ports. The difference is that the 8G4 is based on our successfulIBM System x3550 server.This baby screams, so I look forward to seeing the updated SPC-1 and SPC-2 performance benchmark ratings.The new SVC 4.2 software provides additional authentication policies for more granular administration support, andmulti-destination FlashCopy (one source copied to up to 16 destination copies at the same time).
- DS8000 Scalability
The DS8000 series now supports having third and fourth expansion frames. This was actually already available via RPQ, but now it can be directly ordered.This means that you can now hold up to half a Petabyte in a single disk system.
- Productivity Center
IBM TotalStorage Productivity Center v3.3 offers policy and performance-based guidance in configuring disk system volumes, specification of paths between hosts and disk systems during storage provisioning, policy-based specification of zone membership, configuration analysis capabilities, configuration change management, extended tape management, and both content-sensitive and scalable enterprise-wide reports. There is also a version specifically designedto manage disk replication on System z platforms.
- Deep Computing Storage
The IBM System Storage DCS9550 Storage System comes in a 4U controller and 3U disk expansion drawers. It is designed for High Performance Computing (HPC) such as genome medical research, government research and rich media applications.
Our clients tell us they need performance to meet their dynamic business demands, and policies to help them manage the ever growing size of their storage infrastructure. We listened!
technorati tags: IBM, disk, storage, system, May, 2007, announcement, N5300, N5600, Advanced Single Instance Storage, EXN4000, DS8000, SAN Volume Controller, DCS9550, TotalStorage, Productivity Center, System z, Replication, FlashCopy, SVC, policy, performance, HPC, genome, research, rich media
Stephen Colbert, of The Colbert Report
, explains the name changes in recent mergers
of the Telecommunications industry. A discussion on "changing names" and how that impacts storage seems like a good way to wrap up the week's theme on naming conventions.
Name changes are sometimes painful, but often times done for a purpose, such as to promote a family. In the US, when a man and woman marries, the woman often changes her family name to match her husband, and the kids all adopt the father's family name. I say "often" because there are times where the woman keeps her name, or adds to it in a hyphenated way. ABC News reported that a Man Fights to Take Wife's Name in Marriage. KipEsquire, a lawyer, writes about it in his blogA stitch in haste.
IT industry changes the names of products that people knew as something else. Other times, they re-use an existing name, when really it is or should be different from the original. Last year, I took on the job of helping transition from our brand "TotalStorage" to the "System Storage" product line under the new "IBM Systems" brand. I help decide what stays the same name or what changes, when it should change, and how to announce that change.
On the disk side, IBM renamed Fibre Array Storage Technology, or FAStT, which was pronounced exactly like "fast", to DS4000 series. This was a big improvement, as people couldn't seem to spell it properly, with variations like "FastT". Nor could people pronounce it properly, saying "fast-tee" instead. The advantage of "DS" is that it is both easy to spell, and easy to pronounce. The DS4000 series continues to be "fast", providing excellent performance for its midrange price category.
IBM's Enterprise Storage Server (ESS) line went from model E10, to F20, to 750 and 800. When IBM came out with its replacement, the IBM TotalStorage DS8000, some people asked why it wasn't named the ESS 900, for example. The DS8000 is quite different internally, new hardware design and implementation, but is highly compatible with the ESS line, and shares much of the same functionality from microcode. Last year, it was replaced by the IBM System Storage DS8000 Turbo. Again, newer hardware, so it was easy to justify the new name change from "TotalStorage" to "System Storage".
Renaming a product risks losing its certifications and awards. For example, IBM spent a lot of time and money getting the OS/390 operating system certified as a "UNIX" platform. When it was renamed to z/OS, IBM had to do it all over again. Learning from this experience, IBM decided not to rename the SAN Volume Controllerto a new designation like "DS5750", as it enjoys the "number one" spot on both the SPC-1 and SPC-2 performance benchmarks, and is recognized as the leader in the disk storage virtualization marketplace. Renaming this product would mean losing that collateral.
IBM's "other disk systems" the N series posed another set of challenges. The current DS line already has entry-level (DS3000), midrange (DS4000) and enterprise-class (DS6000 and DS8000) products. The OEM agreement that IBM has with Network Appliance (NetApp) resulted in a new set of entry-level, midrange, and enterprise-class products. But these didn't fit nicely into the DS3000-to-DS8000 continuum. Instead, IBM decided to go with N series, using N3000 for entry-level, N5000 for midrange, and N7000 for enterprise-class. These are different than the numbers used by NetApp for their comparable, but not identical, offerings.
On the tape side, IBM decided to name the tape drives TS1000 and TS2000 range, tape libraries and automation with a TS3000 range, and tape virtualization to the TS7000 range. A lot of tape products already had 3000 numbering that had to change to fit this new scheme. This is why IBM's popular 3592 tape drive was renamed to the TS1120. The replacement to the 3494 Virtual Tape Server was named TS7700 Virtualization Engine.
Obviously, you can't change the names of products that are currently in the field, but what about existing software with minor updates? IBM decided to leave "TotalStorage Produtivity Center" under the "TotalStorage" brand until it has a significant version upgrade. Many people say "TPC" as a convenient acronym when referring to this product, but TPC is a registered trademark of the Professional Golfers Association (PGA) to refer to its "Tournament Players Club".
How can anyone confuse "managing storage" with "playing golf"? One activity is full of frustration that takes years or decades to master, involving the need to understand a variety of equipment and techniques to use each properly to accomplish your goals; and the other is an enjoyable activity, immediately productive in front of a single pane of glass managing all of your DAS, SAN and NAS storage, from reporting on your files and databases to managing storage networks and tape libraries.
Enjoy the weekend!
technorati tags: Stephen Colbert, Colbert Report, Telecommunications industry, KipEsquire, IBM, FAStT, DS4000, DS3000, DS8000, OS/390, UNIX, z/OS, SAN Volume Controller, N series, TS1120, TS7700, TotalStorage Productivity Center, TPC, PGA, Golf
This week's theme: you don't have to "go somewhere" to do "something important."
Seth Godin has a great list of suggestions for do-it-yourself [graduate school for unemployed college students]. Here's an excerpt:
- Spend twenty hours a week running a project for a non-profit.
- Teach yourself Java, HTML, Flash, PHP and SQL. Not a little, but mastery. [Clarification: I know you can't become a master programmer of all these in a year. I used the word mastery to distinguish it from 'familiarity' which is what you get from one of those Dummies type books. I would hope you could write code that solves problems, works and is reasonably clear, not that you can program well enough to work for Joel Spolsky. Sorry if I ruffled feathers.]
- Volunteer to coach or assistant coach a kids sports team.
- Start, run and grow an online community.
- Give a speech a week to local organizations.
- Write a regular newsletter or blog about an industry you care about.
- Learn a foreign language fluently.
- Write three detailed business plans for projects in the industry you care about.
- Self-publish a book.
- Run a marathon.
In 2007, 51 percent of graduating college students could find jobs in their field, and this year it has dropped to only 20 percent. If you find yourself with some time on your hands, either recently graduated or recently unemployed, consider volunteerism.Last year, I chose to donate my time and money to an innovative project called "One Laptop per Child" [OLPC]. It was one of my [New Years Resolutions] for 2008. I was actually "recruited" by folks from the OLPC after they read my [series of blog posts] on things that can be done with their now famous green-and-white XO laptop.
The first half of the year, I spent helping "Open Learning Exchange Nepal" [OLE Nepal], a non-government organization (NGO) to help education in that country. XO laptops were provided to second and sixth graders at several schools, and my assignment was to help with the school "XS" server. This would be the server that all the laptops connect to. My blog posts on this included:
Rather than [Move to Nepal], I was able to help by building an identical XS server in Tucson, and provide support remotely. This included getting the "Mesh Antennas"to be properly recognized, having an internet filter using [DansGuardian] software, and working out backup procedures.
For the second half of the year, I was asked to mentor a college student inHyderabad, India as part of the ["Google Summer of Code"] to develop an[Educational Blogger System]on the XS server. We called it "EduBlog" and based it on the popular [Moodle] educational software platform.This was going to be tested with kids from Uruguay, but sending a serverdown to this country proved politically-challenging, so instead, I [builta server and shipped it] to a co-location facility in Pennsylvania that agreed to donate the cost and expenses needed to run the server there with full internet connection. I acted as "system admin" for the box, was able to connect remotely via SSH, while Tarun, the college student I was mentoring, developed the EduBlog software. Twice the system washacked, but I was able to restore the system remotely thanks to a multi-boot configuration that allowedme to reboot to a read-only operating system image and restore the operating system and data.
The students and teachers in Uruguay were helped locally by [Proyecto Ceibal]. We were able to translate the system into Spanish, and the project was a big success, enough to convince local government to provideXO laptops to their students to further the benefits.
In recognition of these efforts, I was awarded last week the bronze[U.S. President's Volunteer Service Award] through the USA Freedom Corps and the President's Council on Service and Civic Participation.To learn more about these projects, see my [OLPC wiki page].
technorati tags: IBM, Seth Godin, Java, HTML, PHP, SQL, LAMP, OLPC, XO laptop, XS server, Nepal, Uruguay, OLE Nepal, Proyecto Ceibal, EduBlog, Volunteer Service Award
In the post [Flowing Workflow
], the folks over at Eightbar
point to the latest 3D work being done with IBM Lotus Sametime.
IBM Sametime is IBM's instant messaging facility, which has been extended to include Voice over IP (VOIP) capability similar to Skype, and now is being developed as a launch point for 3D impromptu meetings "in-world", similar to [Second Life].
With many companies facing hard times and considering travel restrictions for face-to-faceinternal meetings, an information infrastructure that adopts this technology might be a reasonable alternative.
technorati tags: IBM, Lotus, Sametime, IM, chat, Eightbar, Skype, SecondLife, travel restrictions
A few years ago, I was the IBM portfolio manager for storage connected to BladeCenter solutions. I learned a lot about the BladeCenter design, and got to speak at various x86 server events.
My colleague John put together a nice webpage on storage for System x and BladeCenter servers:
IBM System Storage for System x
BladeCenterservers come in many flavors, including blades with Intel, AMD and POWER chipsets, and can be configured in Grid and SuperComputer configurations. Up to 14 blade servers can fit intoa single 7U-high chassis, making this twice as dense as standard 1U-high rack-mounted servers.
System x, the new "IBM Systems" name for our popular xSeries product line, support Intel and AMD chipsets. These come in both rack-mountedand tower configurations. These also are idea for clustered and SuperComputer configurations.[Read More]
Seth Godin has an interesting post titled Times a Million
.He recounts how many people determine the fuel savings of higher-mileage cars to be only $300-$900 per year,and that this is not enough to motivate the purchase of a more-efficient vehicle, such as a hybrid orelectric car. Of course, if everyone drove more efficient vehicles, the benefits "times a million" wouldbenefit everyone and the world's ecology.
When I discuss storage-related concepts, many executives mistakenly relate them to the one area of information technologythey know best: their laptop. Let's take a look at some examples:
- Information Lifecycle Management
Information Lifecycle Management (ILM) includes classifying data by business value, and then using this to determineplacement, movement or deletion. If you think about the amount of time and effort to review the files on yourindividual laptop, and to manually select and move or delete data, versus the benefits for the individual laptopowner, you would dismiss the concept. Most administrative tasks are done manually on laptops, because automatedsoftware is either unavailable or too expensive to justify for a single owner.
In medium and large size enterprises, automated software to help classify, move and delete data makes a lot of sense.Executives who decide that ILM is not for their data center, based on their experiences with their laptop, are losingout on the "times a million" effect.
- Energy Efficiency
Laptops have various controls to minimize the use of battery, and these controls are equally available when pluggedin. Many users don't bother turning off the features and functions they don't need when plugged in, because theyfeel the cost savings would only amount to pennies per day.
Times a million, energy savings do add up, and options to reduce the amount used per server, per TB of data stored, not only save millions of dollars per year, but can also postpone the need to build a new data center, or upgrade the electrical systems in your existing data center.
- Backup and Disaster Recovery planning
I am not surprised how many laptops do not have adequate backup and disaster recovery plans. When executives thinkin terms of the time and effort to backup their data, often crudely copying key files to CDrom or USB key, and worryingabout the management of those copies, which copies are the latest, and when those copies can be destroyed, theymight reject deploying appropriate backup policies for others.
Times a million, the collected data stored on laptops could easily be half of your companies emails and intellectual property. Products like IBM Tivoli Storage Manager can manage a large number of clients with a few administrators,keeping track of how many copies to keep, and how long to keep them.
So, next time you are looking at technology or solutions for your data center, don't suffer from "Laptop Mentality". Focus instead on the data center as a whole.
technorati tags: Seth Godin, IBM, storage, ILM, energy, efficiency, backup, disaster recovery, Tivoli Storage Manager, laptop
The results are finally in. IBMer Wolfgang Singer was awarded "Top Speaker" award for his NAS and iSCSI tutorial at last year's Orlando 2006 conference. Here he is receiving the awardfrom SNIA Executive Director Leo Leger.
Of course, NAS and iSCSI technologies have been around for a while, but they are still new formany customers, which is why tutorials like this are so important.
Not everyone is clear on these technologies. For example, Dave Hitz asksis iSCSI SAN or is iSCSI NAS? I Don’t Know.
To avoid this confusion, IBM adopted clarifying technology.
- When iSCSI is intermixed with NFS, CIFS and other TCP/IP messaging traffic, it's NAS.
- When iSCSI is run on its own dedicated network, for security and performance reasons, it's an IP SAN.
The advantage of IBM System Storage N series "unified storage" concept is that you can choose which method of attachment you prefer.
technorati tags: IBM, SAN, iSCSI, NAS, Wolfgang Singer, SNIA, SNW, Dave Hitz