On Friday November 18, 2011, IBMers around the world engaged in the worlds first group therapy session held entirely in Twitter! (well maybe not the first, and not really group therapy, but it sounds more dramatic when I put it like that).
It focused entirely on tweeting classic lines heard in day to day life at IBM, using the hashtag #stuffibmerssay. The result was an amusing out-pouring that kept growing as the day went on (and has not stopped). Karl Roche did a great summary write-up here where he captured some of the more classic stuff. Holly Neilson also wrote a nice blog post on the subject here.
You will notice many of the tweets focus on phone conferences, which are without a doubt the greatest contributor to and destroyer of, productivity in IBM. Classics such as this one came up again and again (and it's a common problem for me):
Shock horror, I am starting to question the value of my iPhone.....
Actually... I am pondering whether the iPhone is the ideal social media tool I thought it was. Don't get me wrong, I love my little iPhone... but... Should I should stop using it to follow blogs and twitter feeds? This sounds crazy. After all, the iPhone is perfect tool for both functions. I can start up Google Reader while I am on the train; while sitting on an exercise bike at the gym; while waiting for a lift; in a taxi. Its perfect for filling in time and staying productive. Twitter is just the same.... the iPhone is a perfect tool for reading content.
So whats the problem?
The problem is that while using the iPhone for this purpose I rarely interact, I never create content, I rarely contribute to content. By this I mean I almost never comment on blogs and I hardly ever twitter. This is quite simply because I hate using the keyboard. More and more I leave both twitter and feed readers to when I have time to actually interact via a real keyboard.
So I am curious... does anyone else feel the same way? Are there better devices out there for interaction? Or is it just my fat fingers and slow brain getting in the way?
Its been a busy few weeks. I just spent a week in RTP North Carolina, with the STG Education team. We ran through our first "Implementing the Storwize V7000" course in a "Teach the Teacher" format. It was a lot of fun and I met some great fellow IBMers. It gave me a great opportunity to drive the Storwize V7000 GUI and explore all the new possibilities it opens up. First up.... the GUI is fantastic. Don't be fooled by the XIV Icons, its the smarts behind what the GUI does that makes it so powerful. Its a 21st Century GUI following very strong principles of usability and simplicity. . Talking to client after client about this product, I get lots of great questions. Two questions I get asked on a regular basis about Storwize V7000 are: . 1) What is the smallest number of SSDs I can purchase? The answer is that you can purchase just one. However with one disk you don't get any RAID. So its better to buy two SSDs for a RAID1 pair. If you buy three SSDs you can form a RAID5 array. . 2) Will the Storwize V7000 enforce the creation of hot spares? The answer is that the pre-sets that the GUI offers you, will suggest the creation of spares. For every 23 array members with the same drive class on a single SAS chain which are not RAID 0 members, a single spare is created. However the GUI will also allow you great flexibility. You can specify that a smaller or larger number of spares get created. You can choose to create NO hot spares at all. You can convert a hot spare drive into a candidate drive ( a 'free' drive). You can convert a candidate drive into a hot spare drive. You can set a 'spare goal' to set a minimum number of spares that need to exist (or an event will be logged). . So what you get is a great level of flexibility. Either follow the pre-sets and get IBM best practice... or choose your own desired spare levels. If you choose to create no spares using SSDs the Storwize V7000 will use spinning disks to rebuild a failed SSD. Then when the failed SSD is replaced, the contents of that drive will be failed back.
I have previously blogged about two XIV report generation tools that you can download and start using. This is just a short update to let you know there are updated versions of both tools, plus a new one that has just been added. These tools are all on my files section at the IBM developerWorks site (where you can also find my Visios).
To sum up what these tools do:
XIV Capacity Report
This Script creates an XLS or CSV file that contains 4 very useful tabs: Systems, Pools, Hosts, Volumes. You can use this to report on your storage, find un-mapped or un-mirrored volumes, check your consumption, etc. Clients, Business Partners and Cloud providers love this nice and simple tool.
It is currently up to version 3.9 and you can find it here.
XIV Performance Report
This Script creates an XLS or CSV file that gives the same information as the XIV Top utility but for a range of days (so we are looking at historic versus current performance). You could for example see what were the most busy volumes for the past 3 days or for the previous week. You can easily spot if host HBAs are not being used or if XIV interface traffic is not being balanced.
It is currently up to version 3.9 and you can find it here.
XIV Usage Report - NEW!
This Script creates an Excel file that shows you the current and historic usage of your volumes and pools. It also gives a trend prediction that will help estimate when your pools or volumes will be full of data. This is great for trend and growth analysis.
It is currently on version 3.9 and you can find it here.
I am getting this question on a very regular basis:
"We have just upgraded to ESXi 5.0 but we cannot find the VAAI driver on the IBM Website"
The answer? There is no vendor supplied driver because no driver is needed. ESXi 5.0 uses a SCSI T10 compliant set of commands that all vendors need to support for VAAI to work.
But of course in the tradition of all answered questions, it leads to another question:
"Once I have upgraded to ESXi 5.0 how can I tell if VAAI is really working?"
The good news is that it is very easy to spot if ESXi 5.0 has detected a VAAI capable LUN. The moment a new LUN is detected by ESXi 5.0 it tries out an Atomic Test and Set command. If that works, you will see that Hardware Acceleration shows as Supported in vCenter. In the screen capture below I have three datastores, two from XIV and one from Storwize V7000, all presented to an ESXi 5.0 server. I dragged the Hardware Acceleration column over from the right hand side to help with the screen capture (in case your vCenter looks different), but you can see the Hardware Acceleration column shows each DataStore as Supported (and did so the moment the volume was detected).
Of course having seen the Hardware Acceleration Supported message only proves that Atomic Test and Set works. To confirm if XCopy (Hardware Accelerated Move) is working, on SVC or Storwize V7000 we can use the Performance monitoring panel. In the example below I first performed a storage vMotion, moving a virtual machine between two Datastores located on the same Storwize V7000 (running 188.8.131.52 firmware). I then performed a clone of the same virtual machine, where the source was on one datastore and the target was placed on another (but both located on the same Storwize V7000). What you can clearly see is that both operations (storage vMotion and cloning) generated no volume traffic, only MDisk traffic. This means that the ESXi server is doing none of the work and the storage is doing all of the work.
In you case you hear different, it's time for a few simple facts about XIV:
XIV was founded in 2002 and shipped its first product in 2005. XIV is now up to its third generation in the development process. After over 6 years, XIV is a mature and established product in the marketplace.
IBM has sold over 5000 XIVs: A number IBM is proud and happy to disclose. In fact IBM has been open and honest about sales numbers throughout the program, which speaks volumes about how pleased they are with the success of the product.
Another point about sales numbers: Compared to the XIV, the Storwize V7000 can be sold with a starting capacity of less than 1 TB. The smallest XIV Generation 2 has a starting capacity of 27 TB, while the smallest XIV Gen3 starts at 55 TB. So clearly lower entry point products like the Storwize V7000 will outsell larger Enterprise products like the XIV. The sales numbers of both products continue to be outstanding for their size and class.
There are over 2000 XIV customers, including a considerable number of reference accounts. There are 75 success stories on the IBM XIV Website, which you can checkout here.
IBM has announced the first Storage Performance Council (SPC) result with XIV, the very first on the SPC-2/E benchmark. The XIV Storage System demonstrated its ability to handle Big Data as well as providing associated energy use data. The SPC-2/E result showed that the XIV Storage System provides outstanding enterprise price-performance and Large File Processing (LFP) performance. The numbers? 8259.94 MBPS SPC-2 (LFP) Data Rate and $137.07 SPC-2 (LFP). In case these numbers don't mean much to you, they are truly outstanding, there is only one other competitor who is even in the same ball park. (Price-Performance Source: Storage Performance Council SPC-2 Benchmark Results:http://www.storageperformance.org/results/benchmark_results_spc2, Results current as of 10/20/11). (Thanks to Elizabeth Stahl for the SPC-2/E info).
For those of you with Apple iPads, you might consider dropping by the Apple Store and picking up your free IBM XIV Mobile Dashboard.
The IBM XIV Mobile Dashboard application can be used to securely monitor the performance and health of your XIV over a Wi-Fi or 3G link. Having downloaded and installed the Mobile Dashboard you will get a lovely XIV Icon:
When you start the Mobile Dashboard you will have the choice to either run in Demo Mode or to connect to an actual XIV. Demo mode can be accessed by selecting the Demo Mode option deep in the lower right hand corner. So you don't actually need an XIV to give it a test drive.
To logon to a real XIV you will need a valid username, password and IP address.
Once connected you have the choice of viewing volume performance or host performance. If you view (hold) the iPad in portrait mode you get a list of up to 27 volumes or hosts ordered by performance metrics (it defaults to ordering by IOPS). If you view the iPad in landscape mode you will get a more graphical output (as per the examples below). There are no options to perform configuration, the dashboard is intended only for monitoring. This means each panel will show the performance and redundancy state of the XIV.
The volume performance panel is shown by default. The example below shows the output when the iPad is operated in landscape mode. From this panel you can see up to 120 seconds worth of performance for a highlighted volume. Use your finger to rotate the arrow on the blue volume icon to switch the display between IOPS, bandwidth (in megabytes per second or MBps) and latency (in milliseconds or MS). The data redundancy state of the XIV is shown in the upper right hand corner (in this example it is in Full Redundancy, but it could be Rebuilding or Redistributing).
The example above shows the output when the iPad is operated in landscape mode. If you instead rotate the iPad to portrait mode, you will get a list of the performance of up to 27 of your busiest volumes.
Now swipe to the left to navigate to the Hosts panel as shown below.
From this panel you can see up to 120 seconds worth of performance for a highlighted host. Use your finger to rotate the arrow on the purple host icon to switch the display between IOPS, bandwidth (in megabytes per second or MBps) and latency (in milliseconds or MS). The data redundancy state of the XIV is shown in the upper right hand corner (in this example it is in Full Redundancy, but it could potentially also be Rebuilding or Redistributing). Swipe to the right to navigate to the Volumes panel.
The example above shows the output when the iPad is operated in landscape mode. If you instead rotate the iPad to portrait mode, you will get a list of the performance of up to 27 of your busiest hosts.
From either the volumes or the hosts panels you can log off from the mobile dashboard using the icon in the upper right hand-most corner of the display. When you log back on, the last used XIV IP address and username will be displayed (but not the password which will need to be entered again).
I can see some nice use cases here. You get a call regarding performance but you are on the road. Are there any problems with the XIV? You can quickly logon with your iPad and confirm if response times are normal and the redundancy state is Full Redundancy.
A better use case... now you can ask your manager to buy you an iPad, so you can monitor your XIV! Let me know how that goes #
SDDPCM (Subsystem Device Driver Path Control Module) is the multi-pathing plug-in for AIX 5.3 and 6.1. Customers who use IBM SVC, DS6000, DS8000 and/or ESS800 use this package to allow the operating system to handle multiple paths from OS to storage. The good news is that this plug-in is supplied free of charge. The bad news is that it is not included with AIX fixpacks. What this means is that while you may be dilligent with keeping AIX up to date, you may miss SDDPCM in the process.
There are two good reasons to keep SDDPCM in mind when planning updates:
1) Planning an upgrade from AIX 5.3 to 6.1
Before the AIX OS is upgraded, SDDPCM must be uninstalled and then reinstalled after the upgrade. There are cases when the host attachment script must also be uninstalled and reinstalled. This is explained in the SDD Users Guide found here:
If you have already upgraded from AIX 5.3 to 6.1 but you are still using the AIX 5.3 version of SDDPCM, you may need help from IBM before you can upgrade your SDDPCM to the AIX 6.1 version. This will come in the form of some special scripts. 2) General SDDPCM maintenance
As I noted in my previous blog entry, there are quite a few SDDPCM flashes out there right now. You need to check these out and ensure you are not exposed to the issues that are corrected by later versions of SDDPCM. Check out the flashes listed here (or read my previous blog entry): http://www-01.ibm.com/support/search.wss?rs=540&tc=ST52G7&dc=D600&dtm
What about SDD (Subsystem Device Driver) for AIX? Prior to AIX 5.2 FP5, AIX did not offer native multi-pathing (MPIO). This meant that each hardware vendor had to offer their own third-party software to handle multiple paths. To achieve this with the ESS (Shark), IBM released a product called DPO (Data Path Optimiser). This product became SDD and was made available for a wide variety of operating systems.
When AIX offered MPIO, IBM then also offered a vendor plug-in (Path Contol Module) for native AIX MPIO which IBM called SDDPCM. This means you have two choices with AIX: SDD or SDDPCM. If your considering which is best, SDDPCM is my preference. This is because it is native to the operating system and also better supports the possibility of co-existence of multiple PCMs. Note that migrating from SDD to SDDPCM is not supported by the VIOS at this time, so if your running VIOS you will need to stay put for now.
Henry Ford has long been quoted as having said: "Any customer can have a car painted any colour that he wants so long as it is black."
While there is some debate on what Mr Ford exactly said, it's clear that for some time now IBM has heartily embraced this philosophy with a succession of all black machines (occasionally graced with a coloured stripe). So I was rather excited to spot something new in the IBM Melbourne demo center: an IBM Netezza (pronounced net-ease-a) It's rack door is one of the coolest IBM covers I have seen in years!
Even the internal blades look cool (I love the big N).
In case your curious, IBM® Netezza® Analytics is a purpose-built advanced analytics platform that enables your enterprise to get the most out of its data, giving you quicker answers to increasingly complex questions. It is the simple appliance for serious analytics.
Of course while I should have been thinking about big data and smart analytics, instead I have been reminiscing about IBM machines with coloured covers. For instance the IBM 3350 (storage from the 1970s) could be ordered with covers that were red... (Actually I think the correct name was garnet rose).
As far as I can tell, IBM have not offered coloured panels on Enterprise kit since June 28, 2002.
Prior to this devices could be ordered with feature codes like:
#9060 Willow green #9061 Garnet rose #9062 Sunrise yellow #9063 Classic blue #9064 Charcoal brown #9065 Pebble gray.
While it is easy to find pictures of machines with Classic Blue covers like these 3380s (with 3880 control unit)
And even visions of a red computer room (with an all white 3800 printer on the left hand side):
The only picture I have found so far that shows a yellow machine appears to have faded to orange over the years (I don't think IBM sold orange System/38s?).
I did some more digging and found this great Youtube video. You can see some old System 360 kit with red covers and at 00:46 there are some machines in custom bright yellow! The client literally ordered the machines painted with a custom tint. That takescase modding to a whole new level.
So should IBM be embracing the new cool and coming out with a bright orange XIV? How about a Storwize V7000 in fluorescent blue? A man can dream....
And if you want to see more about Netezza and it's incredibly cool rack (and even cooler architecture), check this video out:
For the next few weeks I may not blog as frequently. It's not that I am tired of writing, but I will be writing on a different subject: Helping to update the four XIV Redbooks.
IBM Redbooks are one of the many ways that IBM differentiates itself from its competitors. They are very detailed how-to guides that IBM gives away for free, you don't even need a userid to download them. IBMs customers and business partners use them extensively and IBMs competitors love them too (for different reasons).
With the General Availability date of XIV Gen3 rapidly approaching, the Redbooks of course need to be updated. We have assembled a great team of residents (which due to the timing of the residency, could not include non-IBMers) and have started our chapter updates. Some of the chapters I have already picked up include Monitoring, Volume Copies and SVC migration with XIV. I am re-learning the joys of Adobe FrameMaker and have actually had the pleasure of running I/O to an XIV Gen3 (in fact two of them).
So my only question to all of you is: What do you want to see in the next XIV Redbook? I cannot promise you that everything you ask for will appear, but here is your chance to ask. And while your thinking about it, click on the image below to like IBM Redbooks onFacebook.
The IBM Storage Management Console for VMware vCenter version 2.5.1 is now available for download and install. This version supports XIV, SVC and Storwize V7000 as per the versions on the following table (the big change being support for version 6.2):
If you want to see a video showing the capabilities of the new console, check out this link.
After installing the console, you will get this lovely new icon:
Start it up and select the option to add new storage, you now get three choices:
If your using SVC or Storwize V7000 you need to specify an SSH private key. This key MUST be in Open SSH format. This caused me a problem as I kept getting this message when trying to add my Storwize V7000 to the plug-in:
Unable to connect to 10.1.60.107. Please check your network connection, user name, and other credentials.
I could use the same IP address, userid and SSH private key to logon to the Storwize V7000 using putty, so I knew none of these things were wrong.
I reread the Installation Instructions closely and realized my mistake. It clearly states:
Important: The private SSH key must be in the OpenSSH format.
If your key is not in the OpenSSH format, you can use a certified
OpenSSH conversion utility.
I pondered what conversion utility I could use when I realized I had the utility all the time:Puttygen. I opened PuttyGen, imported my private key (the .ppk file) and exported my SSH private key using OpenSSH format. You don't need to do anything with the public key.
I was then able to add the Storwize V7000 by specifying the private SSH key exported using OpenSSH format.
Now I have both IBM XIV and Storwize V7000 in the vCenter plug-in and can get detailed information about and manipulate both. In this example I have highlighted the Storwize V7000, revealing it is on 184.108.40.206 firmware.
I was tempted to detail all the many things you can do with the plug-in, but your better off watching the video via this link.
So are you using the plug-in? Have you upgraded to version 2.5.1 yet? Comments very welcome!
Storage IT offers up many choices, some of which provoke argument so heated, you could almost describe the adherents as religious. I think you might know the sort of arguments I am talking about:
File vs Block I/O
iSCSI vs Fibre Channel
CLI vs GUI
OK.... so maybe that last one isn't quite in the same league. But it is still fascinating to see the variation in usage patterns from sites where every command (of any description) is run via a command line interface (a CLI), to sites where the CLI is viewed with either great fear... or even greater distaste. There are those who view the CLI as... well... so 1970s.....
But the reality is that the CLI will always be with us for one principal reason: scripting. If you cannot script it, you cannot automate it (well actually thats not true, but stick with me here, I am on a roll). Every single major implementation I have ever done (whether it be SVC, XIV, DS8000), I have automated with scripting. I regularly use the concatenatecommand in Excel to build large numbers of commands that I can then run as a script.
So its pleasing to see that all of our products are working towards making the scripters life even easier. For example the XIV has offered a command log in the GUI for some time. I blogged about it here. You simply do a command once in the GUI and then consult the log to find the syntax, making scripting very easy:
With last years release of SVC 6.1 and Storwize V7000, we added this level of smarts to those two products as well. Now every command you run in the GUI will offer you the exact CLI command that was used under-the-covers to do this work. Simply toggle the details tab on the completion panel to see the command (or toggle it back to hide it!).
This weeks announcement of release 6.2 of the SVC and Storwize V7000 firmware, has brought in two more important usability improvements:
Now when logging onto the CLI using individual user-ids, you can logon using the actual user-id itself, rather than admin. This change has been a long time coming and removes the confusion generated by logging onto the GUI as sayanthony, but then logging into a matching CLI session as admin. Now you would logon to either interface as anthony.
Now when issuing CLI commands, you have the choice to drop the svctask and svcinfo headers. So instead of issuing the command svcinfo lsnode, you can issue the command lsnode. Both choices remain valid (so we don't break your existing scripts). Making this change is part of a bigger plan to move to a more common CLI.
And there are more improvements coming, so as always, watch this space....
.... and please... share with me... are you a GUI... or a CLI person? Whats your reasoning behind your choice?
For someone who blogs so frequently about the IBM XIV, I will let you in on a little pet hate of mine: The XIV uses decimal volume sizes.
The XIV GUI and CLI has the user create volumes using decimal sizing, meaning 1 GB = 1,000,000,000 bytes (1000 to the power of three). Nearly every host system out there (i.e. Windows, AIX, Linux, VMware, Solaris) display volume sizes in binary, meaning 1 GiB = 1,073,741,824 bytes (1024 to the power of three).
This disparity has a quirky consequence. If the XIV says a volume is 17 GB, the host that uses that volume says it is 16 GiB (which the host often then mis-states as GB). This doesn't mean there is a loss of space, this isn't headroom or formatting - its just a different way of counting bytes. Its not a road block and its easy to understand and work with. But it is a little annoying. (Then again, so is my 32 GB iPhone reporting it has 29.3 GB of space).
The other point is that the IBM SVC, Storwize V7000, DS8000 and DS3000/DS4000/DS5000 families have always used binary sizing (even if their respective interfaces use the term GB as opposed to GiB - yet another pet hate of mine and theStorage Buddhist).
So whats the point of this rant?
The IBM XIV Storage System GUI (Version 3.0) will allow volume creation in both GiB and GB units. The IBM XIV Storage System management GUI version 3.0 will support the creation of volumes in Gigabyte (GB) or in Gibibyte (GiB) or Blocks (where each block is 512 bytes).
So this is a really good change.
The new GUI has not hit the download site yet... but I will be sure to tell you as soon as it has!
*** Update 08/09/2011 - corrected GUI version from 2.5 to 3.0, removed some confusing terms ***
After 3.5 years of reliable service, the 19" LCD monitor on my sons computer died... and
would not power back on. Warranty long since expired and replacement LCDs being
relatively cheap, I replaced his monitor with a 22" LCD and he happily updated his
Facebook status to suit.
Except there was a problem.... what to do with the dead monitor?
I had three choices:
Put it in the back shed to collect dust.
Disassemble it and shove the shredded carcase into the red bin for the weekly council rubbish collection (it being too large to just drop into the bin).
Wait for the annual council hard garbage collection and place it out the front of our property with all the other unwanted eWaste.
But there was a further problem. This lovely sticker on the back made no mention ofROHS and made even more disturbing mentions of mercury!
What to do with this monitor?
So I did the usual thing... I googled for a solution.
What I found was this site at Sustainability Victoria, which took me to this site which told me all about a program called ByteBack. One trip to Officeworks in Dandenong later and my dead LCD was off to be recycled at no charge to myself. Not only was my shed less cluttered, but I might even have helped the environment.
I would be curious to know if other people have been able to find similar programs in their locations? If so... please let know, lets spread the word!
In my last post I talked about the versions of XIV code and XIV Management GUI needed for QoS. It leads to the question of how to match the Management GUI version to the XIV code version.... which goes with which? Many storage products have management software that is separate from the software that runs inside the box (more commonly known as the firmware). So for XIV, is there a best practice? What if I have multiple XIVs.... do I need them all to be on the same firmware version? The good news is that you can simply use the highest available version of Management GUI, regardless of what code versions the XIVs are on. In other words, if you see a new version of XIV GUI on the download site.... just upgrade to it. Check out the screen capture below. This was taken with version 2.4.4 of the Management GUI (which is the latest at February 2011): The XIV on the left is running the GA XIV code, version 10.0.0.a (which came out well over two years ago). So I am able to run one of the oldest versions of XIV code, with the latest Management GUI. This is very good. But why is the right hand machine greyed out? To mix things up. the XIV on the right is running a brand new development (and thus un-released) version of code. Because I am running a (relatively) older GUI version against a (relatively) newer XIV code version, the GUI is protecting us from a potential mismatch. This means the GUI is always backward compatible, but is not always forward compatible. Now speaking of greyed out, we saw a bug on previous versions of the GUI code, where a read-only user id would be confronted with a similar sight. The machine would be greyed out and thus unmanageable. The work around was to use the older 2.4.2b build 10 version of the GUI. The good news is that 2.4.4 contains the fix for this bug (so its another reason to upgrade). . That is all for now. Don't forget you can always logon as p10demomode to get a demo mode view of the XIV GUI.