IBM SAN Volume Controller (SVC) has offered Fibre Channel Storage Virtualization since June 2003. Two SVC nodes communicate with each other via fibre channel to form a high availability I/O group. They then communicate with the storage that they virtualize via Fibre Channel and with the hosts they serve that virtual storage to, via Fibre Channel. When IBM added real-time (metro mirror) and near real-time (global mirror) replication it was also done using Fibre Channel, with each SVC cluster communicating to the other by connecting using fibre channel protocol transported over dark fibre with or without a WDM or via FCIP (Fibre Channel over IP) routers.
Each Fibre Channel port on an SVC node can be a SCSI initiator to backend storage, a SCSI target to hosts and all the time communicate to its peer nodes using those same ports. With every generation of SVC node, these ports got faster and faster, going from 2 Gbps to 4 Gbps to 8 Gbps. In SVC firmware V5.1 IBM added iSCSI capability to the SVC using the two 1 Gbps ethernet ports in each node. This allowed each node to also be an iSCSI target to LAN attached hosts.
When the Storwize V7000 came out in Oct 2010 it offered all of this capability, plus offered two fundamental changes to the design.
Firstly the two controllers in a Storwize V7000 can communicate with each other across an internal bus, eliminating the need to zone them together (or even attach the Storwize V7000 to Fibre Channel fabrics).
The other more obvious difference is that a Storwize V7000 comes with its own disks, which it communicates with via multi-lane 6 Gbps SAS.
When IBM added 10 Gbps Converged Enhanced Ethernet adapters to the SVC and to the Storwize V7000, these adapters operated as iSCSI Targets, allowing clients to access their volumes via a high-speed iSCSI network. In V6.4 code IBM allowed these adapters to also be used for FCoE (Fibre Channel over Ethernet). These are also effectively SCSI targets ports allowing hosts that use CEE adapters to connect to the SVC or V7000 over a converged network.
If you have a look at the Configuration limits page for SVC and Storwize V7000 version 6.4 (the Storwize V7000 one is here), you will see this interesting comment:
"Partnerships between systems, for Metro Mirror or Global Mirror replication, do not require Fibre Channel SAN connectivity and can be supported using only FCoE if desired"
So does this mean we can stop using FCIP routers to achieve near real-time replication between SVC clusters or Storwize V7000s? The short answer is most likely not. Lets look at why...
The whole reason Fibre Channel became the standard method to interconnect Enterprise Storage to Enterprise hosts is simple: Packet loss is prevented by buffer credit flow control. Frames are not allowed to enter a Fibre Channel network unless there are buffers in the system to hold them. Frames are normally only dropped if there is no destination to accept them. Fibre channel is a highly reliable, scalable and mature architecture. When we extend Fibre Channel over a WAN we do not want to lose this reliable nature, so we use FCIP routers like Brocade 7800s, that continues to ensure frames are reliably delivered in order, from one end point to another.
Converged enhanced ethernet allows Fibre Channel to be transported inside enhanced ethernet frames. The one fundamental that CEE brings to the table is the same principle that a frame should not enter the network without a buffer to hold it. Extending FCoE over distance has the same challenge: the moment you start moving those frames over a WAN connection you need to ensure frames are not lost due to congestion. How do we do this? The same way we did with Fibre Channel: we use Dark Fibre, we use WDMs or we use routers. The same issues and requirements exist.
For more information on FCoE over distance check out this fantastic Q&A from Cisco:
IBM has today announced a whole swag of planned new features across the entire IBM Storage product line. You can read the announcement letter here and I have also dropped the text at the bottom of this blog post (to save you clicking on the link).
It's a very impressive list, but to hone in on a few of the more exciting offerings:
IBM Easy Tier will be enhanced to cache hot data in SSD storage installed in a client server. Looks like it will initially be a combination of DS8700/DS8800 and AIX with or Linux servers. I am sure there are plenty who will immediately think of EMC VFCache, so I am keen to get more details so I can see how the two compare. If you are curious in the meantime, check out this EMC fact sheet and then read this fascinating interview with the CMO of FusionIO.
A new high density storage module will be made available, initially I suspect for the DS8800. This is a really important step as we are seeing a lot of new technologies emerging in the SSD space. This is because the technical requirements of SSD don't always line up with the architectures of existing storage controllers, so a custom built enclosure designed just for SSD makes perfect sense.
The IBM XIV will be enhanced with the ability to cluster multiple XIVs together and migrate volumes non-disruptively between them. The non-disruptive volume migration is a great new feature which should definitely help with swapping XIVs out as new models come available.
There are plenty of other new features as well, so check out the announcement letter reproduced below:
IBM® intends to support a number of new enhancements to a variety of IBM storage systems in the future. These enhancements will leverage innovative research on intelligent algorithms, automation, and virtualization that is being incorporated into products in the IBM storage portfolio. The statements of direction highlighted here are intended to provide a glimpse into the IBM storage roadmap for selected product capabilities.
IBM intends to deliver:
Advanced Easy Tier™ capabilities on selected IBM storage systems, including the IBM System Storage® DS8000® , designed to leverage direct-attached solid-state storage on selected AIX® and Linux™ servers. Easy Tier will manage the solid-state storage as a large and low latency cache for the "hottest" data, while preserving advanced disk system functions, such as RAID protection and remote mirroring.
An application-aware storage application programming interface (API) to help deploy storage more efficiently by enabling applications and middleware to direct more optimal placement of data by communicating important information about current workload activity and application performance requirements.
A new high-density flash storage module for selected IBM disk systems, including the IBM System Storage DS8000 . The new module will accelerate performance to another level with cost-effective, high-density solid-state drives (SSDs).
IBM intends to extend IBM Active Cloud Engine™ capabilities to:
Allow files on selected NAS devices to be virtualized by SONAS and Storwize® V7000 Unified. Virtualization capabilities provide access across a unified global namespace, while facilitating transparent file migrations in parallel with normal operations. This capability will help provide customer investment protection as clients continue to leverage their existing NAS assets while exploiting the capabilities of IBM Active Cloud Engine .
Enable file collaboration globally via IBM Active Cloud Engine . This capability will help enhance productivity where users at geographically dispersed locations can both share and modify the same file.
IBM intends to deliver Cloud features to SONAS and Storwize V7000 Unified to support:
Web Storage Services, a standards-based object store and API that implements the Cloud Data Management Interface (CDMI) standard from Storage Networking Industry Association (SNIA) to support the implementation of storage cloud services.
Self-service portal designed to speed storage provisioning, monitoring, and reporting.
IBM intends to support an increased scalability of capacity, performance, and host bandwidth by clustering IBM XIV® Gen3 systems together and providing the capability to migrate volumes across the cluster without disrupting applications. Management of the cluster will remain simple with consolidated views and shared configurations across the systems. These capabilities are intended to help clients address the scalability and management requirements for effective cloud computing.
IBM intends to extend NAS data retention enhancements for IBM Storwize V7000 Unified and IBM SONAS to provide file "immutability" to help support file integrity from the time the file is designated as immutable through its lifecycle. Immutability is intended to secure files from inadvertent or malicious change or deletion.
IBM intends to enable Real-time Compression for block and file workloads on Storwize V7000 Unified systems. This enhancement is designed to help clients experience the same high-performance compression for active primary block and file workloads on Storwize V7000 Unified that is being announced for block workloads on Storwize V7000. IBM Storwize V7000 Real-time Compression is designed to deliver enhanced storage efficiency with potential benefits including lower storage acquisition cost (because of the ability to purchase less hardware), reduced storage growth, and lower rack space, power, and cooling requirements.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information in the above paragraphs are intended to outline our general product direction and should not be relied on in making a purchasing decision. The information is for informational purposes only and may not be incorporated into any contract. This information is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.
One common question that I hear on a regular basis regards the availability of an SRA for VMware SRM 5.0 when using Storwize V7000 or IBM SVC running V6.3 firmware. This combination is currently unsupported as per the alert found here.
The good news is that there are now IBM SRAs available for clients running SRM in combination with V6.3 firmware. While this combination is still not listed on the VMware support matrix found here, you can download the SRAs direct from IBM if your need is urgent.
It is ironic that only days after I wrote that 497 is the IT number of the beast, I learn that Linux has another unfortunate number: 208.
The reason for this is a defect in the internal Linux kernel used in recent firmware levels of SVC, Storwize V7000 and Storwize V7000 Unified nodes. This defect will cause each node to reboot after 208 days of uptime. This issue exists in unfixed versions of the 6.2 and 6.3 level of firmware, so a large number of users are going to need to take some action on this (except those who are still on a 4.x, 5.x, 6.0 or 6.1 release). If you have done a code update after June 2011, then you are probably affected. This means that if you are an IBM client you need to read this alert now and determine how far you are into that 208 day period. If you are an IBMer or an IBM Business Partner, you need to make sure your clients are aware of this issue, though hopefully they have signed up for IBM My Notifications and have already been notified by e-mail.
In short what needs to happen is that you must:
Determine your current firmware level.
Check the table in the alert to determine if you are affected at all, and if so, how far you are potentially into the 208 day period.
Prior to the 208 day period finishing, either reboot your nodes (one at a time, with a decent interval between them) or install a fixed level of software (as detailed in the alert).
To give you an example of the process, my lab machine is on software version 18.104.22.168 which you can see in the screen capture below. So when I check the table in the alert, I see that version 22.214.171.124 was made available on January 24, 2012, which means the 208 day period cannot possibly end before August 19, 2012.
Earliest possible date that a system running this release could hit the 208 day reboot.
SAN Volume Controller and Storwize V7000 Version 6.3
30 November 2011
25 June 2012
24 January 2012
19 August 2012
Regardless, I need to know the uptime of my nodes, so I download the Software Upgrade Test Utility (in case you have an older copy, we need at least version 7.9) and run it using the Upgrade Wizard (NOTE! We are NOT updating anything here, just checking):
I Launch the Upgrade Wizard, use it to upload the tool and follow the prompts to run it, so that I get to see the output of that tool. The output in this example shows the uptime of each node is 56 days, so I have a maximum of 152 days remaining before I have to take any action. At this point I select Cancel. You can run this tool as often as you like to keep checking uptime.
Note if you are on 6.1 or 6.2 code you may see a timeout error when running the tool, especially for the first time. If you do see an error, please follow the instructions in the section titled "When running the the upgrade test utility v7.5 or later on Storwize V7000 v6.1 or v6.2" at the Test Utility download site.
As per the Alert:
If you are running a 6.0 or 6.1 level of firmware, you are not affected.
If you are running a 6.2 level of firmware, the fix level is v126.96.36.199 which is available here for Storwize V7000 and here for SVC.
If you are running a 6.3 level of firmware, the fix level is v188.8.131.52 which is available here for Storwize V7000 and here for SVC.
If you are using a Storwize V7000 Unified, the fix level is v184.108.40.206 which is available here.
You should keep checking the alert to find out any new details as they come to hand. If you are curious about Linux and 208 day bugs, try this Google search.
*** Updated April 4, 2012 with links to fix levels ***
If you have any questions or need help, please reach out to your IBM support team or leave me a comment or a tweet.
*** April 10: The IBM Web Alert has been updated with new information on what to do if your uptime has actually gone past 208 days without a reboot. In short you still need to take action. Please read the updated alert and follow the instructions given there. ***
I always laugh when people say to me: I wouldn't know what to blog about!
When you work in pre-sales support, you constantly get asked questions and each one of them could be the subject of a new blog post. Right now the most common question I am getting is:
I am implementing VMware Site Recovery Manager (SRM). One of the components I need are vendor specific Site Recovery Agents (SRA). I have searched IBM's website but cannot find them. Where are they?
So the short answer is: you get them from the VMware SRM download site. However before downloading, there is a key task that absolutely needs to be performed:
Visit the VMware vCenter Site Recovery Manager Storage Partner Compatibility Matrix. This site will confirm what products are supported by each version of SRM. You can find it here, but clearly you need to check back regularly to ensure you have the latest information.
Now find your storage device in the matrix and confirm what firmware levels are supported. This is really important. For example, the Feb 27, 2012 edition of the matrix tells me that the Storwize V7000 is supported for SRM version 5.0, but only when running Storwize V7000 firmware version 6.1 or 6.2. This is significant because if you upgrade to version 6.3 you are not supported. In fact that combination doesn't actually work yet, as detailed here. Clearly something you need to be aware of when planning firmware updates.
So where are the SRAs? On each of the pages below use the Show Details button to see what version SRAs are being shipped with that SRM (although sometimes the pages take a few days between an SRA being added and the page being updated):
There are a few more questions I routinely get asked:
Does IBM actually have an SRA download site?
The answer is yes, but it is an FTP site only for SRAs written by IBM. It is principally a repository for older SRAs and beta SRAs but you can also find the current SRAs on it. You can find the site here. Note however that it is NOT the official source. For that you need to use the VMware site.
What about the SRA for LSI/Engenio based products like the DS4800?
These used to also be found on the LSI site, but since LSI sold Engenio to NetApp, it is no longer available from the LSI or NetApp websites. You need to download the current version from the VMware sites listed above. There is a version for SRM 5 on the VMware download site.
What about nSeries SRAs?
If you need an nSeries SRA, again you should go to the VMware download pages. There are separate SRAs listed and available for IBM nSeries (as opposed to an SRA for NetApp branded filers).
What about an SRA for XIV with SRM version 5?
The answer: The SRA for XIV with SRM 5 (and 5.0.1) is now available from VMware. If you have access to download SRM, you will be able to download SRA version 2.1.0. It is the same SRA for both XIV Generation2 and Gen3.
What about an SRA for Storwize V7000 and SVC version 6.3 code?
The answer: It is coming. We are working to make it available as soon as possible. I will update this post as soon as I have a date for you (we are talking weeks, not months).
*** Update March 23, 2012 - Added details on SRM 5.0.1 ***
Many years ago I picked up a book that literally blew my mind. It was the Cuckoo's Egg by Clifford Stoll and it's a genuine classic, a true tale of hackers and how one was tracked down in the very early days of the internet.
Now the story is about events in 1986, so it captures the state of technology at the time (which rather dates the book), but wow, what a great story.
So why mention the book? Well apart from the fact that it is well worth a read, the key issue that Clifford saw again and again was default passwords. The hacker would identify a target and then try to logon using default IDs and default passwords, usually with great success.
Now I have blogged in the past about the determined (but often ignored) way that Brocade switches berate you into changing default passwords. But pretty well all products need to do this, as they all have the same issue (and a truly problematic counter-point). You absolutely need to do two things with every product in your data center:
Change the default passwords on every device you deploy.
Record what those passwords got set to (preferably using a logical or physical password safe).
Now don't laugh, but forgotten/lost passwords on data center kit (like switches) is a VERY common problem. When I worked in the IBM Storage Support team I took calls EVERY WEEK from clients who had devices they could not logon to, for all manner of reasons. For some, supplying them with the default passwords saved them (and condemned their employer?), but for others they needed much more detailed assistance.
My preferred solution to this challenge is to use external authentication (like LDAP) but being able to reset passwords with an external tool is also a nice option to have available.
The reason I started thinking about this is a nice tool IBM offer for the Storwize V7000 called the Initialization Tool that you can download from here. Using this tool you can reset the password of the Superuser ID on a Storwize V7000 back to the default (passw0rd). The tool runs on a USB key. After requesting the tool to help you to reset the superuser password, you insert the USB key into the Storwize V7000, wait for the orange indicator light on the relevant node canister to stop blinking and the task is complete. Then put the USB key back into your laptop and run the init tool again to get a completion report that should look like this:
This is great to rescue customers who have lost their passwords, but the question then gets raised: Can I block this?
My first response is: if you are concerned about unauthorized people with malicious intent placing USB keys into your Storwize V7000, then don't let them into your computer room (presuming you can spot them by the colour of the hat they are wearing). If that is not an option, lock the rack that the Storwize V7000 resides in (change control does have its benefits). If that is not an option, there is one more alternative, but it is a tad extreme.
What we can do is prevent password reset via USB key (or in the case of the SVC, via the front panel). We do this by issuing the following CLI command: setpwdreset -disable
In the following example, I confirm that password reset is possible (value 1), I then disable it and confirm that password reset is no longer possible (value 0). If curious I could then get some help on that command:
Only if your paranoia is matched by your attention to detail.
My reason to hesitate recommending it is simple: If you prevent password reset and then forget your password (and have no other local Security Administrator accounts), you have locked the door and thrown away the key. Far better to physically lock the rack.
In the end though, your company needs to set a policy that is actively enforced (with no exceptions). So get to it.
When IBM brought out the SAN Volume Controller (SVC) in 2003, the goal was clear: support as many storage vendors and products as possible. Since then IBM has put a huge ongoing effort into interoperation testing, which has allowed them to continue expanding the SVC support matrix, making it one of the most comprehensive in the industry. When the Storwize V7000 was released in 2010 it was able to leverage that testing heritage, allowing it to have an amazingly deep interoperation matrix on launch date. It almost felt like cheating.
However I recently got challenged on this with a simple question: Where is the VNX? If you check out the Supported Hardware list for SVC V6.3 or Storwize V7000 V6.3 you can find the Clariion up to a CX4-960, but no VNX.
The short answer is that while the VNX is not listed there yet, IBM are actively supporting customers using VNX virtualized behind SVC and Storwize V7000. If you have a VNX 5100, 5300, 5500, 5700 or 7500 then ask your IBM pre-sales Technical Support to open an Interoperation Support Request. The majority are being approved very quickly. The official support sites that I referenced above will be updated soon (but don't wait, if you need support, ask for it). IBM is working methodically with EMC to be certain that when a general publication of support is released for VNX (soon!), both companies will agree with the published details.
And for the wags who think that this is a ringing encouragement to buy VNX, you would be missing the point. You cannot be a serious Storage Virtualization vendor if you are not willing to support your clients purchasing decisions, regardless of which vendor they buy their storage from. IBM have been staying that course and demonstrating that willingness since 2003. It's a pretty good track record and one that they are determined to maintain.
I have updated my IBM Storage WWPN Determination Guide to version 6.5. You can find the updated guide on IBM Techdocs here.
The main change is that new DS8800s are now presenting slightly different WWPNs, so I added three new pages to describe the changes.
If this guide is new to you, its purpose it to let you take a WWPN and decode it so you can work out not only which type of storage that WWPN came from, but the actual port on that storage. People doing implementation services, problem determination, storage zoning and day to day configuration maintenance will get a lot of use out of this document. If you think there is an area that could be improved or products you would like added, please let me know.
It is also important to point out that IBM Storage uses persistent WWPN, which means if a host adapter in an IBM Storage device has to be replaced, it will always present the same WWPNs as the old adapter. This means no changes to zoning are needed after a hardware failure.
I also host the book on slideshare, so you can also view and download it from there:
Here are two common statement I often hear from clients:
I don't just want SAS drives, I also want SATA drives. SATA drives are cheaper than SAS drives.
Nearline SAS drives are just SATA drives with some sort of converter on them.
So is this right? Is this the actual situation?
First up, if your storage uses a SAS based controller with a SAS backplane, then normally you can plug SAS drives into that enclosure, or you can plug SATA drives into that enclosure. This is great because when you plug SATA drives into a SAS backplane, you can actually send SCSI commands to the drive plus you can send native SATA commands t00 (which is handy when you are writing software for RAID array drivers).
But (and this is a big but) what we do know is that equivalent (size and RPM) SAS drives perform better than SATA drives. This is because:
SAS is full-duplex, SATA is simplex.
SAS uses the native SCSI command set which has more functionality (which leads to the next point).
A SAS drive uses SCSI error checking and reporting which is much more robust than the SATA error reporting. This allows your storage system to collect richer information from the drive if errors are occurring (such as a failing or marginal disk).
SAS drives are dual ported which is vital in dual controller enclosures.
So given a choice (and a very small price differential), why choose SATA over SAS? SAS is the clear winner. What we should instead differentiate on is speed (7.2K RPM vs 10K RPM vs 15K RPM vs SSD) and size (2.5" vs 3.5" form factor).
Which leads us to Nearline SAS
It is a common belief, that if you buy a Nearline SAS (or NL-SAS) drive it is really a SATA drive with a SAS connector (interposer) stuck on it. But this is confusion from the past.
What led to the confusion?
Most midrange and enterprise storage controllers and enclosures up until recent years, used disks that had fibre channel interfaces on them. We plugged those disks into fibre channel enclosures. Examples include the DS4700 or the DS8100. And yet these devices also offered SATA drives. How did they do this?
They took a SATA drive and added a SATA to Fibre Channel converter card to the disk. We call this extra piece of hardware an interposer or bridge card. So people start assuming that this is common practice in every product. In fact we are now seeing SAS drives being put into fibre channel disk enclosures by using a SAS to fibre channel interposer.
There are in indeed older products that did take a SATA drive and add a SATA to SAS interposer to achieve a similar thing. But that really is not necessary any more. The reason? The same hard drive can now be ordered from the factory as either a SAS drive or a SATA drive.
Seagate have a nice selector tool to let you see all their possible combinations. For instance you can order a 2 TB drive with a 6 Gbps SAS interface, which is a model ST32000444SS:
Or you can order a 2 TB drive with a 6 Gbps SATA interface, which is a model ST2000NM0011:
So what you get is very similar drive hardware (same spindles, heads, motors) but with different adapter hardware, built with the desired adapter at manufacture time. Meaning that if we install this drive into a SAS enclosure, there is no need to add an interposer or bridge card to the drive after you bought it.
This leads to the next question:
OK. So this is good, so Nearline SAS drives are MADE as SAS drives. Does that mean a drive manufactured with a SAS adapter is a SAS drive or a Nearline SAS drive?
Now we are mixing up two different things. SAS as a standard is a combination of a connection technology (the Serial Attached part) and a command set (the SCSI part). Actually SCSI as a standard also defines both connection methods and command sets. So SAS is really talking about how we connect to the disk and what command set we use to control the disk.
Nearline on the other hand is a statement about the disks rotational speed and it's mean time between failure (MBTF). A Nearline-SAS drive is Nearline because:
It rotates slower (7200 RPM) than the higher specified Enterprise drives (that spin at 10 K or 15K RPM). Because they are slower they can also hold way more data.
It has a lower MBTF (1.2 million hours) than the higher specified Enterprise drives (which are normally specified at 1.6 million hours).
So we have now gone full circle. A Nearline-SAS drive can use the same physical disk hardware as a SATA drive, but with a superior adapter that uses a superior command set, built onto the drive at manufacture time.
Still confused or want to read some more? Check out these links:
I love USB keys, I love free ones, conferences give away ones and ones shaped like lego blocks. The exciting thing (for me) is that if you buy a Storwize V7000, you also get a USB key: A key which has two fundamental purposes:
It's used to make installation very quick and easy (which it does very well!).
It's used to reset the superuser password (in case you forget what it is) or to set the service IP addresses (in case you didn't set them like I suggested you do ).
This is all well and good but what happens when you lose it, borrow it or accidentally throw it out? (oops) If you are searching for it, yours may well have looked like this:
So what to do? The answer is: It's OK, there is nothing magic about this key. In fact the key contains just one piece of software, which you can get from here. Just download the initialization tool and copy it onto your own USB key. The original key also had an Autorun file, but you don't need that (actually I object to auto-running USB keys anyway).
BUT... and there is always a but... I cannot guarantee that EVERY USB key you try will work. Why not? Because some USB keys are formatted strangely or insist on running unique applications before they will work. There is some good, simple advice on the InfoCenter that you can find here. The main trick is to use a USB key that is formatted with the FAT32, EXT2, or EXT3 file system on its first partition and does not need to auto-run any applications before working.
The IBM SVC has has been setting records in SPC-1 (OLTP-like) benchmarks for many years. However recently HP stole the crown with a 3Par benchmark of 450,212.66 IOPS.
But in breaking news, the SVC is back on top with the very first SPC benchmark that exceeded 500,000 IOPS (520,043.99 to be precise!). You can see the executive summary here.
This benchmark used eight of the current SVC engines (model CG8s) with Storwize V7000 as the backend disk. It shows the awesome power of SVC, its ability to scale and to handle very large configurations with very large throughput requirements. It also shows the power of IBM pSeries which was used to drive these IOPS.
The first update for Storwize V7000 and SVC release 6.3 is now available. You will find it here for Storwize V7000 and here for SVC (note both links will require you to login to Fix Central with your IBM ID). As usual the new release contains a combination of new features and fixes. The new features are:
New features in SVC 220.127.116.11
* Support for multi-session iSCSI host attachment
* Language Support for Brazilian Portuguese, French, German, Italian, Japanese, Korean, Spanish, Turkish, Simplified Chinese and Traditional Chinese
There are also several fixes (with some variation between SVC and Storwize V7000, mainly around the platform hardware). The release notes (which you can find at the links above) detail them all. Two fixes I have been looking forward to are:
IC80253 Unable to log into the GUI if password contains special characters. This meant that a password with a comma in it could not be used in the GUI (you got a backspace instead). Passwords with commas could be used in the CLI. This bug was picked up by one of my clients when trying out LDAP and is now fixed in 18.104.22.168.
IC80501 Performance statistics collection fails to record read and write response times for internal drives. This issue meant that SVC internal SSD drives always showed 0 ms response times in TPC.
Note that the Drive firmware does not need to be updated with this release. The new upgrade test tool (version 7.3) will not ask you to update them. I will let you know when that situation changes.
The big question of course is which drive type to choose? The answer is that ideally you should possess three pieces of information:
How much usable space do you need in GB or TiB? Don't confuse binary and decimal!
What is your typical I/O profile. For instance 70% reads 30% writes, 32KB block size.
What are your IOPS and response time requirements?
Armed with this information, get your IBM Sales Rep or Business Partner to model your requirements using Capacity Magic and Disk Magic. These modelling tools will tell you how much usable capacity a particular configuration will give you and what performance you can expect to get from it (given a particular I/O profile). If you don't know your I/O profile or IOPS requirements, you can still see performance modeling using industry standard benchmarks.
I am getting this question on a very regular basis:
"We have just upgraded to ESXi 5.0 but we cannot find the VAAI driver on the IBM Website"
The answer? There is no vendor supplied driver because no driver is needed. ESXi 5.0 uses a SCSI T10 compliant set of commands that all vendors need to support for VAAI to work.
But of course in the tradition of all answered questions, it leads to another question:
"Once I have upgraded to ESXi 5.0 how can I tell if VAAI is really working?"
The good news is that it is very easy to spot if ESXi 5.0 has detected a VAAI capable LUN. The moment a new LUN is detected by ESXi 5.0 it tries out an Atomic Test and Set command. If that works, you will see that Hardware Acceleration shows as Supported in vCenter. In the screen capture below I have three datastores, two from XIV and one from Storwize V7000, all presented to an ESXi 5.0 server. I dragged the Hardware Acceleration column over from the right hand side to help with the screen capture (in case your vCenter looks different), but you can see the Hardware Acceleration column shows each DataStore as Supported (and did so the moment the volume was detected).
Of course having seen the Hardware Acceleration Supported message only proves that Atomic Test and Set works. To confirm if XCopy (Hardware Accelerated Move) is working, on SVC or Storwize V7000 we can use the Performance monitoring panel. In the example below I first performed a storage vMotion, moving a virtual machine between two Datastores located on the same Storwize V7000 (running 22.214.171.124 firmware). I then performed a clone of the same virtual machine, where the source was on one datastore and the target was placed on another (but both located on the same Storwize V7000). What you can clearly see is that both operations (storage vMotion and cloning) generated no volume traffic, only MDisk traffic. This means that the ESXi server is doing none of the work and the storage is doing all of the work.
The Storwize V7000 and SVC have a command line interface that you access via SSH. Every-time you logon, whether it is to transfer a file (using a tool like pscp), issue a single shot command from a script (using a tool like plink) or logon to issue commands interactively (using a tool like PuTTY), you clearly need to authenticate yourself. Since June 2003, the way you did this was to use a public/private key pair, where the SVC or Storwize V7000 had the public key and the SSH client (such as PuTTY) authenticated using the private key (the PPK file).
However with release 6.3 of the SVC and Storwize V7000 firmware, the use of key files is now optional. A user can now authenticate purely by using a password. This includes using your domain ID. So if you defined LDAP to your machine, as I documented here, you could now SSH direct to your Storwize V7000 or SVC, use your Domain user id and password and not go through the key file setup task. Nice!
The choice to continue to authenticate just with an SSH key remains available. If a user has both a password and a configured key file, then either method will work (you only need to use one - not both). Existing scripts will be unaffected by this change, so nothing gets broken because of this.
I think this is a very positive change and one I openly welcome. Combined with LDAP, this really makes user account setup an easy and simple task.
IBM recently released a new version of firmware for the SAN Volume Controller and Storwize V7000. This is known as release 6.3 and continues the tradition of two major updates per year, each adding significant new functions.
So the 6.3 release notes for both Storwize V7000and SVC listed the following new feature:
Support for 4096 host WWPNs
Since I blithely listed this feature in a recent post I have received lots of emails asking exactly what it means, so I thought I had better explain.
The IBM SVC and Storwize V7000 have always had very clearly published maximum capabilities such as the ones listed here for Storwize V7000 release 6.3 and here for SVC release 6.3.
Most of these numbers are very high and few customers actually approach these maximums. The main issue I am seeing for some of our larger AIX customers is this one:
Total Fibre Channel ports (WWPNs) per I/O group: 512
The reason this can become an issue is the combination of NPIV and AIX Live Partition Mobility. NPIV allows one physical HBA to be shared among multiple operating system instances, each one believing it has exclusive access to the HBA with each one allocated its own unique WWPNs. Suddenly a single HBA which used to present just one WWPN through the SAN to the SVC, can now present vast numbers of them. In addition AIX Live Partition Mobility (which lets you move AIX operating systems between LPARs on the fly) needs additional pre-configured WWPNs defined on the target LPAR to support the move. This further increases the quantity of WWPNs that need to be defined to the SVC (one easy way to spot NPIV generated WWPNs is they normally start with the letter C).
So the bottom line is that IBM needs to make this limit bigger and SVC and Storwize V7000 6.3 code contains the necessary architectural changes to allow this. The first phase is to start potentially supporting up to 2048 WWPNs per I/O group although clearly based on the initial version of the release notes, the long-term plan is to support 4096.
But there is a problem and it has nothing to do with the SVC or Storwize V7000. The problem is that there are certain SAN configurations which may have issues with these large numbers of WWPNs (mainly around older SAN switches not having the CPU power for the switch fibre channel name-server and login-server to handle vast numbers of WWPNs coming out of one HBA).
So what should you do if you need to push the limits?
Contact your IBM Pre-Sales support and ask for a SCORE request to be opened (also known as an RPQ). You will need to detail your current SAN configuration (especially switch models and firmware levels) so that SVC development can ensure you won't overwhelm your switches. It will also allow our development team to learn how many clients our there need this support. All approvals will include a requirement to upgrade to release 6.3, so you should include this in your planning.
Any questions? Feel free to leave a comment or send me a tweet or an email.
With the 6.3 release of the Storwize V7000 and SVC code (which I blogged about here), there are so many new features and functions that I have plenty more to blog about!
The first new feature I blogged about was LDAP support, but an existing feature that has been enhanced is the performance monitor (brought in with release 6.2). When this first came out I put a video on You Tube showing what metrics could be displayed in that release. This is a sped up image with no voiceover:
Now with release 6.3 IBM has added separate graphs for reads and writes plus the ability to display IOPS or MBPS, plus the ability to display graphs of read and write latency. Nice! I got so excited I made another You Tube video, this one with narration. So now you can compare the new to the old:
Lets imagine a new rack server or a new blade server has been added to your Fibre Channel SAN. The first job for the SAN administrator is to zone it to the storage it requires access to. The task normally runs something like this:
Identify the WWPNs for the new server HBA. We can do this using Qlogic SAN Surfer or Emulex HBAnywhere, or by looking at the WWPNs reported by the Fibre Channel switch or by using datapath query wwpn (with SDD and SDDDSM) or by using the xiv_fc_admin -P command with the XIV HAK. There are lots of different ways, you get the idea.
On fabric 1 create a new alias for the server HBA port cabled to that fabric.
For each storage device that the server needs access to on fabric 1 (or possibly just switch 1), create a new zone and include the new server alias and the alias for every relevant storage port on that device. Repeat if you have other storage devices (so two XIVs means two new zones).
Put the new zone (or zones) into the active zoneset (or a clone of it) and activate it.
Repeat on fabric 2 (after waiting a decent interval to ensure no mistakes were made in fabric 1... well I hope you wait.... you do don't you?).
The main trap here is that when creating a zone, you need to ensure you select all of the correct storage aliases for your selected storage device. For instance we could have a simple layout like this:
Fabric 1 contains our new server (in this example an IBM x3850) and three XIV ports:
This means when creating the zone I need to identify and select four separate aliases. What I could do instead is create an alias with all my XIV target ports in it. Now I only have two aliases to select in that fabric:
This means when creating the zone I need to identify and select three separate aliases. What I could do instead is create an alias with both my Storwize V7000 WWPNs in it. Now I only have two aliases to select in that fabric:
This method of amalgamating multiple storage port aliases works fine for devices like DS8000, SVC, Storwize V7000 and XIV. I use this method all the time to simplify zoning and I find it reduces both mistakes and the time required to complete zoning tasks.
The only exceptions are:
Don't do it for DS3000, DS4000, DS5000 or DCS3700 as the controllers on these devices do not like to see each other through the switch.
Don't combine ports from different storage devices, so if you have two XIVs in a fabric create one alias for the target ports of each XIV (although you could combine ports from different SVC I/O groups within the same SVC cluster into one alias). You should still use individual aliases for ports being used for migration or replication purposes.
Don't use the WWNN to create an alias. Always create multi-WWPN aliases so you have granular control of which ports go into the alias. If you use the WWNN from an XIV you will also implicitly include any ports that are being used for replication or migration and thus zone them to the host, which makes no sense.
I would love to hear any techniques you have to make your (and my) life easier.
Once your SVC or Storwize V7000 is upgraded to version 6.3 you can start using LDAP for authentication. This means that when you logon, you authenticate with your domain user-id and password rather than a locally created user-id and password.
So why is this important?
It saves you having to configure every user on every SVC or Storwize V7000. If you have multiple machines this makes it far more efficient to set up authentication.
It means that when commands are executed on the SVC or Storwize V7000, the audit log will show the domain username that issued that command, rather than a local username, or worse just superuser (i.e. who mapped that volume? The superuser did.... who? )
It gives you central control over access. If someone leaves the company you just need to remove access at the domain controller, meaning there won't be orphan user-ids left on your Storage equipment.
So as an exercise I added my lab Storwize V7000 to our domain to show how it is done. This example also applies to an SVC so don't be confused if I only refer to Storwize V7000 from now on.
The first task is to negotiate with your Domain administrator to get a new group setup on the domain. In this example I use a group called IBM_Storage_Admins which lets me use this group for various storage devices (such as an XIV or a SAN Switch).
To create this group we need to logon to the Domain Controller and configure Active Directory. An easy way to do this from the AD controller is to go to Start → Run and type dsa.msc and hit OK. The Active Directory Users and Computers Management Console should open.
Select the groups icon to create a new group.
Enter your group name, in my case: IBM_Storage_Admins and hit OK.
Now right select relevant users who need access to the storage and add them to the IBM_Storage_Admins group. In this example I have selected Anthony (which uses anthonyv as a username).
In this example we are adding anthony into the IBM_Storage_Admins group:
Now it is time to configure the Storwize V7000 so start the Web GUI and logon as Superuser.
Firstly we go to Settings → Directory Services:
We choose the button to Configure Remote Authentication:
We choose LDAP and hit next.
We choose Microsoft Active Directory with no Transport layer Security. We then expand the Advanced Settings. My lab domain is ad.mel.stg.ibm so I use the Administrator ID on the Domain Controller to authenticate access. You could use any user that has authority to query the LDAP directory. We then hit Next.
We then add the domain controller which in this example is 10.1.60.50 and the base domain name chopped into pieces (so ad.mel.stg.ibm becomes dc=ad,dc=mel,dc=stg,dc=ibm ) and hit Finish.
Provided the command completes successfully we have defined the domain controller to the Storwize V7000. Now we need to add a group. Go to Access → Users.
Select the option to add a New User Group.
In this example we want to add a group for users allowed full admin access to the Storwize V7000. This matches the group we created on the Domain Controller. So we call the group IBM_Storage_Admins and we use the Security Administrator role (which is the most powerful role) and tick the box to enable LDAP for this group.
Now to test, I logon to the Storwize V7000 using the domain user-id anthonyv with that users domain password. Remember this user is not defined on the Storwize V7000 itself and that if it all goes wrong, we can still logon as Superuser.
Now I create a volume and delete it. Then I check the audit log from Access → Audit log.
Sure enough, we see exactly who did that command.
This is a great outcome for security,auditing and for easy access administration.
If you have issues, from the Settings → Directory Services menu, use the Global Actions dropdown on the right hand side to Test LDAP Connections and Authentication or re-configure LDAP.
If you already have existing users (what we call Local users), configuring remote authentication using LDAP does not disable or invalidate those local user-ids. This means you can either logon with a local user-id or logon with a Domain user-id. This is handy if the domain controller fails but can confuse you if your local user name and your domain user name are the same name (for example both anthonyv). The Storwize V7000 will look you up in the local user name list first. I suggest removing all local users (except superuser) as this will reduce confusion but still leave you a backdoor in case remote authentication stops working.
If you see any mistakes or have suggestions to improve the way I described this, please let me know.
The latest release of SVC and Storwize V7000 firmware is now available for download. The major new features that are added with this release are:
Global Mirror with Change Volumes
Native LDAP Authentication
Extended distance split clusters (for SVC)
Support for 4096 host WWPNs
These are some great new features. The ability to use Global Mirror with Change Volumes means clients can now mirror across far smaller pipes, while the increase in host WWPNs is very welcome news for NPIV installations that are suffering from WWPN sprawl.
If you plan to upgrade, firstly grab the new Upgrade Test Utility from here. The links to the Storwize V7000 and SVC versions are both on that page. Remember you can run this test as many times as you want whenever you want, to check the health of your device for upgrade. When you run the upgrade test utility on a Storwize V7000 you may get a message that your disks have down-level firmware. The process to update them is documented here.
If you're using a Storwize V7000 you can grab the 126.96.36.199 code from here. If you're using an SVC you can grab the 188.8.131.52 code from here. I am sending you to the compatibility matrix page because you should always check that your from level is ok for your to level.
To run the upgrade go to Configuration (the spanner icon) → Advanced → Upgrade Software →Launch Upgrade Wizard
I have not shown all the panels you will see because it is very much a follow-your-nose task, but in essence, first we feed it the Upgrade Test Utility file and run that test.
If you get warnings you may need to act on these. If you are unsure what to do to resolve a warning message, place a service call.
Once the test passes or you're happy you understand the warnings, we now point it at the code package and wait for it to copy across and keep hitting Next.
The application of the code shuts down and reboots each controller, with a 30 minute gap in between. You will transition from this (both nodes down-level, node1 being upgraded):
To this (node1 upgraded, node2 still online but waiting for 30 minutes):
When node2 starts the upgrade the GUI will failover to node1 and be upgraded to the new version. You will notice the difference immediately, it has a different look and feel. Please don't be tempted to play with the new functions until both controllers are upgraded! Wait until you see this (note a slight change, the GUI flow is now Settings (the spanner icon) → General → Upgrade Software:
Now your complete it is time to start checking out what is new... but that's a whole different blog post!
Wordpress (where I mirror this blog) as a blogging platform has some very nice features. One of them is that you can find out what search terms led people to your blog. I have noticed that searches like these have become very common:
XIV GUI and XIV GUI V3 or version 3
IBM vcenter plugin
This suggest to me that people are having trouble finding these files, or more importantly: maybe Google is having trouble helping them to find them!
The reason is simple: They have been moved to IBM FixCentral rather than being posted on separate easily trawled web pages. The good news is that once you know about Fix Central, finding any file becomes very very easy.
First up you HAVE to bookmark this URL (Do it now! Yes NOW!):
************************************************************************** Update 17/12/2011: A flash reporting a possible issue that could occur if a drive fails during drive firmware update can be found here.
Until the flash is updated showing how to avoid this issue, only update drive firmware when installing a new machine or if all hosts are offline.
IBM recently released new drive firmware for the Storwize V7000, so I thought I would share the process of how I update that firmware. You can download it from here. The details for this new package can be found here. I recommend you perform the drive update before you next update your Storwize V7000 microcode.
I want to be clear that one of the central goals of the Storwize V7000 is to ensure that performing drive firmware updates can be done online without host disruption. This is possible because each drive can be updated in less than around 4 seconds. The scripts I share below leave a 10 second delay between drives just to be safe. I would still prefer that you did the update during a quiet period.
We need to perform this procedure using the command line as there is no way to do this procedure from the GUI (yet).
There are four steps:
Upload the Software Upgrade Test Utility to determine which drives need updating.
Upload the drive microcode package.
Apply the drive software.
Confirm all drives are updated.
Step 1: Upload and run the upgrade utility
You will need the upgrade test utility which you can get from here.
You will need the Putty utility PSCP which you can get from here (although most of you should already have it).
You will need to have created a public/private key pair and assigned it to a user. In all the examples the user name I use is anthonyv. You need to use your own user-id, although you could also use admin. The process to create and associate the key pair is described here. Place the PPK file into the putty folder along with the upgrade test utility.
From the Putty folder we need to upload the test utility. You will need to change the key file name, userid and IP address (all highlighted in red) to suit your installation.
NOTE: The following command is being run in a Windows command prompt. You need to be in the C:\Program Files\Putty or C:\Program Files (x86)\Putty folder.
If you get a warning window like the one shown below, indicating we have down-level drives, we need to proceed to the next step (note that the enclosure and slot numbers are not the same as drive IDs). If you have a lot of drives, you can drop the -d from the svcupgradetest command to get a summary list.
******************* Warning found *******************
| Model | Latest FW | Current FW | Drive Info |
| HK230041S | 2920 | 291E | Drive in slot 24 in enclosure 1 |
| | | | Drive in slot 23 in enclosure 1 |
| ST9450404SS | B548 | B546 | Drive in slot 22 in enclosure 1 |
| | | | Drive in slot 21 in enclosure 1 |
| | | | Drive in slot 20 in enclosure 1 |
| | | | Drive in slot 19 in enclosure 1 |
| | | | Drive in slot 18 in enclosure 1 |
| | | | Drive in slot 17 in enclosure 1 |
| | | | Drive in slot 16 in enclosure 1 |
| | | | Drive in slot 15 in enclosure 1 |
| | | | Drive in slot 14 in enclosure 1 |
| | | | Drive in slot 13 in enclosure 1 |
| | | | Drive in slot 12 in enclosure 1 |
| | | | Drive in slot 11 in enclosure 1 |
| | | | Drive in slot 10 in enclosure 1 |
| | | | Drive in slot 9 in enclosure 1 |
| | | | Drive in slot 8 in enclosure 1 |
| | | | Drive in slot 5 in enclosure 1 |
| | | | Drive in slot 6 in enclosure 1 |
Step 2: Upload the drive microcode package
Download the drive update package from here. Put it into the PuTTY folder. From a Windows command prompt we need to upload the package using the following command. You will need to change the key file name, userid and IP address (all highlighted in red) to suit your installation. Note yet again that you are running this in a Windows command prompt from the PuTTY folder (not from inside an SSH session):
I have written some scripts to help you list the drive IDs that need to be updated and perform the updates. You can upgrade the drives one at a time, or in bulk, depending on how you want to do this. All the remaining commands are all run in a PuTTY session.
Firstly run this script to list all the drive IDs and current firmware levels. We need the drive IDs if we want to update individual drives.
svcinfo lsdrive -nohdr |while read did error use;do svcinfo lsdrive $did |while read id value;do if [[ $id == "firmware_level" ]];then echo $did" "$value;fi;done;done
The output will look something like this, showing the drive ID and that drive's current firmware level. From step 1 we know what the latest firmware level is, so we can compare to the current firmware level:
However you may have a lot of drives and want to upgrade them in bulk. So you could use this command, which updates drive ID 19 and 20 (highlighted in red). You could change and also add extra drives to the list as required:
for did in 19 20;do echo "Updating drive "$did;svctask applydrivesoftware -file IBM2076_DRIVE_20110928 -type firmware -drive $did;sleep 10s;done
If we just wanted to upgrade every single drive in the machine (regardless of their level), we could run this command:
svcinfo lsdrive -nohdr |while read did name IO_group_id;do echo "Updating drive "$did;svctask applydrivesoftware -file IBM2076_DRIVE_20110928 -type firmware -drive $did;sleep 10s;done
When updating multiple drives, I have inserted a 10 second sleep between updates, just to ensure the process runs smoothly. This means each drive takes about 13-15 seconds.
Once we have upgraded every drive, it is time for a final check.
Step 4: Confirm all drives are updated
You have two ways to confirm this. Firstly run the following command to list the firmware level of each drive. Is each drive reflecting the levels reported in Step 1?
svcinfo lsdrive -nohdr |while read did error use;do svcinfo lsdrive $did |while read id value;do if [[ $id == "firmware_level" ]];then echo $did" "$value;fi;done;done
Now run the software upgrade test utility again:
svcupgradetest -f -d
Provided you receive no warnings about drives not being at the recommended levels, you are now finished with the drive updates. Of course you could now proceed to install 184.108.40.206 firmware, but you can do that from the GUI.
The SVC and Storwize V7000 offers a command line interface that you access via SSH. You start your favorite SSH client (such as PuTTY or Mindterm) and then logon as adminor as your own user-id. Right now to do this you need to generate a private/public key pair, although with release 6.3 (which will be available November 2011), you will be able to logon via SSH using just a user-id and password.
Having logged on there are three categories of commands you can issue:
svcinfo: Informational commands that let you examine your configuration. svctask: Task commands that let you change your configuration. satask: Service commands that are only used in specific circumstances.
There are several CLI usability features that I routinely find users are not aware of, so I thought I would share some of them here:
1) Listing all possible commands
If you cannot remember a command, here is a simple trick to list them all. Issue one of the following commands:
svcinfo -h or svcinfo -?
svctask -h or svctask -?
You can also type either svcinfo or svctask and then hit the tab key twice to get a full listing. With svctask you will need to type y to list them all, as per the example shown below:
IBM_2076:STG_V7000:admin>svctask (HIT TAB twice!)
Display all 139 possibilities? (y or n) y
2) Getting help on a particular command
Having found the command you want, issue that command with either -? or -h to get help information. For instance:
svctask mkvdisk -?
svctask mkvdisk -h
You will be shown the same help information that you can find in the Infocenter, including examples of syntax.
3) Drop the svctask and svcinfo prefixes
In release 6.2 of the SVC and Storwize V7000 firmware, the requirement to prefix a command with svcinfo or svctask has been removed. However I tend to keep using them because I write a lot of example commands for clients and I cannot be sure which version of firmware they are running.
4) Use the shell
When we SSH to the SVC or Storwize V7000 we are connecting to a Linux operating system using a special restricted shell. Some of the common Unix commands don't work (such as ls or grep or awk), but any commands that are provided by the shell itself, will work, such as while, if, read, pipe and echo.
We can use this to construct some really clever commands.
For instance creating volume copies is very popular, but the default copy rate is rather slow (50, which equals 2 MBps). It is not unusual for end users to speed up the background copy and then forget to slow it down when they are finished. So I wrote two commands to help me out. Firstly I run a command to display the copy rate of every volume. Ideally I should see 50 alongside each volume. However I often find that some volumes are set to higher numbers, such as the maximum value of 100 (which is 64 MBps).
svcinfo lsvdisk -nohdr |while read id name IO_group_id;do svcinfo lsvdisk $id |while read id value;do if [[ $id == "sync_rate" ]];then echo $value" "$name;fi;done;done
Lets break down this command. The structure looks like this:
We start with svcinfo lsvdisk -nohdr. This gives us a list of every VDisk in column format with no header information.
We pipe the output of that lsvdisk command to while read. This reads the output one line at a time and lets us work with that output. We read the first three columns of output and label the data in the first column id, the second column name and the third column IO_Group. I find we need to label at least three columns. We could read extra columns if we wanted to, but all I want is the VDisk id and name.
For every line of data we issue an lsvdisk command against each listed VDisk using the VDisk id. This output is not in column format so we need to do something different here.
We now examine the output of the lsvdisk command for each VDisk by piping the output to while read. Since each line contains a descriptor and a value, we label them id and value. We use if to look for a line that starts with sync_rate.
When we find the sync_rate for a VDisk we print the value of the sync_rate and the VDisk name. We are done for this VDisk.
We now examine the next VDisk and again look for the sync_rate for that VDisk.
Once we have examined every VDisk, we are done.
I then run the following command which sets the copy rate for every volume to the default value of 50 (2 MBps):
svcinfo lsvdisk -nohdr |while read id name IO_group_id;do svctask chvdisk -syncrate 50 $id;done
Clearly you could edit this second command to change the copy rate to any value between 0 and 100. In each case you just paste the entire command in, and hit Enter.
Lets break down this command. The structure looks like this:
We start with svcinfo lsvdisk -nohdr. This gives us a list of every VDisk in column format with no header information.
We pipe the output of that lsvdisk command to while read. This reads the output one line at a time and lets us work with that output. We read the data ion the first three columns of output and label the data in the first column id, the second column name and the third column IO_Group. I find we need to label at least three columns. We could read extra columns if we wanted to, but all I want is the VDisk id.
For every line of data we read, we do the following command: svctask chvdisk -syncrate 50 $id. Since we labelled the first column of output from the lsvdisk command as id, and that column contains VDisk IDs, we are going to issue this command against every VDisk that got listed.
Once we have run the chvdisk command against every VDisk listed, we aredone.
There are lots of possible clever combinations and I will list a few more in upcoming posts.
I have also been getting lots of requests to write a post about updating drive firmware, so expect something on that very soon.
IBM have offered Enterprise Storage Virtualization since June 2003 with the IBM SAN Volume Controller (SVC). October 2010 saw IBM releasing the Storwize V7000, taking the SVC code and packaging it into a midrange disk product. So now you have four possible choices:
Use SVC to virtualize your storage.
Use Storwize V7000 to provide internal SAS drives plus virtualize your storage.
Use Storwize V7000 as a midrange disk product.
Use Storwize V7000 virtualized behind SVC.
The great thing is that all four choices are valid and all four choices work just fine. But for customers already using SVC, or considering SVC, the question then becomes, should I virtualize a Storwize V7000 behind an SVC? Does this makes sense?
The short answer: YES!
We have a great many customers happily doing this, so I thought I would share some common questions I get around configuration. Firstly there is an InfoCenter page on this which you will find here. Secondly there is a debate about whether we should create individual volume/arrays on the Storwize V7000 or just create a single pool on the Storwize V7000 (which equates to striping on striping). More bench marking is being done to see if one method is truly better than the other, so until then I recommend the method described below. If you have already done stripe on stripe, don't go changing anything until I update this post.
How many ports should I use for Zoning?
The Storwize V7000 has 8 Fibre Channel ports, 4 from each node canister. You need to zone at least two ports from each node canister to your SVC cluster. This is no different to how you would zone a DS5100 or an EMC VNX.
How will the SVC detect the Storwize V7000?
On the SVC you will see two storage controllers, one for each node canister. This is quite normal. The reason for this is that each node canister reports its own WWNN. This is not a problem and will not affect volume failover if one node canister goes offline.
In the example below the SVC has detected two new controllers. The confusing factor is that both report as 2145s, but they are a Storwize V7000. Rename them to reflect what they really are (something like StorwizeV7000_1_Node1 and StorwizeV7000_1_Node2).
How should I define the SVC on the Storwize V7000?
You need to create a new host on the Storwize V7000 and call it something like SVC_1. if the SVC WWPNs don't appear in the WWPN dropdown, you will need to manually add them as shown below:
You can get the SVC WWPNs from your existing zoning or by doing an svcinfo lsnodeagainst each SVC node or display them in the SVC GUI as shown below:
What size Storwize V7000 volumes should I create?
My recommendation is to do the following on the Storwize V7000
Create arrays of preferably 8 disks in size. The ideal number will depend on how many disks you have. On my machine I have 22 disks, so I create three arrays each with seven disks (and one hot spare):
Create one pool for each array:
Create one volume out of each pool (using all space in the pool).
Define the SVC to the Storwize V7000 as a host (as described above) and map all volumes to the SVC.
On the SVC detect all the Storwize V7000 LUNs as MDisks and create one pool.
Now you should have a pool on the SVC that you can use to create volumes to present to your hosts. They will be striped by default, which is exactly what you want.
Hopefully all of this makes sense. Questions and comments very welcome.
I know the walls are coming down... but there are still many organizational barriers that can exist in IT. How about:
The Networking team (who may possibly be allied to or at war with the firewall/security team)
The Storage admin team (possibly split between open and System z)
The Systems admin teams (possibly split between System z and open and then split again into Windows and Unix or VMware and Unix)
The Applications admin teams (don't get me started on how those guys and gals can get split up)
The Security team
Team work and co-operation? Sure it's an option.... but then an option means its optional.... right?
So when vendors come along with plug-ins and products that dare to connect two worlds... is this a unifying force, or is it anti-matter, or do they just get ignored and not used?
A case in point being the IBM Storage Management Console for VMware vCenter which you can download from here. I have written about this plug-in before, but with the release of version 2.6 (that supports vSphere 5.0), I thought I would try something out. Installing the plug-in potentially offloads a lot of storage management from the storage admin to the VMware admin. But what if the storage admin does not WANT to offload this work?
The answer is to give the VMware admin read-only access.
When you configure your IBM storage device to the plug-in, you supply the plug-in with log-in credentials (so it can log into your IBM storage device and collect the required information). If the user-id supplied only has read-only access to the XIV for instance, the plug-in still works... but not for any operations that change resources. You cannot see the pools on the XIV, but you can still see your volumes and any snapshots that have been created (but annoyingly you cannot see mirrors).
This does have one big advantage. You can clearly match the VMware datatstore name to the XIV volume name. You can also identify which XIV supplied the volume.
This is must for large installations regardless of what storage admin tasks (if any) you want to allow the VMware admin to perform.
I also tested this with Storwize V7000 with a user in the Monitor category and got pretty well the same results. A nice bonus is that I could also see the state of the mirrors as well as the flashcopies. In the example below, all of this information would normally not be visible to the VMware admin, so this is very handy stuff.
Of course I get to also visit one-man bands where the same (exhausted) individual manages the VMware servers, the Operating System Guests, the Network, the Firewall, the Exchange server, the SQL servers and pretty well everything else including getting the elevators and coffee machine fixed. For those people, they need all the help they can get.
One of the things I like about creating host definitions on XIV, Storwize V7000 or SVC is that for the vast majority of host types, the default choice is the correct choice. This saves me having to specifically tell the storage what operating system each host is running.
However in recent times more and more choices have snuck in. For instance in 220.127.116.11 code the Storwize V7000 and SVC had to add an extra option for OpenVMS hosts (there is an alert related to this here). Fortunately most of these exceptions are (from my perspective) for relatively obscure operating systems (users can start spamming me now).
So you need to warn the XIV if your host is running HP-UX or zVM:
On the SVC or Storwize V7000 you need to change from default if your host is running HP-UX or OpenVMS. There is also TPGS (or Target Port Group Support) for use with Apple OS and Solaris hosts using MPxIO. For more details visit this link for Solaris and this linkfor Apple.
Why does this requirement exist? Or put differently, why can't the storage be smart enough to just detect the host OS and save the storage admin this bother? Sadly it all has to do with how the hosts issues and responds to SCSI commands.
HP-UX for instance uses flat space addressing (rather than peripheral device addressing). If the storage does not respond in the expected way it will see only 8 LUNs per port (which is not many). I have seen some strange things when HP-UX hosts have not been defined correctly. I have seen even stranger things when non HP-UX have been defined as HP-UX!
OpenVMS on the other hand gets very upset if the storage reports itself compliant with a SCSI specification higher than SPC2 (which is now a very old SCSI specification).
The good news is that for AIX, Linux, Windows and VMware hosts, default works just fine.
I love PuTTY, it is one of my favourite pieces of open source software. For those who don't know what PuTTY is, it is a free implementation of Telnet and SSH for Windows and Unix platforms. Personally I use PuTTY for connecting to Storwize V7000s, SVCs, SAN switches and Unix servers in my lab and at clients.
Being someone who does implementation services, I want to always be sure about what I do, what I did and what affect it had. So keeping logs of every time I connect to any device is very important to me. I want to be able to go back in time to any point at any client I have ever worked with (sounds like time travel, but hopefully you get the idea).
I achieve this by changing the default behaviour of Putty so that every new Session I save uses my preferred default settings. Note all the screen captures below use version 0.61 of PuTTY (which came out July 2011).
First start up PuTTY. We are going to change the Default Settings (which I have highlighted):
Now select Logging from the left hand panel and change all the indicated settings (2, 3 and 4). The folder I use is C:\PuttyLogs but you could of course choose a different one. Note the &H_&Y&M&D&T means the session file name will be the host name, year, month, day and time for that session.
Now go back to Session, highlight Default Settings and hit Save:
Now every new session you open and/or save will automatically log your session output to the specified folder. Existing Saved Sessions will not be changed. You will need to update each of these separately.
The Storwize V7000 is a modular storage system. This means you buy it in enclosures that are two rack units (2U) in size. The first enclosure you buy is a control enclosure which contains two node canisters and two power supplies (with integrated batteries).
The purpose of the batteries (that are only found in the control enclosure) is to protect the node canisters from brown-outs (so they can ride through sudden drops in input voltage) and to give the node canisters time to gracefully shutdown should power be lost. Any data in the cache of the Storwize V7000 will be persistently written to the internal solid state drives. So the batteries are only needed if power is lost and only used while the nodes are shutting down. The Storwize V7000 Information Center contains a great write-up on how this works here.
In the image below you can see a control enclosure battery unit on the right, with an arrow to show where it fits into a control enclosure power supply.
On August 9, 2011 IBM announced that the feature code (8001) for the Storwize V7000 battery was going away without replacement. This left some people a little bit concerned as to what was happening with the batteries. We don't want to have a Storwize V7000 without batteries! The answer is that IBM had separate feature codes for the control enclosure power supplies (Feature Code 9801) and the batteries (Feature Code 8001). Since you cannot have one without the other, IBM have decided to remove one of the feature codes (the battery feature code). Thus this change is purely an administration change and the machine will continue to ship with two batteries. Phew!
The IBM Storage Tier Adviser Tool (known as STAT for short) is a clever piece of software that lets you predict how much business value you would get from adding SSDs to your Storwize V7000, SVC, DS8700 or DS8800. This is because you can add SSDs to all of these products and then have hot spots dynamically and automatically migrated to SSD using IBM's Easy Tier technology (which is offered as a no charge feature).
For clients who have not yet purchased SSDs, or who are unsure which Storage Pools to deploy them into, the STAT tool will help with decision-making.
I recently struck a rather simple problem with the STAT tool after installing it: I kept getting a CMUA00007E error. I downloaded the tool from here and installed it successfully onto my Windows 2008 64-bit lab machine (running on a IBM x3850). The install went fine so I then proceeded to download the heatmap from my Storwize V7000. The heatmap file is automatically generated by Easy Tier and is used as an input file for the STAT tool. You can see an example of where to find the heatmap file in the screen capture below:
I then placed the heatmap into the same folder as the STAT tool and tried to generate a report. It failed with this rather annoying message:
C:\Program Files (x86)\IBM\STAT>stat dpa_heat.78G01A6-2.110823.072309.data
CMUA00007E The STAT.exe command failed to produce the heat distribution output.
I initially thought I had a bad heatmap, but since it is a binary file, opening it in a text editor did not tell me anything.
Actually the issue was simple: I did not have write authority to that folder. To get around this I instead started the command prompt as Administrator:
I then re-ran the command:
C:\Program Files (x86)\IBM\STAT>stat dpa_heat.78G01A6-2.110823.072309.data
CMUA00019I The STAT.exe command has completed.
Having run the tool I was now able to open the index.html in the STAT folder and see much hot data I have in my lab. Turns out that I don't actually have any hot data right now! Don't tell my manager though, he might try to take my SSDs away. #
Having run the tool once, I did not need to use this trick again. It now runs without starting the command prompt as Administrator.
IBM today announced two new disk choices for the Storwize V7000 in the 2.5-inch Small Form Factor:
300 GB 15000 RPM SAS 2.5-inch disk drive
1 TB 7200 RPM Nearline (NL) SAS 2.5-inch disk drive
This means there are now a total of 8 disk choices for Storwize V7000:
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
2.5” Small Form Factor (SFF)
3.5” Large Form Factor (LFF)
You can add any of these to an existing machine (provided you have the space). Planned availability date is August 26, 2011 but orders can be placed right now. More details can be found in the announcement letter here.
One nice thing that the Announce letter does not mention is that you will NOT need to do a firmware update to use these new disks. All you need to do is plug them in.
IBM have offered Brocade switch modules for their BladeCenters for many years. One common question I get asked regards which of these Switch Modules are supported with the IBM Storwize V7000 or IBM XIV.
Your first port of call is the System Storage Interoperation center or SSIC. However that will only list the switch modules by the IBM Feature Part Number (for example 26K5601), which may confuse you. So to help you determine which switch module you possess, here is a history of Brocade SAN switch modules for IBM BladeCenter:
2 Gbps Brocade Switch Modules for IBM BladeCenter
There were two switches 26K5601 and90P0165 but they are actually the same switch with different software features. The Enterprise version had extra licenses like Trunking and Extended Fabrics. These products were withdrawn from marketing on May 26, 2006.
4 Gbps Brocade Switch Modules for IBM BladeCenter
IBM offered two models which were physically identical. You could upgrade from the 10 port to the 20 port with a software activation key. These products were withdrawn from marketing on December 31, 2010.
IBM Product Name
Brocade 10-port SAN Switch Module for IBM eServer™ BladeCenter
Brocade 20-port SAN Switch Module for IBM eServer BladeCenter
3 external 7 internal
6 external 14 internal
IBM Feature Part Number
Brocade Product Number
Brocade Switch Type Numer
IBM FRU Number
Brocade ASIC Type
Minimum FOS Version
Maximum FOS Version
8 Gbps Brocade Switch Modules for IBM BladeCenter
There are actually three models available but I have collapsed them down to two. The 20 port switch is offered as both Enterprise and non-Enterprise. The 42C1828 switch module is the Enterprise version that has more licensed software features such as Trunking, Fabric Watch and Extended Fabrics (and is indicated with an *).
IBM Product Name
Brocade 10-port 8Gb SAN Switch Module for IBM BladeCenter
Brocade 20-port 8Gb SAN Switch Module for IBM BladeCenter
3 external 7 internal
6 external 14 internal
IBM Feature Part Number
Brocade Product Number
Brocade Switch Type Numer
IBM FRU Number
Brocade ASIC Type
Minimum FOS Version
Maximum FOS Version
At the moment the SSIC does not list every switch module. However support is available for many configurations via the IBM SCORE request system (sometimes called an RPQ). Your IBM pre-sales storage specialist can raise one of these. Depending on your request you may get a support statement as quickly as overnight.
If your wondering what an IBM FRU number is: a FRU or Field Replacement Unit is the part number used in the IBM spare parts system to replace your part under warranty or maintenance agreement.
If you want the information above in a spreadsheet format, you can find it here.
VMware vSphere 4.1 brings in a brilliant new function to offload storage related workload. This function is called VAAI (vStorage APIs for Array Integration) and requires that your SAN storage supports VAAI and that your ESX or ESXi server has a driver installed to utilize it.
IBM first supported VAAI with the IBM XIV using an IBM supplied VAAI driver. IBM then added support to the Storwize V7000 and SVC, so IBM has now released a new VAAI driver to support all three products at once. You can find the driver, installation guide and release notes at this URL.
I discovered some quirks in the process to update the IBM VAAI driver from version 18.104.22.168 to version 22.214.171.124 on VMware ESXi. The benefit in moving to version 126.96.36.199 is that the updated driver supports both the IBM XIV as well as the Storwize V7000 and IBM SVC.
I downloaded the new driver from here and which uses the following naming convention:
Version 188.8.131.52 is named IBM-ibm_vaaip_module-268846-offline_bundle-395553.zip Version 184.108.40.206 is named IBM-ibm_vaaip_module-268846-offline_bundle-406056.zip Version 220.127.116.11 is named IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
The last 6 digits in the file name is what differentiates them. However when I ran the --query command against an ESXi box, I got confused:
Both the uplevel and downlevel VAAI driver files start with: IBM-ibm_vaaip_module-268846 So which one is installed? The 18.104.22.168 version or the 22.214.171.124 version? I ran the following command to confirm if the updated bulletin applies (the one ending in
613937 ). This confirmed my ESXi server was using version 126.96.36.199 and needed an upgrade to version 188.8.131.52.
vihostupdate.pl --server 10.1.60.11 --username root --password passw0rd --scan --bundle IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
The bulletins which apply to but are not yet installed on this ESX host are listed.
---------Bulletin ID--------- ----------------Summary-----------------
IBM-ibm_vaaip_module-268846 vmware-esx-ibm-vaaip-module: ESX release
To perform the upgrade I first used vMotion to move all guests off the server I was upgrading. I then placed the server in maintenance mode and installed the new driver:
There are no commands needed to activate VAAI or claim VAAI capable devices in ESXi. You simply need to confirm that the both boxes shown in the example below have the number 1 in them (for hardware accelerated move and for fast init):
To test VAAI I normally do a storage migration (storage vMotion) moving a VMDK between datastores on the same storage device. What you should see is very little VMware to Storage I/O, as I depicted in this blog post and this blog post.
My colleague Alexandre Chabrol from Montpellier Benchmarking Center also helped me out with the ESXCLI commands to control VAAI. We can confirm the state of each of the three VAAI functions and switch them off and on. We use the -g switch to display them, the -s 0 switch to turn them off and the -s 1 switch to turn them on. In this example I first confirm that VAAI is active for hardware accelerated moves, hardware accelerated inititialization (write zeros) and then hardware assisted locking. I then disable and enable hardware accelerated moves.
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedInit
Value of HardwareAcceleratedInit is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /VMFS3/HardwareAcceleratedLocking
Value of HardwareAcceleratedLocking is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 0 /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 0
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 1 /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 1
Final thought: Most if not all of these commands can be done via the vSphere Client GUI, you do not need to use CLI. But I am surprised how many people like to use the CLI and want to see example syntax. Got a preference yourself? Love to hear about your experiences.
*** Update February 20, 2012 ***
The IBM Storage Device Driver for VMware VAAI was updated to version 184.108.40.206 in February 2012. This new version fixes a rare case where XIV, Storwize V7000, or SVC LUNs are not claimed by the IBM Storage device driver. If you are using version 220.127.116.11 without issue, there is no need to upgrade. I have updated this post to reflect the new version.
Here is a little test. Check your documentation: Do you know how to power down and power up the equipment in your computer room? If you had to power off your site in a hurry would you know how? If you wanted to script a shutdown, could you do it?
Here are some hints and tips that might help you with some of my favourite products:
The process to power up or down your DS8000 is documented in the Information Center here.
If you want to script powering off a DS8000 storage unit you can use thechsu -pwroff DSCLI command. This command will shut down and power off the specified DS8000 unit. Be careful before powering off the unit to ensure that all I/O activity has been stopped. An example of the command is shown below. Your machine will have a different IP address, password and serial number to mine. Note the serial number always ends in zero because we send the command to the storage unit.
The IBM Storage Management Console for VMware vCenter version 2.5.1 is now available for download and install. This version supports XIV, SVC and Storwize V7000 as per the versions on the following table (the big change being support for version 6.2):
If you want to see a video showing the capabilities of the new console, check out this link.
After installing the console, you will get this lovely new icon:
Start it up and select the option to add new storage, you now get three choices:
If your using SVC or Storwize V7000 you need to specify an SSH private key. This key MUST be in Open SSH format. This caused me a problem as I kept getting this message when trying to add my Storwize V7000 to the plug-in:
Unable to connect to 10.1.60.107. Please check your network connection, user name, and other credentials.
I could use the same IP address, userid and SSH private key to logon to the Storwize V7000 using putty, so I knew none of these things were wrong.
I reread the Installation Instructions closely and realized my mistake. It clearly states:
Important: The private SSH key must be in the OpenSSH format.
If your key is not in the OpenSSH format, you can use a certified
OpenSSH conversion utility.
I pondered what conversion utility I could use when I realized I had the utility all the time:Puttygen. I opened PuttyGen, imported my private key (the .ppk file) and exported my SSH private key using OpenSSH format. You don't need to do anything with the public key.
I was then able to add the Storwize V7000 by specifying the private SSH key exported using OpenSSH format.
Now I have both IBM XIV and Storwize V7000 in the vCenter plug-in and can get detailed information about and manipulate both. In this example I have highlighted the Storwize V7000, revealing it is on 18.104.22.168 firmware.
I was tempted to detail all the many things you can do with the plug-in, but your better off watching the video via this link.
So are you using the plug-in? Have you upgraded to version 2.5.1 yet? Comments very welcome!
I have received this question several times, so it's clearly something people are interested in.
The Storwize V7000 has two controllers known as node canisters. It's an active/active storage controller, in that both node canisters are processing I/O at any time and any volume can be happily accessed via either node canister.
The question then gets asked: what happens if a node canister fails and can I test this? The answer to the question of failure is that the second node canister will handle all the I/O on its own. Your host multipathing driver will switch to the remaining paths and life will go on. We know this works because doing a firmware upgrade takes one node canister offline at a time, so if you have already done a firmware update, then you have already tested node canister fail over. But what if you want to test this discretely? There are four ways:
Walk up to the machine and physically pull out a node canister. This is a bit extreme and is NOT recommended.
Power off a node canister using the CLI (using the satask stopnode command). This will work for the purposes of testing node failure, but the only way to power on the node canister is to pull it out and reinsert it. This is again a bit extreme and is not recommended. This is also different to an SVC, since each SVC has it's own power on/off button.
Use the CLI to remove one node from the I/O group (using the svctask rmnode command). This works on an SVC because the nodes are physically separate. On a Storwize V7000 the nodes live in the same enclosure and a candidate node will immediately be added back to the cluster, so as a test this is not that helpful.
Place one node into service state and leave it there will you check all your hosts. This is my recommended method.
First up this test assumes there is NOTHING else wrong with your Storwize V7000. We are not testing multiple failure here. You need to confirm the Recommend Actions panel as shown below, contains no items. If there are errors listed, fix them first.
Once we are certain our Storwize V7000 is clean and ready for test, we need to connect via the Service Assistant Web GUI. If you have not set up access to the service assistant, please read this blog post first.
So what's the process?
Firstly logon to the service assistant on node 1 and place node 2 into service state. I chose node 2 because normally node 1 is the configuration node (the node that owns the cluster IP address). You need to confirm your connected to node 1 (check at top right) and select node 2 (from the Change Node menu) and then choose to Enter Service State from the drop down and hit GO.
You will get this message confirming your placing node 2 into service state. If it looks correct, select OK.
The GUI will pause on this screen for a short period. Wait for the OK button to un-grey.
You will eventually get to this with Node 1 Active and Node 2 in Service.
Node 2 is now offline. Go and confirm that everything is working as desired on your hosts (half your paths will be offline but your hosts should still be able to access the Storwize V7000 via the other node canister).
When your host checking is complete, you can use the same drop down to Exit Service State on node2 and select GO.
You will get a pop up window to confirm your selection. If the window looks correct, select OK.
You will get the following panel. You will need to wait for the OK button to become available (to un-grey).
Provided both nodes now show as Active, your test is now complete.
Over at SearchStorage.com.AU they recently published an article entitled Six reasons to adopt storage virtualisation. You can find the article here. The six given reasons are:
Storage virtualisation reduces complexity
Storage virtualisation makes it easier to allocate storage
Better disaster recovery
Better tiered storage
Virtual storage improves server virtualisation
Virtual storage lets you take advantage of advanced virtualisation features
Its a well written article and I agree with every point. But one could be forgiven for reading the article and thinking that either storage virtualisation is new, or that storage virtualisation is something you might consider AFTER doing server virtualisation. Both of which are not true.
IBM embraced storage virtualisation in June 2003 when we announced our SAN Volume Controller (the IBM SVC). I even found a CNET.com article from way back then. You can find it here (the image below is a screen capture of that CNET website).
IBM's SVC product has been enhanced repeatedly since 2003 with an enormous list of supported host servers and backend storage controllers. We have added new functions every year including Easy Tier, split cluster, VAAI, an enhanced GUI and a new form factor for the SVC code in the form of the Storwize V7000.
So let me give you a seventh reason for adopting storage virtualisation: A vendor who has shown genuine support for this technology. No vendor has embraced storage virtualisation with more enthusiasm than IBM. We have an industry leading solution with phenomenalSPC benchmarks, an enormous number of case studies and an architecture that does not lock you in. Indeed it is an architecture that can grow as you grow and that can be upgraded without disruption.
So please consider storage virtualisation from IBM, using either the SVC or the Storwize V7000. If your in Australia, we have demo centers dotted around the country. Many of our Business Partners can also demonstrate IBM storage virtualisation using their own Storwize V7000s. If your in Melbourne feel free to give me a call and schedule a time to drop into Southgate.
The Storwize V7000 and SVC release 6.1 introduced a new WEB GUI interface to assist with service issues, known as the Service Assistant. The Service Assistant interface is a browser-based GUI that is used to service your nodes. Much of what you traditionally did with the SVC front panel can all be done using the Service Assistant GUI. You can see a screen capture of the Service Assistant below:
While I would like to be optimistic and hope that you will never have to use the Service Assistant, you should always ensure your toolkit is equipped with every possible tool. I say this because one thing I have noted is that the majority of installs are not configuring the Service Assistant IP addresses. This is particularly apparent as clients upgrade their SVC clusters to release 6.1.
By default on Storwize V7000, the Service Assistant is accessible on IP addresseshttps://192.168.70.121 for node 1 and https://192.168.70.122 for node 2 (don't try and point your browser at them right now, as your network routing won't work - you would need to set your laptop IP address to the same subnet and be on the same switch. Details to do that are here). For SVC there are no default IP addresses, although we traditionally asked the client to configure one service address per cluster. The best thing for you to do is approach your network admin and ask for two more IP addresses for each Storwize V7000 and/or SVC I/O group. Once you have these two extra IP addresses, record them somewhere and then set them using the normal GUI.
Its an easy five step process as shown in the screen capture below. Go to the Configuration group and then choose Network (step 1). From there select Service IP addresses (step 2) and the relevant node canister (step 3). Choose port one or port two (step 4) and then set the IP address, mask and gateway (step 5).
You can also set them using CLI (replace the word panelname with the panel name of each node, which you can get using the svcinfo lsnode command).
If you forget these IP addresses, you can reset them using the same CLI commands or using the Initialization tool as documented here.
Finally having set the IP addresses, visit the service assistant by pointing your browser at each address. This is just to confirm you can access it. You logon with your Superuser password. With the process complete, ensure the IP addresses are clearly documented and filed away. So now if requested, you will be able to perform recovery tasks (in the unlikely chance they are needed). If for some reason your browser keeps bringing you to the normal GUI rather than the Service Assistance GUI, just add /service to the URL, e.g. browse to https://10.10.10.10/service rather than https://10.10.10.10.
So what should you do now?
If your an SVC customer on SVC code version v5 and below, please get two IP addresses allocated for each SVC I/O group, so you can set them the moment you upgrade to V6. Do this once the upgrade is complete.
If your an existing Storwize V7000 client or an SVC client already on V6.1 or V6.2 code, then hopefully you should already have set the service IP addresses. If not, please do so and test them.
I recently got a great email from an IBMer in the Netherlands by the name of Jack Tedjai. He sent me two screen shots, taken with the new performance monitor panel (that comes with the SVC and Storwize V7000 6.2 code). He wrote:
I am working on a project to migrate VMware/SRM/DS5100 to SVC Stretch Cluster and one of the goals is to prevent using ISL (4Gbps) and VMware Hypervisor/HBA load during the migration. For the migration we are using VMware Storage vMotion. To minimize the impact of the migration on production, we tested VAAI for Storage vMotion and template deployment and it worked perfectly.
So whats this all about? Well one of the improvements provided with VAAI support is the ability to dramatically offload the I/O processing generated by performing a storage vMotion. Normally a storage vMotion requires an ESX server to issue lots of reads from the source datastore and lots of writes to the target datastore. So there is a lot of I/O flowing from ESX to the SVC, and then from the SVC to its backend disk. What you get is something that looks like the image below. In the top right graph we have traffic from SVC to ESX (host to volume traffic). In the bottom right graph we have traffic from the SVC to its backend disk controllers (DS5100 in this case). This is SVC to MDisk traffic.
When we add VAAI support to the SVC, we suddenly change the picture. Suddenly VMWare does not need to do any of the heavy lifting. There is almost no I/O between VMWare and the SVC (no host to SVC volume traffic) related to the vMotion. The SVC is still doing the work, but it is happening in the background without burning VMWare CPU cycles or HBA ports (in that there is still SVC to MDisk traffic).
This difference translates to: Faster vMotion times, far less SAN I/O and far less VMware CPU being used on this process.
So do VMware support this? They sure do! Check this link here. It currently shows something like this (taken on June 23, 2011):
So what are your next steps?
Upgrade your Storwize V7000 or SVC to version 6.2 code. Download details arehere.
Download and install the VAAI driver onto your ESX servers. You can get it from here. If your already using the XIV VAAI driver you need to upgrade from version 22.214.171.124 to version 1.2. There is an installation guide at the same link.
And the blog title? It means friendly greetings in Dutch. So to Jack (and to all of you), vriendelijke groeten and please keep sending me those screen captures.
I thought I would quickly check out two of the announced features of the 6.2 release: the new Performance Monitor panel and support for greater than 2 TiB MDisks. So on Sunday I got busy and upgraded my lab Storwize V7000 to version 126.96.36.199.
Remember that in nearly every aspect the firmware for the SVC and Storwize V7000 are functionally identical, so while I am showing you a Storwize V7000, it equally applies to an SVC.
Firstly I tried the performance monitor panel, and what better way to show you what I saw than on YouTube? This is my first YouTube video so please forgive me if its not slick. I started the performance monitor and captured two minutes of performance data using Camtasia Recorder. Because it is fairly boring to stare at graphs slowly moving right to left, I then sped it up eight times, and this is the result:
The video is shot in HD, so if what your seeing is grainy or hard to read, change the display to 720p or 1080p. Now if you want to see the performance monitor at its actual speed, here is the original normal speed video. Remember this is the same video as above, just slower. It can also be viewed in 720p.
The top right hand quadrant is volume throughput in MBps as well as current volume latency and current IOPS.
The bottom left hand quadrant is Interface throughput (FC, SAS and iSCSI).
The bottom right hand quadrant is MDisk throughput in MBps as well as current MDisk latency and current IOPS.
You will note that each metric has a large number (which is the current metric in real time) and a historical graph showing the previous five minutes. You can also change the display to show either node in the I/O group.
I found the monitor to be genuinely real time: the moment I changed something in the SAN (such as starting or stopping IOMeter or starting or stoping a Volume Mirror), I immediately saw a change.
Greater than 2 TB MDisk support
Next I logged onto my lab DS4800 and created two 3.3TiB volumes to present to the Storwize V7000. I chose this size because I had exactly 6.6 TiB worth of available free space on the DS4800 and I wanted to demonstrate multiple large MDisks. On versions 6.1 and below, the reported size of the MDisks would have been 2 TiB (as I discussedhere). Now that I am on release 6.2 with a supported backend controller, I can present larger MDisks. In the example below you can clearly see that the detected (and useable size) is 3.3 TiB per MDisk.
What controllers are supported for huge MDisks?
The supported controller list for large MDisks has been updated. The links for Storwize V7000 6.2 are here and for SVC here. If your backend controller is not on the list, then talk to your IBM Sales Representative about submitting a support request (known as an RPQ).
There was a time when 32 bits was considered a lot. A hell of a lot.
With 32 bits, you can create a hexadecimal number as big as 0xFFFFFFFE (presuming we reserve one bit). In decimal that's 4,294,967,295. Hey... imagine a bank account balance that big? If you use 32 bits to count out 512 byte sectors on a disk, you could have a disk that's 4,294,967,295 times 512... or 2,199,023,255,040 bytes! That's sounds huge, right?
Well... actually...no... that's 2 TiB, which most people would refer to as 2 Terabytes. Mmm.. Suddenly I am less impressed (still wouldn't mind that as a bank account though).
Now there are plenty of running Systems that still cannot work with a disk that is larger than 2 TiB. One of the more common is ESX. I am presuming this limitation is going to disappear, so Storage susbsystems need to be ready to create volumes that are larger than 2 TiB.
The good news is that with the May 2011 announcements, IBM is removing the last 2 TiB sizing limitations from its current storage products. There appears to have been some confusion in the past, so I thought I would go through and be clear where each product is at:
Firmware version 07.35.41.00 added support to create volumes larger than 2 TB. The maximum volume size is limited only by the size of the largest array you can create. This capability has been available for some time and hopefully you are already on a much higher release.
DS4000 and DS5000
Firmware version 07.10.22.00 added support to create volumes larger than 2 TB. The maximum volume size is limited only by the size of the largest array you can create. This capability has been available for some time and hopefully you are already on a much higher release.
DS8700 and DS8800
The DS8700 and DS8800 will support the creation of volumes larger than 2 TB once a code release in the 6.1 family has been installed. With this release you will be able to create a volume up to 16 TiB in size. The announcement letter for this capability is here.
The volume size on an XIV is limited only by the soft limit of the pool you are creating the volume in. This allows the possibility of a 161 TB volume.
SVC and Storwize V7000
These two products have two separate concepts:
Volumes (or VDisks) that hosts can see.
Managed disk (or MDisks) that are presented by external storage devices to be virtualized. Within this there are two further categories: - Internal MDisks created using the Storwize V7000 SAS disks. - External MDisks created by mapping volumes from external storage (such as from a DS4800).
SVC and Storwize V7000 Volumes (VDisks).
Prior to release 5.1 of the SVC firmware, the largest volume or VDisk that you could create using an SVC was 2 TiB in size. With the 5.1 release this was raised to 256 TiB, as announced here. When the Storwize V7000 was announced (with the 6.1 release) it also inherited the ability to create 256 TiB volumes.
Because the Storwize V7000 has its own internal disks, it can create RAID arrays. Each RAID array becomes one Mdisk. This means the largest MDisk we can create is limited only by the size of the largest disk (currently 2 TB), times the size of the largest array (16 disks). This means we can make arrays of over 18 TiB in size (using a 12 disk RAID6 array with 2 TB disks). Thus internally the Storwize V7000 supports giant MDisks. We can also present these giant MDisks to an SVC running 6.1 code and the SVC will be able to work with them.
SVC and Storwize V7000 External Managed Disks.
When presenting a volume to the SVC or Storwize V7000 to be virtualized into a pool (a managed disk group) we need to ensure two things are confirmed. Firstly you need to be on firmware version 6.2 as confirmed here for SVC and here for Storwize V7000. Secondly that the controller presenting the volume has to be approved to present a volume greater than 2 TiB. From an architectural point of view, MDisks can be up to 1 PB in size as confirmed here, where it says:
Capacity for an individual external managed disk
Note: External managed disks larger than 2 TB are only supported for certain types of storage systems. Refer to the supported hardware matrix for further details.
I recommend you go to the supported hardware matrix and confirm if your controller is approved. The links for Storwize V7000 6.2 are here and for SVC here. As of this writing, the list has still not been updated, but I am reliably informed it will include the DS3000, DS4000, DS5000, DS8700 and DS8800. It will not initially include XIV, which will come later. Please also note the following:
Support for giant MDisks (greater than 2 TiB) is firmware controlled. If the controller (e.g. a DS5300) presenting a giant MDisk is not on the supported list for your SVC/Storwize V7000 firmware version, then only the first 2 TiB of that MDisk will be used.
If your already presenting a giant MDisk (and using just the first 2 TiB), then just upgrading your SVC/Storwize V7000 firmware won't make the extra space useable. You will need to remove the MDisk from the pool, then do an MDisk discovery and then add the MDisk back to the pool. All of this can of course be done without disruption, using the basic data migration features we have supported since 2003.
What to do in the meantime?
If your currently using an SVC or external MDisks with a Storwize V7000, then you need to work within the 2 TiB MDisk limit (except for Storwize V7000 behind SVC). The recommendation is a single volume per Array for performance reasons (so the disk heads don't have to keep jumping all over the disk to support consecutive extents on different parts of the disk). This can require careful planning. For instance using 7+P RAID5 Arrays of 450 GB drives makes an array that is over 3 TB. What to do in this example?
Divide it in half? (by creating two 1.5TB volumes)
Waste space? (a whole 1 TB)
Use smaller arrays? (a 4+P array of 450GB disks is 1.8 TB)
The answer is that where possible, create single volume arrays using 4+P or larger. If the disk size precludes that, then create multiple volumes per array and preferably split these volumes across different pools (MDisk groups).
Anything else to consider?
Well first up, will your Operating System support giant volumes? Googling produces so much old material that it becomes hard to nail down exact limits. For Microsoft, read this article here. For AIX check out this link. For ESX, check out this link.
Second of course is the consideration of size. File systems that utilize the space of giant volumes could potentially lead to giant timing issues. How long will it take to backup, defragment, index or restore a giant file system based on a giant volume (the restore part in particular)? Outside the scientific, video or geo-physics departments, are giant volumes becoming popular? Are they being held back by practical realities or plain fear? Would love to hear your experiences in the real world.
And a big thank you to Dennis Skinner, Chris Canto and Alexis Giral for their help with this post.
Hi Team! Just wanted to let everyone know that VisioCafe has been updated with IBM's latest official stencils for use with Microsoft Visio. These include all models of the Storwize V7000, including the newest models: The 2076-312 and 2076-324 (which have the dual port 10 Gbps iSCSI card).
Its a story told many times..... You order a new storage solution and the world is good. It's lovely, it's new and it offers mountains of new disk space.... but then... you.... fill it up! So its off to order some new disks. The order is in, the order is filled, the disks arrive. What next? How about we just stick them in? By just inserting the new disks, they will be made available to configure into RAID arrays from the Internal tab of the Physical Storage Group.
If the drives are showing as Unused, mark them to be Candidate. If they are already showing as Candidate (like most of the disks in my example below), then you are ready to hit the Configure Storage button and follow the guidance of the Wizard.
Of course maybe your enclosures are all full. In this case it's time to order another enclosure (remember we can have up to 10). Once you have racked the enclosure up and cabled the new enclosure to the correct SAS Chain, then use the Add Enclosure menu item shown below to kick off the configuration:
Storage IT offers up many choices, some of which provoke argument so heated, you could almost describe the adherents as religious. I think you might know the sort of arguments I am talking about:
File vs Block I/O
iSCSI vs Fibre Channel
CLI vs GUI
OK.... so maybe that last one isn't quite in the same league. But it is still fascinating to see the variation in usage patterns from sites where every command (of any description) is run via a command line interface (a CLI), to sites where the CLI is viewed with either great fear... or even greater distaste. There are those who view the CLI as... well... so 1970s.....
But the reality is that the CLI will always be with us for one principal reason: scripting. If you cannot script it, you cannot automate it (well actually thats not true, but stick with me here, I am on a roll). Every single major implementation I have ever done (whether it be SVC, XIV, DS8000), I have automated with scripting. I regularly use the concatenatecommand in Excel to build large numbers of commands that I can then run as a script.
So its pleasing to see that all of our products are working towards making the scripters life even easier. For example the XIV has offered a command log in the GUI for some time. I blogged about it here. You simply do a command once in the GUI and then consult the log to find the syntax, making scripting very easy:
With last years release of SVC 6.1 and Storwize V7000, we added this level of smarts to those two products as well. Now every command you run in the GUI will offer you the exact CLI command that was used under-the-covers to do this work. Simply toggle the details tab on the completion panel to see the command (or toggle it back to hide it!).
This weeks announcement of release 6.2 of the SVC and Storwize V7000 firmware, has brought in two more important usability improvements:
Now when logging onto the CLI using individual user-ids, you can logon using the actual user-id itself, rather than admin. This change has been a long time coming and removes the confusion generated by logging onto the GUI as sayanthony, but then logging into a matching CLI session as admin. Now you would logon to either interface as anthony.
Now when issuing CLI commands, you have the choice to drop the svctask and svcinfo headers. So instead of issuing the command svcinfo lsnode, you can issue the command lsnode. Both choices remain valid (so we don't break your existing scripts). Making this change is part of a bigger plan to move to a more common CLI.
And there are more improvements coming, so as always, watch this space....
.... and please... share with me... are you a GUI... or a CLI person? Whats your reasoning behind your choice?
The May 9 announcement that SVC and Storwize V7000 will support VAAI is very welcome news. The fundamental point is that the SVC and Storwize V7000 virtualise external storage. This means that the mountains of DS3000, DS4000, DS5000, AMS1000s, CX3s, etc, that are currently being virtualized behind these products, will inherit VAAI as soon as the virtualization layer supports it. This is yet another feature to add to the list of functions that IBM Storage virtualization can provide, such as: EasyTier; Thin Provisioning; multiple consistency groups; snapshots; remote mirroring; dynamic data relocation... the list goes on.
In addition we are releasing a plug-in for vCenter that enables VMware administrators to manage their SVC or Storwize V7000 from within the VMware management environment
Functions will include:
Volume provisioning and resizing
Displaying information about volumes
Viewing general information about Storwize V7000 and SVC systems
Receiving events and alerts for Storwize V7000 systems and SVC attached to vSphere
The Storwize V7000 and SVC plug-in for vCenter will also supports virtualized external disk systems
The plug-in will be available at no charge on June 30 (for Version 6.1 software) and July 31 (Version 6.2). Here is a sneak peak of what it will look like:
And to get an independent viewpoint have a read of Stephen Fosketts blog entry here:
With the announced release of DS8000 6.1 code, IBM has moved its three major storage systems to a common GUI platform. This makes me think of aircraft manufacturers who utilize a common cockpit design. For airlines, this is major drawcard when choosing aircraft models. It cuts down on training costs for your pilots. Except in storage IT, there is a major difference in motivation....
First and foremost, the design of the XIV GUI (that has inspired such dramatic change in IBMs other GUIs), was made possible, not by clever XIV GUI developers (don't get me wrong - they ARE clever), but by a remarkably user-friendly architecture. The XIV GUI is a miracle of ease-of-use for end users, made possible because first and foremost, by design, the XIV made it almost impossible to make it hard.
The good news for Storage administrators, is that unlike a jet aircraft, where a pilot needs to spend hundreds of hours in the cockpit before they are considered potentially competent, the XIV GUI can be picked up in minutes and lends itself very well to casual contact. You don't need to keep using it to stay competent.
The challenge for IBM was take more complex products, which require more user decisions, and make the usage experience just as easy. To add to this, the SVC and DS8000 GUIs were driven by WebSphere. Changing these GUIs would require a complete re-write to employ Java script.
First off the rank was the SVC and Storwize V7000. With the release last year of the SVC 6.1 update, the transformation was nothing less than remarkable. End user experience ruled every decision. The key again is that the user does not need to spend hundred of hours learning this GUI or re-learning it every time they go to perform a configuration task. Everything is in its right place. Its much more than an XIV-like GUI. Its a GUI that took the ease of use experience of the XIV and used that to inspire something just as remarkable.
With the release of the 6.1 update for the DS8000, we complete another fundamental step towards a truly common GUI. The DS8000 GUI has undergone a complete re-write. Essentially it has been rebuilt from the ground up. This highlights something fundamental: It confirms the DS8000 has a very strong roadmap.
As you can see from the image below, the transformation from the old design (to the left) to an ease of use model is complete:
In short it a common flight deck, that almost anyone can fly.
A question I get routinely asked relates to Windows disk partition alignment with XIV. If you don't know what I am talking about, take some time to read these very useful pages from our friends at Microsoft. Once you have had a look, come on back and read my perspective.
Back already? Hopefully you now know that disk partition alignment is all about starting an IO at a logical block address that best matches how the underlying hardware stores your data. So now your wondering, what does this have to do with XIV? Well XIV has two concepts that relate to this: cache and partitions.
XIV cache (the server memory used to speed up reads and writes) is organised into 4 KB blocks (which is nice and small).
So the XIV cache does not care about disk alignment.
But when it comes to writing and read from disk, the XIV writes data into chunks of consecutive logical block addresses (LBAs) that we call partitions. These partitions are 1 MiB in size. What does that concept mean? It means the magical number for XIV is 1024 KB or 1 MB. (actually KiB and MiB, but for the sake of ease, I will stick to the naming used by Microsoft. Given this number is fairly large (other hardware often aligns to 32KB, 64KB or 256KB), for XIV this reduces the potential impact of misaligned partitions. Which is good.
Correct Windows Disk Alignment could give up to a 7% performance improvement when using an offset of 1024 KiB. (1 MiB). I need to be clear, that's not a guaranteedimprovement of 7%. It's a maximum possible improvement. Your particular server will see an improvement somewhere between 0% and 7%. It depends on your workload patterns. The more small and random your workload, the more useful setting the 1024 KB offset will be. The more sequential your workload, the less useful it will be, as only the first and last parts of an I/O could potentially be misaligned. This mis-alignment could equate to a tiny percentage of extra work for the XIV. Sadly there is no metric you can display to detect how much impact misalignment is actually having.
So should you do it? The good news is that new volumes created under Windows 2008 prefer the 1 MB boundary. So a fresh install should already be using the correct values. The bad news is that volumes created under earlier Windows Operating Systems (Such as Windows 2000 and 2003) will almost certainly be misaligned, and correcting the alignment is destructive to the data in the partition.
How to check alignment at the host? Here is an example:
I start diskpart:
Microsoft DiskPart version 6.1.7600
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: ANTHONYV-PC
I list my disks. In this example I have two disks installed in my laptop. I select disk 0:
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ------------- ------- ------- --- ---
Disk 0 Online 238 GB 5724 MB
Disk 1 Online 232 GB 1024 KB
DISKPART> select disk 0
Disk 0 is now the selected disk.
Now I list the partitions and see the offset for each one.
Partition 1 has an offset of 1024 KB, which is 1 MB, which is perfect for XIV. Partition 2 has an offset of 101 MB, which is still on the 1MB boundary (it was pushed there by the combination of the size of the first partition (100 MB) and its offset (1 MB). So this is perfect.
The DS8000 (regardless of model), also prefers 64 KB offsets. The DS8000 use the concept of logical tracks where each logical track is 64KB.
The DS3000/DS4000/DS5000 range allow the user to set the segment size of a logical volume on creation. The setting that you define should match the segment size defined for the logical drive being used. In the example below, it is 64 KB.
What about VMWare?
The answers are no different. Misalignment can indeed make a difference to client performance. Check this link from NetApp and this document from VMware:
I searched around looking for an image to highlight the theme of alignment. I found this image in the IBM archives for the IBM Mass Storage Facility announced back in 1974. I am sure this product had some interesting alignment challenges.
(edited 24/5/2011 --> removed old Visio Stencils link).
VisioCafehas been updated with IBM's latest official stencils for use with Microsoft Visio. These include all models of theStorwize V7000, including the newest models: The2076-312and2076-324(which have the dual port 10 Gbps iSCSI card).
When IBM first released the Storwize V7000, we announced it was capable of supporting ten enclosures, but would on initial release support only five. We stated that this restriction would be lifted in Q1.
The good news is that this restriction is indeed now lifted by the release of Storwize V7000 software version 188.8.131.52, which is available for download from here: 184.108.40.206 Code
This new level also contains an additional enhancement which I think users will really like, called Critical Fix Notification. The new Critical Fix Notification function enables IBM to warn Storwize V7000 and SVC users if we discover a critical issue in the level of code that they are using. The system will warn users when they log on to the GUI using an internet connected web browser. It works only if the browser being used to connect to the Storwize V7000 or SVC, also has access to the Internet. (The Storwize V7000 and SVC systems themselves do not need to be connected to the Internet.) The function cannot be disabled (which is a good thing) and each time we display a warning, it must be acknowledged (with the option to not warn the user again for that issue).
The demand for Visio stencils is very strong, which says something about how popular Microsoft Visio is. Its good because it means people are documenting their environments. I am being asked for ETAs for stencils that are not yet in the IBM Disk zip file found on Visio Cafe. The most recent update on Jan 17, 2011, added more nSeries stencils as can be seen here or as reproduced below:
17-Jan-11 IBM-Common.zip IBM-Racks.vss- Added Rack HMC CR5 and CR6 Front and Rear Views - Renamed existing HMC models to follow CR# naming scheme IBM-Disk.zip IBM-SystemStorage-NAS.vss- Added N3400, N6060, N7700 and N7900 Front and Rear Views and Controller Modules - Added EXN3000 Front and Rear Views - Updated N3300 and N3600 Front Open Views with new disk sizes
I am getting a lot of requests right now for the Storwize V7000 stencils. The good news is that I have them. I am unsure why they are not on Visio Cafe yet, so I am posting them here as a zip file. Just right click on the link and save as, then unzip and your good to go. . // Edit Feb 3, 2011 - I changed the file to one with correct IBM naming // . Enjoy.
Its been over a week since the Sydney Storage Symposium finished and I now have the feedback results. The symposium itself was a great success and was a fully booked event (a full house!). We ran 48 separate sessions across two and a half days, all of them focused on storage and how IBM provides solutions for this space (including things like Cloud and DR). These included many lab sessions on SVC, TPC and XIV. Sessions were also given by IBM partners including Brocade and Qlogic. In addition Brocade, Qlogic and LSI each manned a vendor stall to share information about their products. And to top this off, Alexis Giral from the IBM Storage Pre-Sales team in Sydney manned a Storwize V7000 demonstration stand for the entire event. Attendees were able to see, hear, touch and most importantly play with, an actual Storwize V7000.
Feedback was sought from every attendee for every session. Feedback was also sought for the entire Symposium. Of the sessions, the majority scored a net satisfaction rating of 100% (which is an excellent result). The overall satisfaction rating for the conference was 99%. The most popular session was: "Introducing the Storwize V7000"
Some of the feedback given across all of the sessions included:
Great symposium with more value than I expected
The event was extremely well organised
Concise and very knowledgeable, ........ is an extremely good presenter, his use of practical experience to relate to the topic is excellent
Excellent presentation: flowed very well answering questions that previous points raised
Fantastic as always. Thanks.
Outstanding! Excellent presentation! Thank you
Excellent presentation!!! Nothing to improve on.
Excellent speaker & deep understanding of session.
Very engaging speaker.
Had a great session & very interesting topic to attend - thanks.
Excellent - very knowledgeable & enthusiastic.
Fantastic. Thank you!
....a superb presenter with real world experience.
Very knowledgeable presenter able to share real-life situations to illustrate the topic
This has been the most enjoyable & informative session yet. Most of what learned will put in practice - very good session.
Best session ever! Brilliant.
Wow!!! Excellent work - enjoyed the presentation.
Very knowledgeable, fluent & interactive with attendees & fielded questions excellently!
As always great presentation, room was listening very intently throughout presentation
Some of the suggestions given for future symposiums included:
include more "best practice"
include more labs
provide wireless access for attendees
I personally presented four different presentations, which were:
Storage Pools and Easy Tier
SAN Best Practices and Problem Determination
The last session listed (SAN Best Practices) was my favourite. The room was packed to over flowing and the audience was very interactive.
So with all of this feedback and experience under our belts we can begin planning for the 2011 event. In 2011 we will be holding another IBM System Storage event, this time in Melbourne, so watch this space for more updates.
Its been a busy week! We just completed our first IBM System Storage Symposium in Sydney.... and it was a great success. Thanks to everyone who attended and presented. Meanwhile... there are quite a few updates to the IBM Support site, and as usual Rob Jackard from the ATS Group has created a great summary which I will reproduce here. Things of particular note are:
If you installed SVC code version 220.127.116.11 please update to 18.104.22.168 as per this issue
Being a person who walks dogs, visits the gym and uses public transport.... I have plenty of time to listen to podcasts. The main challenge being that while listening to a podcast.... you actually need to LISTEN. On more than one occasion I have zoned out, missed something interesting and suddenly thought... what did he/she just say? One podcast that is a favourite of mine is "Security Now" with Steve Gibson. You can find it here, I highly recommend it (you wont zone out while listening to it). This weeks episode (episode 274) discusses two themes that keep cropping up again and again:
Yet another zero-day vulnerability in Internet Explorer (IE).
The need for important websites to default to HTTPS rather than HTTP (due to the appearance of FireSheep).
In this podcast Steve discusses how to protect yourself against this latest IE vulnerability, including using this technique: "Of course you could also just not use IE, which would be a fantastic solution" I heartily agree with Steve and I was pleased that this year IBM chose to standardise on the Firefox browser (read that story here). One of the stated reasons being that Firefox is more Open Standards compliant. So its nice to see that the new Storwize V7000 (and SVC 6.1) Web based management GUI uses both of these things, in that:
It works perfectly with Firefox (and Google Chrome).
It defaults to HTTPs even if you start the GUI using HTTP
Amusingly if you use IE, you will get this rather cheeky comment on the logon panel.
This makes it a simple, safe and secure GUI that uses industry standard best practice. Please note that you can still choose to use IE... and it will work perfectly. Its just not our recommended Browser. Of course if you use the CLI, it will also be secured using SSH v2 public/private key encryption (as the SVC has always done). . My hearty recommendation of Steve Gibsons work, like everything I express in this blog, is my personal opinion, and not that of my employer.
I had a great time yesterday running a one day seminar for IBM Business Partners on the Storwize V7000. Interest was so strong we had to change locations to get a larger room... and then we had to ask for an even larger room! It was a very positive session with lots of great questions. A really like it when I get questions.... it means people are awake, listening, interested and more importantly THINKING. . Even more exciting: IBM announced today general availability (GA) of the new IBM Storwize V7000 mid-range disk system. We have started shipping across multiple geographies around the world, including: Australia, Bolivia, Denmark, Germany, India, Japan, Korea, the Netherlands, Romania, Saudi Arabia, South Africa, Sweden, the United Kingdom and the United States. . Don't hesitate to contact your IBM Sales Rep or BP and ask for a demo.
Its been a busy few weeks. I just spent a week in RTP North Carolina, with the STG Education team. We ran through our first "Implementing the Storwize V7000" course in a "Teach the Teacher" format. It was a lot of fun and I met some great fellow IBMers. It gave me a great opportunity to drive the Storwize V7000 GUI and explore all the new possibilities it opens up. First up.... the GUI is fantastic. Don't be fooled by the XIV Icons, its the smarts behind what the GUI does that makes it so powerful. Its a 21st Century GUI following very strong principles of usability and simplicity. . Talking to client after client about this product, I get lots of great questions. Two questions I get asked on a regular basis about Storwize V7000 are: . 1) What is the smallest number of SSDs I can purchase? The answer is that you can purchase just one. However with one disk you don't get any RAID. So its better to buy two SSDs for a RAID1 pair. If you buy three SSDs you can form a RAID5 array. . 2) Will the Storwize V7000 enforce the creation of hot spares? The answer is that the pre-sets that the GUI offers you, will suggest the creation of spares. For every 23 array members with the same drive class on a single SAS chain which are not RAID 0 members, a single spare is created. However the GUI will also allow you great flexibility. You can specify that a smaller or larger number of spares get created. You can choose to create NO hot spares at all. You can convert a hot spare drive into a candidate drive ( a 'free' drive). You can convert a candidate drive into a hot spare drive. You can set a 'spare goal' to set a minimum number of spares that need to exist (or an event will be logged). . So what you get is a great level of flexibility. Either follow the pre-sets and get IBM best practice... or choose your own desired spare levels. If you choose to create no spares using SSDs the Storwize V7000 will use spinning disks to rebuild a failed SSD. Then when the failed SSD is replaced, the contents of that drive will be failed back.
If your considering a midrange storage solution, there are many choices and many vendors. I often get asked: Why IBM? What makes your solution special? So when considering Storwize V7000, consider this: . The 24 disk Storwize V7000 enclosure currently offers four different disk types:
300 GB Solid State Drive (E-MLC SSD) with SAS version 2 interface (6 Gbps)
300 GB Spinning Disk (10K RPM) with SAS version 2 interface (6 Gbps)
450 GB Spinning Disk (10K RPM) with SAS version 2 interface (6 Gbps)
600 GB Spinning Disk (10K RPM) with SAS version 2 interface (6 Gbps)
Thats four choices and 24 slots per enclosure. But the questions are:
Can I mix and match?
Do I need to order these in 8 packs or 16 packs?
Do I need a separate enclosure for each disk type?
What are the rules?
. The answer is: There are no rules. You can order drives in any quantity you like and you can mix drives in an enclosure in any order you like. So this is a very flexible set of choices. . Now there are rules on RAID arrays, but these are equally flexible. For instance a RAID10 array can be anywhere from 2 disks (with is practically RAID1) to 16 disks. RAID5 arrays can use anywhere from 3 to 16 drives, RAID6 uses 5 to 16 drives.
1 - 8
3 – 16
5 – 16
2 – 16
. So when making that evaluation, ask our competitors: What are your rules when ordering disks? How restrictive are you? The Storwize V7000 answer is: There are no rules.
There has been a lot of chatter on the blog verse about so called vendor blockers (I think you can shorten that to vblock). This is the idea that a pre-blessed solution is the safest way to build infrastructure in the data centre. I can see the attraction, but I suggest the best way to get there is with the vendor who is most willing to work with the widest range of their competitors. And you cannot get much wider than IBM SVC. . IBM has been supporting SVC in a mixed OS and hardware environment since 2003. The IBM Storwize V7000 inherits all of the interoperability testing done over 7 years with IBM SVC. This is an astonishing way to bring a new product into the marketplace. I cannot think of a new midrange entrant that has somehow managed to get the same level of interoperability testing and ISV integration on the very day of its birth. When you look at the picture below you can see the depth of IBM's support matrix for SVC and its cousin, the Storwize V7000. I have on occasion raised a laugh from an audience when I describe IBM SVC as a vendor independent virtualization layer. But the picture doesn't lie... with the IBM Storwize V7000 or IBM SVC... there is no vendor blocking. Plus with the ability to move your data out of the IBM virtualization layer (using the the migrate to image command), you can remove IBM from the picture at any time... meaning no vendor blocking... and no vendor locking.
Its that time again! Rob Jackard from the ATS Group has shared with me his list of IBM Storage related updates. Storwize V7000 already has a huge amount of great material out there, links are below. Have a quick look... you may see links that are relevant to you!
(2010.10.13) Support Matrix for Subsystem Device Driver, Subsystem
Device Driver Path Control Module, and Subsystem Device Driver Device Specific
(2010.09.06) IBM SDDPCM- Open HyperSwap status may report incorrectly via the Tivoli Productivity Center for Replication GUI, If the HyperSwap was incomplete and then another unplanned HyperSwap occurs, both copies of the data will be corrupted.
(2010.09.29) RETAIN Tip# H197680-
VIOS 2.1.3 support for BladeCenter
hosts removed from System Storage Interoperation Center (SSIC) web site - IBM
System Storage DS3512 (Type 7146), IBM System Storage DS3524 (Type