IBM SAN Volume Controller (SVC) has offered Fibre Channel Storage Virtualization since June 2003. Two SVC nodes communicate with each other via fibre channel to form a high availability I/O group. They then communicate with the storage that they virtualize via Fibre Channel and with the hosts they serve that virtual storage to, via Fibre Channel. When IBM added real-time (metro mirror) and near real-time (global mirror) replication it was also done using Fibre Channel, with each SVC cluster communicating to the other by connecting using fibre channel protocol transported over dark fibre with or without a WDM or via FCIP (Fibre Channel over IP) routers.
Each Fibre Channel port on an SVC node can be a SCSI initiator to backend storage, a SCSI target to hosts and all the time communicate to its peer nodes using those same ports. With every generation of SVC node, these ports got faster and faster, going from 2 Gbps to 4 Gbps to 8 Gbps. In SVC firmware V5.1 IBM added iSCSI capability to the SVC using the two 1 Gbps ethernet ports in each node. This allowed each node to also be an iSCSI target to LAN attached hosts.
When the Storwize V7000 came out in Oct 2010 it offered all of this capability, plus offered two fundamental changes to the design.
Firstly the two controllers in a Storwize V7000 can communicate with each other across an internal bus, eliminating the need to zone them together (or even attach the Storwize V7000 to Fibre Channel fabrics).
The other more obvious difference is that a Storwize V7000 comes with its own disks, which it communicates with via multi-lane 6 Gbps SAS.
When IBM added 10 Gbps Converged Enhanced Ethernet adapters to the SVC and to the Storwize V7000, these adapters operated as iSCSI Targets, allowing clients to access their volumes via a high-speed iSCSI network. In V6.4 code IBM allowed these adapters to also be used for FCoE (Fibre Channel over Ethernet). These are also effectively SCSI targets ports allowing hosts that use CEE adapters to connect to the SVC or V7000 over a converged network.
If you have a look at the Configuration limits page for SVC and Storwize V7000 version 6.4 (the Storwize V7000 one is here), you will see this interesting comment:
"Partnerships between systems, for Metro Mirror or Global Mirror replication, do not require Fibre Channel SAN connectivity and can be supported using only FCoE if desired"
So does this mean we can stop using FCIP routers to achieve near real-time replication between SVC clusters or Storwize V7000s? The short answer is most likely not. Lets look at why...
The whole reason Fibre Channel became the standard method to interconnect Enterprise Storage to Enterprise hosts is simple: Packet loss is prevented by buffer credit flow control. Frames are not allowed to enter a Fibre Channel network unless there are buffers in the system to hold them. Frames are normally only dropped if there is no destination to accept them. Fibre channel is a highly reliable, scalable and mature architecture. When we extend Fibre Channel over a WAN we do not want to lose this reliable nature, so we use FCIP routers like Brocade 7800s, that continues to ensure frames are reliably delivered in order, from one end point to another.
Converged enhanced ethernet allows Fibre Channel to be transported inside enhanced ethernet frames. The one fundamental that CEE brings to the table is the same principle that a frame should not enter the network without a buffer to hold it. Extending FCoE over distance has the same challenge: the moment you start moving those frames over a WAN connection you need to ensure frames are not lost due to congestion. How do we do this? The same way we did with Fibre Channel: we use Dark Fibre, we use WDMs or we use routers. The same issues and requirements exist.
For more information on FCoE over distance check out this fantastic Q&A from Cisco:
One common question that I hear on a regular basis regards the availability of an SRA for VMware SRM 5.0 when using Storwize V7000 or IBM SVC running V6.3 firmware. This combination is currently unsupported as per the alert found here.
The good news is that there are now IBM SRAs available for clients running SRM in combination with V6.3 firmware. While this combination is still not listed on the VMware support matrix found here, you can download the SRAs direct from IBM if your need is urgent.
It is ironic that only days after I wrote that 497 is the IT number of the beast, I learn that Linux has another unfortunate number: 208.
The reason for this is a defect in the internal Linux kernel used in recent firmware levels of SVC, Storwize V7000 and Storwize V7000 Unified nodes. This defect will cause each node to reboot after 208 days of uptime. This issue exists in unfixed versions of the 6.2 and 6.3 level of firmware, so a large number of users are going to need to take some action on this (except those who are still on a 4.x, 5.x, 6.0 or 6.1 release). If you have done a code update after June 2011, then you are probably affected. This means that if you are an IBM client you need to read this alert now and determine how far you are into that 208 day period. If you are an IBMer or an IBM Business Partner, you need to make sure your clients are aware of this issue, though hopefully they have signed up for IBM My Notifications and have already been notified by e-mail.
In short what needs to happen is that you must:
Determine your current firmware level.
Check the table in the alert to determine if you are affected at all, and if so, how far you are potentially into the 208 day period.
Prior to the 208 day period finishing, either reboot your nodes (one at a time, with a decent interval between them) or install a fixed level of software (as detailed in the alert).
To give you an example of the process, my lab machine is on software version 22.214.171.124 which you can see in the screen capture below. So when I check the table in the alert, I see that version 126.96.36.199 was made available on January 24, 2012, which means the 208 day period cannot possibly end before August 19, 2012.
Earliest possible date that a system running this release could hit the 208 day reboot.
SAN Volume Controller and Storwize V7000 Version 6.3
30 November 2011
25 June 2012
24 January 2012
19 August 2012
Regardless, I need to know the uptime of my nodes, so I download the Software Upgrade Test Utility (in case you have an older copy, we need at least version 7.9) and run it using the Upgrade Wizard (NOTE! We are NOT updating anything here, just checking):
I Launch the Upgrade Wizard, use it to upload the tool and follow the prompts to run it, so that I get to see the output of that tool. The output in this example shows the uptime of each node is 56 days, so I have a maximum of 152 days remaining before I have to take any action. At this point I select Cancel. You can run this tool as often as you like to keep checking uptime.
Note if you are on 6.1 or 6.2 code you may see a timeout error when running the tool, especially for the first time. If you do see an error, please follow the instructions in the section titled "When running the the upgrade test utility v7.5 or later on Storwize V7000 v6.1 or v6.2" at the Test Utility download site.
As per the Alert:
If you are running a 6.0 or 6.1 level of firmware, you are not affected.
If you are running a 6.2 level of firmware, the fix level is v188.8.131.52 which is available here for Storwize V7000 and here for SVC.
If you are running a 6.3 level of firmware, the fix level is v184.108.40.206 which is available here for Storwize V7000 and here for SVC.
If you are using a Storwize V7000 Unified, the fix level is v220.127.116.11 which is available here.
You should keep checking the alert to find out any new details as they come to hand. If you are curious about Linux and 208 day bugs, try this Google search.
*** Updated April 4, 2012 with links to fix levels ***
If you have any questions or need help, please reach out to your IBM support team or leave me a comment or a tweet.
*** April 10: The IBM Web Alert has been updated with new information on what to do if your uptime has actually gone past 208 days without a reboot. In short you still need to take action. Please read the updated alert and follow the instructions given there. ***
I always laugh when people say to me: I wouldn't know what to blog about!
When you work in pre-sales support, you constantly get asked questions and each one of them could be the subject of a new blog post. Right now the most common question I am getting is:
I am implementing VMware Site Recovery Manager (SRM). One of the components I need are vendor specific Site Recovery Agents (SRA). I have searched IBM's website but cannot find them. Where are they?
So the short answer is: you get them from the VMware SRM download site. However before downloading, there is a key task that absolutely needs to be performed:
Visit the VMware vCenter Site Recovery Manager Storage Partner Compatibility Matrix. This site will confirm what products are supported by each version of SRM. You can find it here, but clearly you need to check back regularly to ensure you have the latest information.
Now find your storage device in the matrix and confirm what firmware levels are supported. This is really important. For example, the Feb 27, 2012 edition of the matrix tells me that the Storwize V7000 is supported for SRM version 5.0, but only when running Storwize V7000 firmware version 6.1 or 6.2. This is significant because if you upgrade to version 6.3 you are not supported. In fact that combination doesn't actually work yet, as detailed here. Clearly something you need to be aware of when planning firmware updates.
So where are the SRAs? On each of the pages below use the Show Details button to see what version SRAs are being shipped with that SRM (although sometimes the pages take a few days between an SRA being added and the page being updated):
There are a few more questions I routinely get asked:
Does IBM actually have an SRA download site?
The answer is yes, but it is an FTP site only for SRAs written by IBM. It is principally a repository for older SRAs and beta SRAs but you can also find the current SRAs on it. You can find the site here. Note however that it is NOT the official source. For that you need to use the VMware site.
What about the SRA for LSI/Engenio based products like the DS4800?
These used to also be found on the LSI site, but since LSI sold Engenio to NetApp, it is no longer available from the LSI or NetApp websites. You need to download the current version from the VMware sites listed above. There is a version for SRM 5 on the VMware download site.
What about nSeries SRAs?
If you need an nSeries SRA, again you should go to the VMware download pages. There are separate SRAs listed and available for IBM nSeries (as opposed to an SRA for NetApp branded filers).
What about an SRA for XIV with SRM version 5?
The answer: The SRA for XIV with SRM 5 (and 5.0.1) is now available from VMware. If you have access to download SRM, you will be able to download SRA version 2.1.0. It is the same SRA for both XIV Generation2 and Gen3.
What about an SRA for Storwize V7000 and SVC version 6.3 code?
The answer: It is coming. We are working to make it available as soon as possible. I will update this post as soon as I have a date for you (we are talking weeks, not months).
*** Update March 23, 2012 - Added details on SRM 5.0.1 ***
Many years ago I picked up a book that literally blew my mind. It was the Cuckoo's Egg by Clifford Stoll and it's a genuine classic, a true tale of hackers and how one was tracked down in the very early days of the internet.
Now the story is about events in 1986, so it captures the state of technology at the time (which rather dates the book), but wow, what a great story.
So why mention the book? Well apart from the fact that it is well worth a read, the key issue that Clifford saw again and again was default passwords. The hacker would identify a target and then try to logon using default IDs and default passwords, usually with great success.
Now I have blogged in the past about the determined (but often ignored) way that Brocade switches berate you into changing default passwords. But pretty well all products need to do this, as they all have the same issue (and a truly problematic counter-point). You absolutely need to do two things with every product in your data center:
Change the default passwords on every device you deploy.
Record what those passwords got set to (preferably using a logical or physical password safe).
Now don't laugh, but forgotten/lost passwords on data center kit (like switches) is a VERY common problem. When I worked in the IBM Storage Support team I took calls EVERY WEEK from clients who had devices they could not logon to, for all manner of reasons. For some, supplying them with the default passwords saved them (and condemned their employer?), but for others they needed much more detailed assistance.
My preferred solution to this challenge is to use external authentication (like LDAP) but being able to reset passwords with an external tool is also a nice option to have available.
The reason I started thinking about this is a nice tool IBM offer for the Storwize V7000 called the Initialization Tool that you can download from here. Using this tool you can reset the password of the Superuser ID on a Storwize V7000 back to the default (passw0rd). The tool runs on a USB key. After requesting the tool to help you to reset the superuser password, you insert the USB key into the Storwize V7000, wait for the orange indicator light on the relevant node canister to stop blinking and the task is complete. Then put the USB key back into your laptop and run the init tool again to get a completion report that should look like this:
This is great to rescue customers who have lost their passwords, but the question then gets raised: Can I block this?
My first response is: if you are concerned about unauthorized people with malicious intent placing USB keys into your Storwize V7000, then don't let them into your computer room (presuming you can spot them by the colour of the hat they are wearing). If that is not an option, lock the rack that the Storwize V7000 resides in (change control does have its benefits). If that is not an option, there is one more alternative, but it is a tad extreme.
What we can do is prevent password reset via USB key (or in the case of the SVC, via the front panel). We do this by issuing the following CLI command: setpwdreset -disable
In the following example, I confirm that password reset is possible (value 1), I then disable it and confirm that password reset is no longer possible (value 0). If curious I could then get some help on that command:
Only if your paranoia is matched by your attention to detail.
My reason to hesitate recommending it is simple: If you prevent password reset and then forget your password (and have no other local Security Administrator accounts), you have locked the door and thrown away the key. Far better to physically lock the rack.
In the end though, your company needs to set a policy that is actively enforced (with no exceptions). So get to it.
When IBM brought out the SAN Volume Controller (SVC) in 2003, the goal was clear: support as many storage vendors and products as possible. Since then IBM has put a huge ongoing effort into interoperation testing, which has allowed them to continue expanding the SVC support matrix, making it one of the most comprehensive in the industry. When the Storwize V7000 was released in 2010 it was able to leverage that testing heritage, allowing it to have an amazingly deep interoperation matrix on launch date. It almost felt like cheating.
However I recently got challenged on this with a simple question: Where is the VNX? If you check out the Supported Hardware list for SVC V6.3 or Storwize V7000 V6.3 you can find the Clariion up to a CX4-960, but no VNX.
The short answer is that while the VNX is not listed there yet, IBM are actively supporting customers using VNX virtualized behind SVC and Storwize V7000. If you have a VNX 5100, 5300, 5500, 5700 or 7500 then ask your IBM pre-sales Technical Support to open an Interoperation Support Request. The majority are being approved very quickly. The official support sites that I referenced above will be updated soon (but don't wait, if you need support, ask for it). IBM is working methodically with EMC to be certain that when a general publication of support is released for VNX (soon!), both companies will agree with the published details.
And for the wags who think that this is a ringing encouragement to buy VNX, you would be missing the point. You cannot be a serious Storage Virtualization vendor if you are not willing to support your clients purchasing decisions, regardless of which vendor they buy their storage from. IBM have been staying that course and demonstrating that willingness since 2003. It's a pretty good track record and one that they are determined to maintain.
I have updated my IBM Storage WWPN Determination Guide to version 6.5. You can find the updated guide on IBM Techdocs here.
The main change is that new DS8800s are now presenting slightly different WWPNs, so I added three new pages to describe the changes.
If this guide is new to you, its purpose it to let you take a WWPN and decode it so you can work out not only which type of storage that WWPN came from, but the actual port on that storage. People doing implementation services, problem determination, storage zoning and day to day configuration maintenance will get a lot of use out of this document. If you think there is an area that could be improved or products you would like added, please let me know.
It is also important to point out that IBM Storage uses persistent WWPN, which means if a host adapter in an IBM Storage device has to be replaced, it will always present the same WWPNs as the old adapter. This means no changes to zoning are needed after a hardware failure.
I also host the book on slideshare, so you can also view and download it from there:
The IBM SVC has has been setting records in SPC-1 (OLTP-like) benchmarks for many years. However recently HP stole the crown with a 3Par benchmark of 450,212.66 IOPS.
But in breaking news, the SVC is back on top with the very first SPC benchmark that exceeded 500,000 IOPS (520,043.99 to be precise!). You can see the executive summary here.
This benchmark used eight of the current SVC engines (model CG8s) with Storwize V7000 as the backend disk. It shows the awesome power of SVC, its ability to scale and to handle very large configurations with very large throughput requirements. It also shows the power of IBM pSeries which was used to drive these IOPS.
The first update for Storwize V7000 and SVC release 6.3 is now available. You will find it here for Storwize V7000 and here for SVC (note both links will require you to login to Fix Central with your IBM ID). As usual the new release contains a combination of new features and fixes. The new features are:
New features in SVC 18.104.22.168
* Support for multi-session iSCSI host attachment
* Language Support for Brazilian Portuguese, French, German, Italian, Japanese, Korean, Spanish, Turkish, Simplified Chinese and Traditional Chinese
There are also several fixes (with some variation between SVC and Storwize V7000, mainly around the platform hardware). The release notes (which you can find at the links above) detail them all. Two fixes I have been looking forward to are:
IC80253 Unable to log into the GUI if password contains special characters. This meant that a password with a comma in it could not be used in the GUI (you got a backspace instead). Passwords with commas could be used in the CLI. This bug was picked up by one of my clients when trying out LDAP and is now fixed in 22.214.171.124.
IC80501 Performance statistics collection fails to record read and write response times for internal drives. This issue meant that SVC internal SSD drives always showed 0 ms response times in TPC.
Note that the Drive firmware does not need to be updated with this release. The new upgrade test tool (version 7.3) will not ask you to update them. I will let you know when that situation changes.
I am getting this question on a very regular basis:
"We have just upgraded to ESXi 5.0 but we cannot find the VAAI driver on the IBM Website"
The answer? There is no vendor supplied driver because no driver is needed. ESXi 5.0 uses a SCSI T10 compliant set of commands that all vendors need to support for VAAI to work.
But of course in the tradition of all answered questions, it leads to another question:
"Once I have upgraded to ESXi 5.0 how can I tell if VAAI is really working?"
The good news is that it is very easy to spot if ESXi 5.0 has detected a VAAI capable LUN. The moment a new LUN is detected by ESXi 5.0 it tries out an Atomic Test and Set command. If that works, you will see that Hardware Acceleration shows as Supported in vCenter. In the screen capture below I have three datastores, two from XIV and one from Storwize V7000, all presented to an ESXi 5.0 server. I dragged the Hardware Acceleration column over from the right hand side to help with the screen capture (in case your vCenter looks different), but you can see the Hardware Acceleration column shows each DataStore as Supported (and did so the moment the volume was detected).
Of course having seen the Hardware Acceleration Supported message only proves that Atomic Test and Set works. To confirm if XCopy (Hardware Accelerated Move) is working, on SVC or Storwize V7000 we can use the Performance monitoring panel. In the example below I first performed a storage vMotion, moving a virtual machine between two Datastores located on the same Storwize V7000 (running 126.96.36.199 firmware). I then performed a clone of the same virtual machine, where the source was on one datastore and the target was placed on another (but both located on the same Storwize V7000). What you can clearly see is that both operations (storage vMotion and cloning) generated no volume traffic, only MDisk traffic. This means that the ESXi server is doing none of the work and the storage is doing all of the work.
The Storwize V7000 and SVC have a command line interface that you access via SSH. Every-time you logon, whether it is to transfer a file (using a tool like pscp), issue a single shot command from a script (using a tool like plink) or logon to issue commands interactively (using a tool like PuTTY), you clearly need to authenticate yourself. Since June 2003, the way you did this was to use a public/private key pair, where the SVC or Storwize V7000 had the public key and the SSH client (such as PuTTY) authenticated using the private key (the PPK file).
However with release 6.3 of the SVC and Storwize V7000 firmware, the use of key files is now optional. A user can now authenticate purely by using a password. This includes using your domain ID. So if you defined LDAP to your machine, as I documented here, you could now SSH direct to your Storwize V7000 or SVC, use your Domain user id and password and not go through the key file setup task. Nice!
The choice to continue to authenticate just with an SSH key remains available. If a user has both a password and a configured key file, then either method will work (you only need to use one - not both). Existing scripts will be unaffected by this change, so nothing gets broken because of this.
I think this is a very positive change and one I openly welcome. Combined with LDAP, this really makes user account setup an easy and simple task.
IBM recently released a new version of firmware for the SAN Volume Controller and Storwize V7000. This is known as release 6.3 and continues the tradition of two major updates per year, each adding significant new functions.
So the 6.3 release notes for both Storwize V7000and SVC listed the following new feature:
Support for 4096 host WWPNs
Since I blithely listed this feature in a recent post I have received lots of emails asking exactly what it means, so I thought I had better explain.
The IBM SVC and Storwize V7000 have always had very clearly published maximum capabilities such as the ones listed here for Storwize V7000 release 6.3 and here for SVC release 6.3.
Most of these numbers are very high and few customers actually approach these maximums. The main issue I am seeing for some of our larger AIX customers is this one:
Total Fibre Channel ports (WWPNs) per I/O group: 512
The reason this can become an issue is the combination of NPIV and AIX Live Partition Mobility. NPIV allows one physical HBA to be shared among multiple operating system instances, each one believing it has exclusive access to the HBA with each one allocated its own unique WWPNs. Suddenly a single HBA which used to present just one WWPN through the SAN to the SVC, can now present vast numbers of them. In addition AIX Live Partition Mobility (which lets you move AIX operating systems between LPARs on the fly) needs additional pre-configured WWPNs defined on the target LPAR to support the move. This further increases the quantity of WWPNs that need to be defined to the SVC (one easy way to spot NPIV generated WWPNs is they normally start with the letter C).
So the bottom line is that IBM needs to make this limit bigger and SVC and Storwize V7000 6.3 code contains the necessary architectural changes to allow this. The first phase is to start potentially supporting up to 2048 WWPNs per I/O group although clearly based on the initial version of the release notes, the long-term plan is to support 4096.
But there is a problem and it has nothing to do with the SVC or Storwize V7000. The problem is that there are certain SAN configurations which may have issues with these large numbers of WWPNs (mainly around older SAN switches not having the CPU power for the switch fibre channel name-server and login-server to handle vast numbers of WWPNs coming out of one HBA).
So what should you do if you need to push the limits?
Contact your IBM Pre-Sales support and ask for a SCORE request to be opened (also known as an RPQ). You will need to detail your current SAN configuration (especially switch models and firmware levels) so that SVC development can ensure you won't overwhelm your switches. It will also allow our development team to learn how many clients our there need this support. All approvals will include a requirement to upgrade to release 6.3, so you should include this in your planning.
Any questions? Feel free to leave a comment or send me a tweet or an email.
With the 6.3 release of the Storwize V7000 and SVC code (which I blogged about here), there are so many new features and functions that I have plenty more to blog about!
The first new feature I blogged about was LDAP support, but an existing feature that has been enhanced is the performance monitor (brought in with release 6.2). When this first came out I put a video on You Tube showing what metrics could be displayed in that release. This is a sped up image with no voiceover:
Now with release 6.3 IBM has added separate graphs for reads and writes plus the ability to display IOPS or MBPS, plus the ability to display graphs of read and write latency. Nice! I got so excited I made another You Tube video, this one with narration. So now you can compare the new to the old:
Lets imagine a new rack server or a new blade server has been added to your Fibre Channel SAN. The first job for the SAN administrator is to zone it to the storage it requires access to. The task normally runs something like this:
Identify the WWPNs for the new server HBA. We can do this using Qlogic SAN Surfer or Emulex HBAnywhere, or by looking at the WWPNs reported by the Fibre Channel switch or by using datapath query wwpn (with SDD and SDDDSM) or by using the xiv_fc_admin -P command with the XIV HAK. There are lots of different ways, you get the idea.
On fabric 1 create a new alias for the server HBA port cabled to that fabric.
For each storage device that the server needs access to on fabric 1 (or possibly just switch 1), create a new zone and include the new server alias and the alias for every relevant storage port on that device. Repeat if you have other storage devices (so two XIVs means two new zones).
Put the new zone (or zones) into the active zoneset (or a clone of it) and activate it.
Repeat on fabric 2 (after waiting a decent interval to ensure no mistakes were made in fabric 1... well I hope you wait.... you do don't you?).
The main trap here is that when creating a zone, you need to ensure you select all of the correct storage aliases for your selected storage device. For instance we could have a simple layout like this:
Fabric 1 contains our new server (in this example an IBM x3850) and three XIV ports:
This means when creating the zone I need to identify and select four separate aliases. What I could do instead is create an alias with all my XIV target ports in it. Now I only have two aliases to select in that fabric:
This means when creating the zone I need to identify and select three separate aliases. What I could do instead is create an alias with both my Storwize V7000 WWPNs in it. Now I only have two aliases to select in that fabric:
This method of amalgamating multiple storage port aliases works fine for devices like DS8000, SVC, Storwize V7000 and XIV. I use this method all the time to simplify zoning and I find it reduces both mistakes and the time required to complete zoning tasks.
The only exceptions are:
Don't do it for DS3000, DS4000, DS5000 or DCS3700 as the controllers on these devices do not like to see each other through the switch.
Don't combine ports from different storage devices, so if you have two XIVs in a fabric create one alias for the target ports of each XIV (although you could combine ports from different SVC I/O groups within the same SVC cluster into one alias). You should still use individual aliases for ports being used for migration or replication purposes.
Don't use the WWNN to create an alias. Always create multi-WWPN aliases so you have granular control of which ports go into the alias. If you use the WWNN from an XIV you will also implicitly include any ports that are being used for replication or migration and thus zone them to the host, which makes no sense.
I would love to hear any techniques you have to make your (and my) life easier.
Once your SVC or Storwize V7000 is upgraded to version 6.3 you can start using LDAP for authentication. This means that when you logon, you authenticate with your domain user-id and password rather than a locally created user-id and password.
So why is this important?
It saves you having to configure every user on every SVC or Storwize V7000. If you have multiple machines this makes it far more efficient to set up authentication.
It means that when commands are executed on the SVC or Storwize V7000, the audit log will show the domain username that issued that command, rather than a local username, or worse just superuser (i.e. who mapped that volume? The superuser did.... who? )
It gives you central control over access. If someone leaves the company you just need to remove access at the domain controller, meaning there won't be orphan user-ids left on your Storage equipment.
So as an exercise I added my lab Storwize V7000 to our domain to show how it is done. This example also applies to an SVC so don't be confused if I only refer to Storwize V7000 from now on.
The first task is to negotiate with your Domain administrator to get a new group setup on the domain. In this example I use a group called IBM_Storage_Admins which lets me use this group for various storage devices (such as an XIV or a SAN Switch).
To create this group we need to logon to the Domain Controller and configure Active Directory. An easy way to do this from the AD controller is to go to Start → Run and type dsa.msc and hit OK. The Active Directory Users and Computers Management Console should open.
Select the groups icon to create a new group.
Enter your group name, in my case: IBM_Storage_Admins and hit OK.
Now right select relevant users who need access to the storage and add them to the IBM_Storage_Admins group. In this example I have selected Anthony (which uses anthonyv as a username).
In this example we are adding anthony into the IBM_Storage_Admins group:
Now it is time to configure the Storwize V7000 so start the Web GUI and logon as Superuser.
Firstly we go to Settings → Directory Services:
We choose the button to Configure Remote Authentication:
We choose LDAP and hit next.
We choose Microsoft Active Directory with no Transport layer Security. We then expand the Advanced Settings. My lab domain is ad.mel.stg.ibm so I use the Administrator ID on the Domain Controller to authenticate access. You could use any user that has authority to query the LDAP directory. We then hit Next.
We then add the domain controller which in this example is 10.1.60.50 and the base domain name chopped into pieces (so ad.mel.stg.ibm becomes dc=ad,dc=mel,dc=stg,dc=ibm ) and hit Finish.
Provided the command completes successfully we have defined the domain controller to the Storwize V7000. Now we need to add a group. Go to Access → Users.
Select the option to add a New User Group.
In this example we want to add a group for users allowed full admin access to the Storwize V7000. This matches the group we created on the Domain Controller. So we call the group IBM_Storage_Admins and we use the Security Administrator role (which is the most powerful role) and tick the box to enable LDAP for this group.
Now to test, I logon to the Storwize V7000 using the domain user-id anthonyv with that users domain password. Remember this user is not defined on the Storwize V7000 itself and that if it all goes wrong, we can still logon as Superuser.
Now I create a volume and delete it. Then I check the audit log from Access → Audit log.
Sure enough, we see exactly who did that command.
This is a great outcome for security,auditing and for easy access administration.
If you have issues, from the Settings → Directory Services menu, use the Global Actions dropdown on the right hand side to Test LDAP Connections and Authentication or re-configure LDAP.
If you already have existing users (what we call Local users), configuring remote authentication using LDAP does not disable or invalidate those local user-ids. This means you can either logon with a local user-id or logon with a Domain user-id. This is handy if the domain controller fails but can confuse you if your local user name and your domain user name are the same name (for example both anthonyv). The Storwize V7000 will look you up in the local user name list first. I suggest removing all local users (except superuser) as this will reduce confusion but still leave you a backdoor in case remote authentication stops working.
If you see any mistakes or have suggestions to improve the way I described this, please let me know.
The latest release of SVC and Storwize V7000 firmware is now available for download. The major new features that are added with this release are:
Global Mirror with Change Volumes
Native LDAP Authentication
Extended distance split clusters (for SVC)
Support for 4096 host WWPNs
These are some great new features. The ability to use Global Mirror with Change Volumes means clients can now mirror across far smaller pipes, while the increase in host WWPNs is very welcome news for NPIV installations that are suffering from WWPN sprawl.
If you plan to upgrade, firstly grab the new Upgrade Test Utility from here. The links to the Storwize V7000 and SVC versions are both on that page. Remember you can run this test as many times as you want whenever you want, to check the health of your device for upgrade. When you run the upgrade test utility on a Storwize V7000 you may get a message that your disks have down-level firmware. The process to update them is documented here.
If you're using a Storwize V7000 you can grab the 188.8.131.52 code from here. If you're using an SVC you can grab the 184.108.40.206 code from here. I am sending you to the compatibility matrix page because you should always check that your from level is ok for your to level.
To run the upgrade go to Configuration (the spanner icon) → Advanced → Upgrade Software →Launch Upgrade Wizard
I have not shown all the panels you will see because it is very much a follow-your-nose task, but in essence, first we feed it the Upgrade Test Utility file and run that test.
If you get warnings you may need to act on these. If you are unsure what to do to resolve a warning message, place a service call.
Once the test passes or you're happy you understand the warnings, we now point it at the code package and wait for it to copy across and keep hitting Next.
The application of the code shuts down and reboots each controller, with a 30 minute gap in between. You will transition from this (both nodes down-level, node1 being upgraded):
To this (node1 upgraded, node2 still online but waiting for 30 minutes):
When node2 starts the upgrade the GUI will failover to node1 and be upgraded to the new version. You will notice the difference immediately, it has a different look and feel. Please don't be tempted to play with the new functions until both controllers are upgraded! Wait until you see this (note a slight change, the GUI flow is now Settings (the spanner icon) → General → Upgrade Software:
Now your complete it is time to start checking out what is new... but that's a whole different blog post!
The SVC and Storwize V7000 offers a command line interface that you access via SSH. You start your favorite SSH client (such as PuTTY or Mindterm) and then logon as adminor as your own user-id. Right now to do this you need to generate a private/public key pair, although with release 6.3 (which will be available November 2011), you will be able to logon via SSH using just a user-id and password.
Having logged on there are three categories of commands you can issue:
svcinfo: Informational commands that let you examine your configuration. svctask: Task commands that let you change your configuration. satask: Service commands that are only used in specific circumstances.
There are several CLI usability features that I routinely find users are not aware of, so I thought I would share some of them here:
1) Listing all possible commands
If you cannot remember a command, here is a simple trick to list them all. Issue one of the following commands:
svcinfo -h or svcinfo -?
svctask -h or svctask -?
You can also type either svcinfo or svctask and then hit the tab key twice to get a full listing. With svctask you will need to type y to list them all, as per the example shown below:
IBM_2076:STG_V7000:admin>svctask (HIT TAB twice!)
Display all 139 possibilities? (y or n) y
2) Getting help on a particular command
Having found the command you want, issue that command with either -? or -h to get help information. For instance:
svctask mkvdisk -?
svctask mkvdisk -h
You will be shown the same help information that you can find in the Infocenter, including examples of syntax.
3) Drop the svctask and svcinfo prefixes
In release 6.2 of the SVC and Storwize V7000 firmware, the requirement to prefix a command with svcinfo or svctask has been removed. However I tend to keep using them because I write a lot of example commands for clients and I cannot be sure which version of firmware they are running.
4) Use the shell
When we SSH to the SVC or Storwize V7000 we are connecting to a Linux operating system using a special restricted shell. Some of the common Unix commands don't work (such as ls or grep or awk), but any commands that are provided by the shell itself, will work, such as while, if, read, pipe and echo.
We can use this to construct some really clever commands.
For instance creating volume copies is very popular, but the default copy rate is rather slow (50, which equals 2 MBps). It is not unusual for end users to speed up the background copy and then forget to slow it down when they are finished. So I wrote two commands to help me out. Firstly I run a command to display the copy rate of every volume. Ideally I should see 50 alongside each volume. However I often find that some volumes are set to higher numbers, such as the maximum value of 100 (which is 64 MBps).
svcinfo lsvdisk -nohdr |while read id name IO_group_id;do svcinfo lsvdisk $id |while read id value;do if [[ $id == "sync_rate" ]];then echo $value" "$name;fi;done;done
Lets break down this command. The structure looks like this:
We start with svcinfo lsvdisk -nohdr. This gives us a list of every VDisk in column format with no header information.
We pipe the output of that lsvdisk command to while read. This reads the output one line at a time and lets us work with that output. We read the first three columns of output and label the data in the first column id, the second column name and the third column IO_Group. I find we need to label at least three columns. We could read extra columns if we wanted to, but all I want is the VDisk id and name.
For every line of data we issue an lsvdisk command against each listed VDisk using the VDisk id. This output is not in column format so we need to do something different here.
We now examine the output of the lsvdisk command for each VDisk by piping the output to while read. Since each line contains a descriptor and a value, we label them id and value. We use if to look for a line that starts with sync_rate.
When we find the sync_rate for a VDisk we print the value of the sync_rate and the VDisk name. We are done for this VDisk.
We now examine the next VDisk and again look for the sync_rate for that VDisk.
Once we have examined every VDisk, we are done.
I then run the following command which sets the copy rate for every volume to the default value of 50 (2 MBps):
svcinfo lsvdisk -nohdr |while read id name IO_group_id;do svctask chvdisk -syncrate 50 $id;done
Clearly you could edit this second command to change the copy rate to any value between 0 and 100. In each case you just paste the entire command in, and hit Enter.
Lets break down this command. The structure looks like this:
We start with svcinfo lsvdisk -nohdr. This gives us a list of every VDisk in column format with no header information.
We pipe the output of that lsvdisk command to while read. This reads the output one line at a time and lets us work with that output. We read the data ion the first three columns of output and label the data in the first column id, the second column name and the third column IO_Group. I find we need to label at least three columns. We could read extra columns if we wanted to, but all I want is the VDisk id.
For every line of data we read, we do the following command: svctask chvdisk -syncrate 50 $id. Since we labelled the first column of output from the lsvdisk command as id, and that column contains VDisk IDs, we are going to issue this command against every VDisk that got listed.
Once we have run the chvdisk command against every VDisk listed, we aredone.
There are lots of possible clever combinations and I will list a few more in upcoming posts.
I have also been getting lots of requests to write a post about updating drive firmware, so expect something on that very soon.
IBM have offered Enterprise Storage Virtualization since June 2003 with the IBM SAN Volume Controller (SVC). October 2010 saw IBM releasing the Storwize V7000, taking the SVC code and packaging it into a midrange disk product. So now you have four possible choices:
Use SVC to virtualize your storage.
Use Storwize V7000 to provide internal SAS drives plus virtualize your storage.
Use Storwize V7000 as a midrange disk product.
Use Storwize V7000 virtualized behind SVC.
The great thing is that all four choices are valid and all four choices work just fine. But for customers already using SVC, or considering SVC, the question then becomes, should I virtualize a Storwize V7000 behind an SVC? Does this makes sense?
The short answer: YES!
We have a great many customers happily doing this, so I thought I would share some common questions I get around configuration. Firstly there is an InfoCenter page on this which you will find here. Secondly there is a debate about whether we should create individual volume/arrays on the Storwize V7000 or just create a single pool on the Storwize V7000 (which equates to striping on striping). More bench marking is being done to see if one method is truly better than the other, so until then I recommend the method described below. If you have already done stripe on stripe, don't go changing anything until I update this post.
How many ports should I use for Zoning?
The Storwize V7000 has 8 Fibre Channel ports, 4 from each node canister. You need to zone at least two ports from each node canister to your SVC cluster. This is no different to how you would zone a DS5100 or an EMC VNX.
How will the SVC detect the Storwize V7000?
On the SVC you will see two storage controllers, one for each node canister. This is quite normal. The reason for this is that each node canister reports its own WWNN. This is not a problem and will not affect volume failover if one node canister goes offline.
In the example below the SVC has detected two new controllers. The confusing factor is that both report as 2145s, but they are a Storwize V7000. Rename them to reflect what they really are (something like StorwizeV7000_1_Node1 and StorwizeV7000_1_Node2).
How should I define the SVC on the Storwize V7000?
You need to create a new host on the Storwize V7000 and call it something like SVC_1. if the SVC WWPNs don't appear in the WWPN dropdown, you will need to manually add them as shown below:
You can get the SVC WWPNs from your existing zoning or by doing an svcinfo lsnodeagainst each SVC node or display them in the SVC GUI as shown below:
What size Storwize V7000 volumes should I create?
My recommendation is to do the following on the Storwize V7000
Create arrays of preferably 8 disks in size. The ideal number will depend on how many disks you have. On my machine I have 22 disks, so I create three arrays each with seven disks (and one hot spare):
Create one pool for each array:
Create one volume out of each pool (using all space in the pool).
Define the SVC to the Storwize V7000 as a host (as described above) and map all volumes to the SVC.
On the SVC detect all the Storwize V7000 LUNs as MDisks and create one pool.
Now you should have a pool on the SVC that you can use to create volumes to present to your hosts. They will be striped by default, which is exactly what you want.
Hopefully all of this makes sense. Questions and comments very welcome.
One of the things I like about creating host definitions on XIV, Storwize V7000 or SVC is that for the vast majority of host types, the default choice is the correct choice. This saves me having to specifically tell the storage what operating system each host is running.
However in recent times more and more choices have snuck in. For instance in 220.127.116.11 code the Storwize V7000 and SVC had to add an extra option for OpenVMS hosts (there is an alert related to this here). Fortunately most of these exceptions are (from my perspective) for relatively obscure operating systems (users can start spamming me now).
So you need to warn the XIV if your host is running HP-UX or zVM:
On the SVC or Storwize V7000 you need to change from default if your host is running HP-UX or OpenVMS. There is also TPGS (or Target Port Group Support) for use with Apple OS and Solaris hosts using MPxIO. For more details visit this link for Solaris and this linkfor Apple.
Why does this requirement exist? Or put differently, why can't the storage be smart enough to just detect the host OS and save the storage admin this bother? Sadly it all has to do with how the hosts issues and responds to SCSI commands.
HP-UX for instance uses flat space addressing (rather than peripheral device addressing). If the storage does not respond in the expected way it will see only 8 LUNs per port (which is not many). I have seen some strange things when HP-UX hosts have not been defined correctly. I have seen even stranger things when non HP-UX have been defined as HP-UX!
OpenVMS on the other hand gets very upset if the storage reports itself compliant with a SCSI specification higher than SPC2 (which is now a very old SCSI specification).
The good news is that for AIX, Linux, Windows and VMware hosts, default works just fine.
I love PuTTY, it is one of my favourite pieces of open source software. For those who don't know what PuTTY is, it is a free implementation of Telnet and SSH for Windows and Unix platforms. Personally I use PuTTY for connecting to Storwize V7000s, SVCs, SAN switches and Unix servers in my lab and at clients.
Being someone who does implementation services, I want to always be sure about what I do, what I did and what affect it had. So keeping logs of every time I connect to any device is very important to me. I want to be able to go back in time to any point at any client I have ever worked with (sounds like time travel, but hopefully you get the idea).
I achieve this by changing the default behaviour of Putty so that every new Session I save uses my preferred default settings. Note all the screen captures below use version 0.61 of PuTTY (which came out July 2011).
First start up PuTTY. We are going to change the Default Settings (which I have highlighted):
Now select Logging from the left hand panel and change all the indicated settings (2, 3 and 4). The folder I use is C:\PuttyLogs but you could of course choose a different one. Note the &H_&Y&M&D&T means the session file name will be the host name, year, month, day and time for that session.
Now go back to Session, highlight Default Settings and hit Save:
Now every new session you open and/or save will automatically log your session output to the specified folder. Existing Saved Sessions will not be changed. You will need to update each of these separately.
The IBM Storage Tier Adviser Tool (known as STAT for short) is a clever piece of software that lets you predict how much business value you would get from adding SSDs to your Storwize V7000, SVC, DS8700 or DS8800. This is because you can add SSDs to all of these products and then have hot spots dynamically and automatically migrated to SSD using IBM's Easy Tier technology (which is offered as a no charge feature).
For clients who have not yet purchased SSDs, or who are unsure which Storage Pools to deploy them into, the STAT tool will help with decision-making.
I recently struck a rather simple problem with the STAT tool after installing it: I kept getting a CMUA00007E error. I downloaded the tool from here and installed it successfully onto my Windows 2008 64-bit lab machine (running on a IBM x3850). The install went fine so I then proceeded to download the heatmap from my Storwize V7000. The heatmap file is automatically generated by Easy Tier and is used as an input file for the STAT tool. You can see an example of where to find the heatmap file in the screen capture below:
I then placed the heatmap into the same folder as the STAT tool and tried to generate a report. It failed with this rather annoying message:
C:\Program Files (x86)\IBM\STAT>stat dpa_heat.78G01A6-2.110823.072309.data
CMUA00007E The STAT.exe command failed to produce the heat distribution output.
I initially thought I had a bad heatmap, but since it is a binary file, opening it in a text editor did not tell me anything.
Actually the issue was simple: I did not have write authority to that folder. To get around this I instead started the command prompt as Administrator:
I then re-ran the command:
C:\Program Files (x86)\IBM\STAT>stat dpa_heat.78G01A6-2.110823.072309.data
CMUA00019I The STAT.exe command has completed.
Having run the tool I was now able to open the index.html in the STAT folder and see much hot data I have in my lab. Turns out that I don't actually have any hot data right now! Don't tell my manager though, he might try to take my SSDs away. #
Having run the tool once, I did not need to use this trick again. It now runs without starting the command prompt as Administrator.
VMware vSphere 4.1 brings in a brilliant new function to offload storage related workload. This function is called VAAI (vStorage APIs for Array Integration) and requires that your SAN storage supports VAAI and that your ESX or ESXi server has a driver installed to utilize it.
IBM first supported VAAI with the IBM XIV using an IBM supplied VAAI driver. IBM then added support to the Storwize V7000 and SVC, so IBM has now released a new VAAI driver to support all three products at once. You can find the driver, installation guide and release notes at this URL.
I discovered some quirks in the process to update the IBM VAAI driver from version 18.104.22.168 to version 22.214.171.124 on VMware ESXi. The benefit in moving to version 126.96.36.199 is that the updated driver supports both the IBM XIV as well as the Storwize V7000 and IBM SVC.
I downloaded the new driver from here and which uses the following naming convention:
Version 188.8.131.52 is named IBM-ibm_vaaip_module-268846-offline_bundle-395553.zip Version 184.108.40.206 is named IBM-ibm_vaaip_module-268846-offline_bundle-406056.zip Version 220.127.116.11 is named IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
The last 6 digits in the file name is what differentiates them. However when I ran the --query command against an ESXi box, I got confused:
Both the uplevel and downlevel VAAI driver files start with: IBM-ibm_vaaip_module-268846 So which one is installed? The 18.104.22.168 version or the 22.214.171.124 version? I ran the following command to confirm if the updated bulletin applies (the one ending in
613937 ). This confirmed my ESXi server was using version 126.96.36.199 and needed an upgrade to version 188.8.131.52.
vihostupdate.pl --server 10.1.60.11 --username root --password passw0rd --scan --bundle IBM-ibm_vaaip_module-268846-offline_bundle-613937.zip
The bulletins which apply to but are not yet installed on this ESX host are listed.
---------Bulletin ID--------- ----------------Summary-----------------
IBM-ibm_vaaip_module-268846 vmware-esx-ibm-vaaip-module: ESX release
To perform the upgrade I first used vMotion to move all guests off the server I was upgrading. I then placed the server in maintenance mode and installed the new driver:
There are no commands needed to activate VAAI or claim VAAI capable devices in ESXi. You simply need to confirm that the both boxes shown in the example below have the number 1 in them (for hardware accelerated move and for fast init):
To test VAAI I normally do a storage migration (storage vMotion) moving a VMDK between datastores on the same storage device. What you should see is very little VMware to Storage I/O, as I depicted in this blog post and this blog post.
My colleague Alexandre Chabrol from Montpellier Benchmarking Center also helped me out with the ESXCLI commands to control VAAI. We can confirm the state of each of the three VAAI functions and switch them off and on. We use the -g switch to display them, the -s 0 switch to turn them off and the -s 1 switch to turn them on. In this example I first confirm that VAAI is active for hardware accelerated moves, hardware accelerated inititialization (write zeros) and then hardware assisted locking. I then disable and enable hardware accelerated moves.
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /DataMover/HardwareAcceleratedInit
Value of HardwareAcceleratedInit is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -g /VMFS3/HardwareAcceleratedLocking
Value of HardwareAcceleratedLocking is 1
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 0 /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 0
esxcfg-advcfg.pl --server 10.1.60.11 --username root --password password -s 1 /DataMover/HardwareAcceleratedMove
Value of HardwareAcceleratedMove is 1
Final thought: Most if not all of these commands can be done via the vSphere Client GUI, you do not need to use CLI. But I am surprised how many people like to use the CLI and want to see example syntax. Got a preference yourself? Love to hear about your experiences.
*** Update February 20, 2012 ***
The IBM Storage Device Driver for VMware VAAI was updated to version 184.108.40.206 in February 2012. This new version fixes a rare case where XIV, Storwize V7000, or SVC LUNs are not claimed by the IBM Storage device driver. If you are using version 220.127.116.11 without issue, there is no need to upgrade. I have updated this post to reflect the new version.
Here is a little test. Check your documentation: Do you know how to power down and power up the equipment in your computer room? If you had to power off your site in a hurry would you know how? If you wanted to script a shutdown, could you do it?
Here are some hints and tips that might help you with some of my favourite products:
The process to power up or down your DS8000 is documented in the Information Center here.
If you want to script powering off a DS8000 storage unit you can use thechsu -pwroff DSCLI command. This command will shut down and power off the specified DS8000 unit. Be careful before powering off the unit to ensure that all I/O activity has been stopped. An example of the command is shown below. Your machine will have a different IP address, password and serial number to mine. Note the serial number always ends in zero because we send the command to the storage unit.
The IBM Storage Management Console for VMware vCenter version 2.5.1 is now available for download and install. This version supports XIV, SVC and Storwize V7000 as per the versions on the following table (the big change being support for version 6.2):
If you want to see a video showing the capabilities of the new console, check out this link.
After installing the console, you will get this lovely new icon:
Start it up and select the option to add new storage, you now get three choices:
If your using SVC or Storwize V7000 you need to specify an SSH private key. This key MUST be in Open SSH format. This caused me a problem as I kept getting this message when trying to add my Storwize V7000 to the plug-in:
Unable to connect to 10.1.60.107. Please check your network connection, user name, and other credentials.
I could use the same IP address, userid and SSH private key to logon to the Storwize V7000 using putty, so I knew none of these things were wrong.
I reread the Installation Instructions closely and realized my mistake. It clearly states:
Important: The private SSH key must be in the OpenSSH format.
If your key is not in the OpenSSH format, you can use a certified
OpenSSH conversion utility.
I pondered what conversion utility I could use when I realized I had the utility all the time:Puttygen. I opened PuttyGen, imported my private key (the .ppk file) and exported my SSH private key using OpenSSH format. You don't need to do anything with the public key.
I was then able to add the Storwize V7000 by specifying the private SSH key exported using OpenSSH format.
Now I have both IBM XIV and Storwize V7000 in the vCenter plug-in and can get detailed information about and manipulate both. In this example I have highlighted the Storwize V7000, revealing it is on 18.104.22.168 firmware.
I was tempted to detail all the many things you can do with the plug-in, but your better off watching the video via this link.
So are you using the plug-in? Have you upgraded to version 2.5.1 yet? Comments very welcome!
Over at SearchStorage.com.AU they recently published an article entitled Six reasons to adopt storage virtualisation. You can find the article here. The six given reasons are:
Storage virtualisation reduces complexity
Storage virtualisation makes it easier to allocate storage
Better disaster recovery
Better tiered storage
Virtual storage improves server virtualisation
Virtual storage lets you take advantage of advanced virtualisation features
Its a well written article and I agree with every point. But one could be forgiven for reading the article and thinking that either storage virtualisation is new, or that storage virtualisation is something you might consider AFTER doing server virtualisation. Both of which are not true.
IBM embraced storage virtualisation in June 2003 when we announced our SAN Volume Controller (the IBM SVC). I even found a CNET.com article from way back then. You can find it here (the image below is a screen capture of that CNET website).
IBM's SVC product has been enhanced repeatedly since 2003 with an enormous list of supported host servers and backend storage controllers. We have added new functions every year including Easy Tier, split cluster, VAAI, an enhanced GUI and a new form factor for the SVC code in the form of the Storwize V7000.
So let me give you a seventh reason for adopting storage virtualisation: A vendor who has shown genuine support for this technology. No vendor has embraced storage virtualisation with more enthusiasm than IBM. We have an industry leading solution with phenomenalSPC benchmarks, an enormous number of case studies and an architecture that does not lock you in. Indeed it is an architecture that can grow as you grow and that can be upgraded without disruption.
So please consider storage virtualisation from IBM, using either the SVC or the Storwize V7000. If your in Australia, we have demo centers dotted around the country. Many of our Business Partners can also demonstrate IBM storage virtualisation using their own Storwize V7000s. If your in Melbourne feel free to give me a call and schedule a time to drop into Southgate.
The Storwize V7000 and SVC release 6.1 introduced a new WEB GUI interface to assist with service issues, known as the Service Assistant. The Service Assistant interface is a browser-based GUI that is used to service your nodes. Much of what you traditionally did with the SVC front panel can all be done using the Service Assistant GUI. You can see a screen capture of the Service Assistant below:
While I would like to be optimistic and hope that you will never have to use the Service Assistant, you should always ensure your toolkit is equipped with every possible tool. I say this because one thing I have noted is that the majority of installs are not configuring the Service Assistant IP addresses. This is particularly apparent as clients upgrade their SVC clusters to release 6.1.
By default on Storwize V7000, the Service Assistant is accessible on IP addresseshttps://192.168.70.121 for node 1 and https://192.168.70.122 for node 2 (don't try and point your browser at them right now, as your network routing won't work - you would need to set your laptop IP address to the same subnet and be on the same switch. Details to do that are here). For SVC there are no default IP addresses, although we traditionally asked the client to configure one service address per cluster. The best thing for you to do is approach your network admin and ask for two more IP addresses for each Storwize V7000 and/or SVC I/O group. Once you have these two extra IP addresses, record them somewhere and then set them using the normal GUI.
Its an easy five step process as shown in the screen capture below. Go to the Configuration group and then choose Network (step 1). From there select Service IP addresses (step 2) and the relevant node canister (step 3). Choose port one or port two (step 4) and then set the IP address, mask and gateway (step 5).
You can also set them using CLI (replace the word panelname with the panel name of each node, which you can get using the svcinfo lsnode command).
If you forget these IP addresses, you can reset them using the same CLI commands or using the Initialization tool as documented here.
Finally having set the IP addresses, visit the service assistant by pointing your browser at each address. This is just to confirm you can access it. You logon with your Superuser password. With the process complete, ensure the IP addresses are clearly documented and filed away. So now if requested, you will be able to perform recovery tasks (in the unlikely chance they are needed). If for some reason your browser keeps bringing you to the normal GUI rather than the Service Assistance GUI, just add /service to the URL, e.g. browse to https://10.10.10.10/service rather than https://10.10.10.10.
So what should you do now?
If your an SVC customer on SVC code version v5 and below, please get two IP addresses allocated for each SVC I/O group, so you can set them the moment you upgrade to V6. Do this once the upgrade is complete.
If your an existing Storwize V7000 client or an SVC client already on V6.1 or V6.2 code, then hopefully you should already have set the service IP addresses. If not, please do so and test them.
I recently got a great email from an IBMer in the Netherlands by the name of Jack Tedjai. He sent me two screen shots, taken with the new performance monitor panel (that comes with the SVC and Storwize V7000 6.2 code). He wrote:
I am working on a project to migrate VMware/SRM/DS5100 to SVC Stretch Cluster and one of the goals is to prevent using ISL (4Gbps) and VMware Hypervisor/HBA load during the migration. For the migration we are using VMware Storage vMotion. To minimize the impact of the migration on production, we tested VAAI for Storage vMotion and template deployment and it worked perfectly.
So whats this all about? Well one of the improvements provided with VAAI support is the ability to dramatically offload the I/O processing generated by performing a storage vMotion. Normally a storage vMotion requires an ESX server to issue lots of reads from the source datastore and lots of writes to the target datastore. So there is a lot of I/O flowing from ESX to the SVC, and then from the SVC to its backend disk. What you get is something that looks like the image below. In the top right graph we have traffic from SVC to ESX (host to volume traffic). In the bottom right graph we have traffic from the SVC to its backend disk controllers (DS5100 in this case). This is SVC to MDisk traffic.
When we add VAAI support to the SVC, we suddenly change the picture. Suddenly VMWare does not need to do any of the heavy lifting. There is almost no I/O between VMWare and the SVC (no host to SVC volume traffic) related to the vMotion. The SVC is still doing the work, but it is happening in the background without burning VMWare CPU cycles or HBA ports (in that there is still SVC to MDisk traffic).
This difference translates to: Faster vMotion times, far less SAN I/O and far less VMware CPU being used on this process.
So do VMware support this? They sure do! Check this link here. It currently shows something like this (taken on June 23, 2011):
So what are your next steps?
Upgrade your Storwize V7000 or SVC to version 6.2 code. Download details arehere.
Download and install the VAAI driver onto your ESX servers. You can get it from here. If your already using the XIV VAAI driver you need to upgrade from version 22.214.171.124 to version 1.2. There is an installation guide at the same link.
And the blog title? It means friendly greetings in Dutch. So to Jack (and to all of you), vriendelijke groeten and please keep sending me those screen captures.
I thought I would quickly check out two of the announced features of the 6.2 release: the new Performance Monitor panel and support for greater than 2 TiB MDisks. So on Sunday I got busy and upgraded my lab Storwize V7000 to version 126.96.36.199.
Remember that in nearly every aspect the firmware for the SVC and Storwize V7000 are functionally identical, so while I am showing you a Storwize V7000, it equally applies to an SVC.
Firstly I tried the performance monitor panel, and what better way to show you what I saw than on YouTube? This is my first YouTube video so please forgive me if its not slick. I started the performance monitor and captured two minutes of performance data using Camtasia Recorder. Because it is fairly boring to stare at graphs slowly moving right to left, I then sped it up eight times, and this is the result:
The video is shot in HD, so if what your seeing is grainy or hard to read, change the display to 720p or 1080p. Now if you want to see the performance monitor at its actual speed, here is the original normal speed video. Remember this is the same video as above, just slower. It can also be viewed in 720p.
The top right hand quadrant is volume throughput in MBps as well as current volume latency and current IOPS.
The bottom left hand quadrant is Interface throughput (FC, SAS and iSCSI).
The bottom right hand quadrant is MDisk throughput in MBps as well as current MDisk latency and current IOPS.
You will note that each metric has a large number (which is the current metric in real time) and a historical graph showing the previous five minutes. You can also change the display to show either node in the I/O group.
I found the monitor to be genuinely real time: the moment I changed something in the SAN (such as starting or stopping IOMeter or starting or stoping a Volume Mirror), I immediately saw a change.
Greater than 2 TB MDisk support
Next I logged onto my lab DS4800 and created two 3.3TiB volumes to present to the Storwize V7000. I chose this size because I had exactly 6.6 TiB worth of available free space on the DS4800 and I wanted to demonstrate multiple large MDisks. On versions 6.1 and below, the reported size of the MDisks would have been 2 TiB (as I discussedhere). Now that I am on release 6.2 with a supported backend controller, I can present larger MDisks. In the example below you can clearly see that the detected (and useable size) is 3.3 TiB per MDisk.
What controllers are supported for huge MDisks?
The supported controller list for large MDisks has been updated. The links for Storwize V7000 6.2 are here and for SVC here. If your backend controller is not on the list, then talk to your IBM Sales Representative about submitting a support request (known as an RPQ).
There was a time when 32 bits was considered a lot. A hell of a lot.
With 32 bits, you can create a hexadecimal number as big as 0xFFFFFFFE (presuming we reserve one bit). In decimal that's 4,294,967,295. Hey... imagine a bank account balance that big? If you use 32 bits to count out 512 byte sectors on a disk, you could have a disk that's 4,294,967,295 times 512... or 2,199,023,255,040 bytes! That's sounds huge, right?
Well... actually...no... that's 2 TiB, which most people would refer to as 2 Terabytes. Mmm.. Suddenly I am less impressed (still wouldn't mind that as a bank account though).
Now there are plenty of running Systems that still cannot work with a disk that is larger than 2 TiB. One of the more common is ESX. I am presuming this limitation is going to disappear, so Storage susbsystems need to be ready to create volumes that are larger than 2 TiB.
The good news is that with the May 2011 announcements, IBM is removing the last 2 TiB sizing limitations from its current storage products. There appears to have been some confusion in the past, so I thought I would go through and be clear where each product is at:
Firmware version 07.35.41.00 added support to create volumes larger than 2 TB. The maximum volume size is limited only by the size of the largest array you can create. This capability has been available for some time and hopefully you are already on a much higher release.
DS4000 and DS5000
Firmware version 07.10.22.00 added support to create volumes larger than 2 TB. The maximum volume size is limited only by the size of the largest array you can create. This capability has been available for some time and hopefully you are already on a much higher release.
DS8700 and DS8800
The DS8700 and DS8800 will support the creation of volumes larger than 2 TB once a code release in the 6.1 family has been installed. With this release you will be able to create a volume up to 16 TiB in size. The announcement letter for this capability is here.
The volume size on an XIV is limited only by the soft limit of the pool you are creating the volume in. This allows the possibility of a 161 TB volume.
SVC and Storwize V7000
These two products have two separate concepts:
Volumes (or VDisks) that hosts can see.
Managed disk (or MDisks) that are presented by external storage devices to be virtualized. Within this there are two further categories: - Internal MDisks created using the Storwize V7000 SAS disks. - External MDisks created by mapping volumes from external storage (such as from a DS4800).
SVC and Storwize V7000 Volumes (VDisks).
Prior to release 5.1 of the SVC firmware, the largest volume or VDisk that you could create using an SVC was 2 TiB in size. With the 5.1 release this was raised to 256 TiB, as announced here. When the Storwize V7000 was announced (with the 6.1 release) it also inherited the ability to create 256 TiB volumes.
Because the Storwize V7000 has its own internal disks, it can create RAID arrays. Each RAID array becomes one Mdisk. This means the largest MDisk we can create is limited only by the size of the largest disk (currently 2 TB), times the size of the largest array (16 disks). This means we can make arrays of over 18 TiB in size (using a 12 disk RAID6 array with 2 TB disks). Thus internally the Storwize V7000 supports giant MDisks. We can also present these giant MDisks to an SVC running 6.1 code and the SVC will be able to work with them.
SVC and Storwize V7000 External Managed Disks.
When presenting a volume to the SVC or Storwize V7000 to be virtualized into a pool (a managed disk group) we need to ensure two things are confirmed. Firstly you need to be on firmware version 6.2 as confirmed here for SVC and here for Storwize V7000. Secondly that the controller presenting the volume has to be approved to present a volume greater than 2 TiB. From an architectural point of view, MDisks can be up to 1 PB in size as confirmed here, where it says:
Capacity for an individual external managed disk
Note: External managed disks larger than 2 TB are only supported for certain types of storage systems. Refer to the supported hardware matrix for further details.
I recommend you go to the supported hardware matrix and confirm if your controller is approved. The links for Storwize V7000 6.2 are here and for SVC here. As of this writing, the list has still not been updated, but I am reliably informed it will include the DS3000, DS4000, DS5000, DS8700 and DS8800. It will not initially include XIV, which will come later. Please also note the following:
Support for giant MDisks (greater than 2 TiB) is firmware controlled. If the controller (e.g. a DS5300) presenting a giant MDisk is not on the supported list for your SVC/Storwize V7000 firmware version, then only the first 2 TiB of that MDisk will be used.
If your already presenting a giant MDisk (and using just the first 2 TiB), then just upgrading your SVC/Storwize V7000 firmware won't make the extra space useable. You will need to remove the MDisk from the pool, then do an MDisk discovery and then add the MDisk back to the pool. All of this can of course be done without disruption, using the basic data migration features we have supported since 2003.
What to do in the meantime?
If your currently using an SVC or external MDisks with a Storwize V7000, then you need to work within the 2 TiB MDisk limit (except for Storwize V7000 behind SVC). The recommendation is a single volume per Array for performance reasons (so the disk heads don't have to keep jumping all over the disk to support consecutive extents on different parts of the disk). This can require careful planning. For instance using 7+P RAID5 Arrays of 450 GB drives makes an array that is over 3 TB. What to do in this example?
Divide it in half? (by creating two 1.5TB volumes)
Waste space? (a whole 1 TB)
Use smaller arrays? (a 4+P array of 450GB disks is 1.8 TB)
The answer is that where possible, create single volume arrays using 4+P or larger. If the disk size precludes that, then create multiple volumes per array and preferably split these volumes across different pools (MDisk groups).
Anything else to consider?
Well first up, will your Operating System support giant volumes? Googling produces so much old material that it becomes hard to nail down exact limits. For Microsoft, read this article here. For AIX check out this link. For ESX, check out this link.
Second of course is the consideration of size. File systems that utilize the space of giant volumes could potentially lead to giant timing issues. How long will it take to backup, defragment, index or restore a giant file system based on a giant volume (the restore part in particular)? Outside the scientific, video or geo-physics departments, are giant volumes becoming popular? Are they being held back by practical realities or plain fear? Would love to hear your experiences in the real world.
And a big thank you to Dennis Skinner, Chris Canto and Alexis Giral for their help with this post.
Storage IT offers up many choices, some of which provoke argument so heated, you could almost describe the adherents as religious. I think you might know the sort of arguments I am talking about:
File vs Block I/O
iSCSI vs Fibre Channel
CLI vs GUI
OK.... so maybe that last one isn't quite in the same league. But it is still fascinating to see the variation in usage patterns from sites where every command (of any description) is run via a command line interface (a CLI), to sites where the CLI is viewed with either great fear... or even greater distaste. There are those who view the CLI as... well... so 1970s.....
But the reality is that the CLI will always be with us for one principal reason: scripting. If you cannot script it, you cannot automate it (well actually thats not true, but stick with me here, I am on a roll). Every single major implementation I have ever done (whether it be SVC, XIV, DS8000), I have automated with scripting. I regularly use the concatenatecommand in Excel to build large numbers of commands that I can then run as a script.
So its pleasing to see that all of our products are working towards making the scripters life even easier. For example the XIV has offered a command log in the GUI for some time. I blogged about it here. You simply do a command once in the GUI and then consult the log to find the syntax, making scripting very easy:
With last years release of SVC 6.1 and Storwize V7000, we added this level of smarts to those two products as well. Now every command you run in the GUI will offer you the exact CLI command that was used under-the-covers to do this work. Simply toggle the details tab on the completion panel to see the command (or toggle it back to hide it!).
This weeks announcement of release 6.2 of the SVC and Storwize V7000 firmware, has brought in two more important usability improvements:
Now when logging onto the CLI using individual user-ids, you can logon using the actual user-id itself, rather than admin. This change has been a long time coming and removes the confusion generated by logging onto the GUI as sayanthony, but then logging into a matching CLI session as admin. Now you would logon to either interface as anthony.
Now when issuing CLI commands, you have the choice to drop the svctask and svcinfo headers. So instead of issuing the command svcinfo lsnode, you can issue the command lsnode. Both choices remain valid (so we don't break your existing scripts). Making this change is part of a bigger plan to move to a more common CLI.
And there are more improvements coming, so as always, watch this space....
.... and please... share with me... are you a GUI... or a CLI person? Whats your reasoning behind your choice?
The May 9 announcement that SVC and Storwize V7000 will support VAAI is very welcome news. The fundamental point is that the SVC and Storwize V7000 virtualise external storage. This means that the mountains of DS3000, DS4000, DS5000, AMS1000s, CX3s, etc, that are currently being virtualized behind these products, will inherit VAAI as soon as the virtualization layer supports it. This is yet another feature to add to the list of functions that IBM Storage virtualization can provide, such as: EasyTier; Thin Provisioning; multiple consistency groups; snapshots; remote mirroring; dynamic data relocation... the list goes on.
In addition we are releasing a plug-in for vCenter that enables VMware administrators to manage their SVC or Storwize V7000 from within the VMware management environment
Functions will include:
Volume provisioning and resizing
Displaying information about volumes
Viewing general information about Storwize V7000 and SVC systems
Receiving events and alerts for Storwize V7000 systems and SVC attached to vSphere
The Storwize V7000 and SVC plug-in for vCenter will also supports virtualized external disk systems
The plug-in will be available at no charge on June 30 (for Version 6.1 software) and July 31 (Version 6.2). Here is a sneak peak of what it will look like:
And to get an independent viewpoint have a read of Stephen Fosketts blog entry here:
With the announced release of DS8000 6.1 code, IBM has moved its three major storage systems to a common GUI platform. This makes me think of aircraft manufacturers who utilize a common cockpit design. For airlines, this is major drawcard when choosing aircraft models. It cuts down on training costs for your pilots. Except in storage IT, there is a major difference in motivation....
First and foremost, the design of the XIV GUI (that has inspired such dramatic change in IBMs other GUIs), was made possible, not by clever XIV GUI developers (don't get me wrong - they ARE clever), but by a remarkably user-friendly architecture. The XIV GUI is a miracle of ease-of-use for end users, made possible because first and foremost, by design, the XIV made it almost impossible to make it hard.
The good news for Storage administrators, is that unlike a jet aircraft, where a pilot needs to spend hundreds of hours in the cockpit before they are considered potentially competent, the XIV GUI can be picked up in minutes and lends itself very well to casual contact. You don't need to keep using it to stay competent.
The challenge for IBM was take more complex products, which require more user decisions, and make the usage experience just as easy. To add to this, the SVC and DS8000 GUIs were driven by WebSphere. Changing these GUIs would require a complete re-write to employ Java script.
First off the rank was the SVC and Storwize V7000. With the release last year of the SVC 6.1 update, the transformation was nothing less than remarkable. End user experience ruled every decision. The key again is that the user does not need to spend hundred of hours learning this GUI or re-learning it every time they go to perform a configuration task. Everything is in its right place. Its much more than an XIV-like GUI. Its a GUI that took the ease of use experience of the XIV and used that to inspire something just as remarkable.
With the release of the 6.1 update for the DS8000, we complete another fundamental step towards a truly common GUI. The DS8000 GUI has undergone a complete re-write. Essentially it has been rebuilt from the ground up. This highlights something fundamental: It confirms the DS8000 has a very strong roadmap.
As you can see from the image below, the transformation from the old design (to the left) to an ease of use model is complete:
In short it a common flight deck, that almost anyone can fly.
A question I get routinely asked relates to Windows disk partition alignment with XIV. If you don't know what I am talking about, take some time to read these very useful pages from our friends at Microsoft. Once you have had a look, come on back and read my perspective.
Back already? Hopefully you now know that disk partition alignment is all about starting an IO at a logical block address that best matches how the underlying hardware stores your data. So now your wondering, what does this have to do with XIV? Well XIV has two concepts that relate to this: cache and partitions.
XIV cache (the server memory used to speed up reads and writes) is organised into 4 KB blocks (which is nice and small).
So the XIV cache does not care about disk alignment.
But when it comes to writing and read from disk, the XIV writes data into chunks of consecutive logical block addresses (LBAs) that we call partitions. These partitions are 1 MiB in size. What does that concept mean? It means the magical number for XIV is 1024 KB or 1 MB. (actually KiB and MiB, but for the sake of ease, I will stick to the naming used by Microsoft. Given this number is fairly large (other hardware often aligns to 32KB, 64KB or 256KB), for XIV this reduces the potential impact of misaligned partitions. Which is good.
Correct Windows Disk Alignment could give up to a 7% performance improvement when using an offset of 1024 KiB. (1 MiB). I need to be clear, that's not a guaranteedimprovement of 7%. It's a maximum possible improvement. Your particular server will see an improvement somewhere between 0% and 7%. It depends on your workload patterns. The more small and random your workload, the more useful setting the 1024 KB offset will be. The more sequential your workload, the less useful it will be, as only the first and last parts of an I/O could potentially be misaligned. This mis-alignment could equate to a tiny percentage of extra work for the XIV. Sadly there is no metric you can display to detect how much impact misalignment is actually having.
So should you do it? The good news is that new volumes created under Windows 2008 prefer the 1 MB boundary. So a fresh install should already be using the correct values. The bad news is that volumes created under earlier Windows Operating Systems (Such as Windows 2000 and 2003) will almost certainly be misaligned, and correcting the alignment is destructive to the data in the partition.
How to check alignment at the host? Here is an example:
I start diskpart:
Microsoft DiskPart version 6.1.7600
Copyright (C) 1999-2008 Microsoft Corporation.
On computer: ANTHONYV-PC
I list my disks. In this example I have two disks installed in my laptop. I select disk 0:
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ------------- ------- ------- --- ---
Disk 0 Online 238 GB 5724 MB
Disk 1 Online 232 GB 1024 KB
DISKPART> select disk 0
Disk 0 is now the selected disk.
Now I list the partitions and see the offset for each one.
Partition 1 has an offset of 1024 KB, which is 1 MB, which is perfect for XIV. Partition 2 has an offset of 101 MB, which is still on the 1MB boundary (it was pushed there by the combination of the size of the first partition (100 MB) and its offset (1 MB). So this is perfect.
The DS8000 (regardless of model), also prefers 64 KB offsets. The DS8000 use the concept of logical tracks where each logical track is 64KB.
The DS3000/DS4000/DS5000 range allow the user to set the segment size of a logical volume on creation. The setting that you define should match the segment size defined for the logical drive being used. In the example below, it is 64 KB.
What about VMWare?
The answers are no different. Misalignment can indeed make a difference to client performance. Check this link from NetApp and this document from VMware:
I searched around looking for an image to highlight the theme of alignment. I found this image in the IBM archives for the IBM Mass Storage Facility announced back in 1974. I am sure this product had some interesting alignment challenges.
When IBM first released the Storwize V7000, we announced it was capable of supporting ten enclosures, but would on initial release support only five. We stated that this restriction would be lifted in Q1.
The good news is that this restriction is indeed now lifted by the release of Storwize V7000 software version 188.8.131.52, which is available for download from here: 184.108.40.206 Code
This new level also contains an additional enhancement which I think users will really like, called Critical Fix Notification. The new Critical Fix Notification function enables IBM to warn Storwize V7000 and SVC users if we discover a critical issue in the level of code that they are using. The system will warn users when they log on to the GUI using an internet connected web browser. It works only if the browser being used to connect to the Storwize V7000 or SVC, also has access to the Internet. (The Storwize V7000 and SVC systems themselves do not need to be connected to the Internet.) The function cannot be disabled (which is a good thing) and each time we display a warning, it must be acknowledged (with the option to not warn the user again for that issue).
Its been a busy week! We just completed our first IBM System Storage Symposium in Sydney.... and it was a great success. Thanks to everyone who attended and presented. Meanwhile... there are quite a few updates to the IBM Support site, and as usual Rob Jackard from the ATS Group has created a great summary which I will reproduce here. Things of particular note are:
If you installed SVC code version 220.127.116.11 please update to 18.104.22.168 as per this issue
There has been a lot of chatter on the blog verse about so called vendor blockers (I think you can shorten that to vblock). This is the idea that a pre-blessed solution is the safest way to build infrastructure in the data centre. I can see the attraction, but I suggest the best way to get there is with the vendor who is most willing to work with the widest range of their competitors. And you cannot get much wider than IBM SVC. . IBM has been supporting SVC in a mixed OS and hardware environment since 2003. The IBM Storwize V7000 inherits all of the interoperability testing done over 7 years with IBM SVC. This is an astonishing way to bring a new product into the marketplace. I cannot think of a new midrange entrant that has somehow managed to get the same level of interoperability testing and ISV integration on the very day of its birth. When you look at the picture below you can see the depth of IBM's support matrix for SVC and its cousin, the Storwize V7000. I have on occasion raised a laugh from an audience when I describe IBM SVC as a vendor independent virtualization layer. But the picture doesn't lie... with the IBM Storwize V7000 or IBM SVC... there is no vendor blocking. Plus with the ability to move your data out of the IBM virtualization layer (using the the migrate to image command), you can remove IBM from the picture at any time... meaning no vendor blocking... and no vendor locking.
Its that time again! Rob Jackard from the ATS Group has shared with me his list of IBM Storage related updates. Storwize V7000 already has a huge amount of great material out there, links are below. Have a quick look... you may see links that are relevant to you!
(2010.10.13) Support Matrix for Subsystem Device Driver, Subsystem
Device Driver Path Control Module, and Subsystem Device Driver Device Specific
(2010.09.06) IBM SDDPCM- Open HyperSwap status may report incorrectly via the Tivoli Productivity Center for Replication GUI, If the HyperSwap was incomplete and then another unplanned HyperSwap occurs, both copies of the data will be corrupted.
(2010.09.29) RETAIN Tip# H197680-
VIOS 2.1.3 support for BladeCenter
hosts removed from System Storage Interoperation Center (SSIC) web site - IBM
System Storage DS3512 (Type 7146), IBM System Storage DS3524 (Type
The SAN Volume Controller (SVC)
has truly come of age in 2010.
With release 6.1 we not only combine it with the very mature RAID adapters of
the DS8000 to produce the Storwize
V7000, but we continue to develop the SVC as the world’s leading Storage
For existing SVC clients, you will be able to download SVC code 6.1 once it
reaches the Generally Available (GA) date, which is scheduled for November 12,
2010. The announcement letter is here: http://ow.ly/2TTVB
Whyte is planning a detailed set of blog posts about the improvements from
both a hardware and software perspective. So I thought I would just list the key changes until he gets round to posting his blogs:
Names have been changed to make SVC GUI similar to more standard names, i.e.
vdisk is now volume, mdisk group is now storage pool, error is now event.
New SVC Graphical
User Interface (GUI)
Fantastic new XIV like GUI that you access direct from the SVC nodes (not from
Master Console). This means you just
point your browser at the SVC IP cluster IP address .
Removal of the16 character name length
restriction (raised to 63), raised WWNN limit to 256, max vdisk now 1PB, many
Easy Tier & SSDs
in SVC v6.1
Easy Tier added at no extra charge. Note: Users of solid-state devices (SSDs) on the SAN Volume Controller 2145-CF8
can not install SVC 6.1.0 at this time. This shoiuld be corrected in the next minor release.
Provisioning new SVC
Changes to make it easier to setup a cluster at install.
SVC Code Upgrades
Removal of GUI package. Now you just update SVC code.There will be a tool to update 22.214.171.124 to 6.1 As with 5.1 code... you cannot install 6.1 on a 2145-4F2 node as the 4F2 cannot run the 64 bit OS used by 5.1 and 6.1
SVC to Backend Controller
Changes in block size SVC uses with back end disk (increase from 32 KB to 256
SVC and Storwize
Support for Storwize V7000 as backend disk to SVC.
Storage Controller /
Host / Tool Interoperability
Added more host and disk support (like VSphere 4.1, EMC V-Max, Compellent and
new Fujitsu Eternus disk controller models).
Addition of the SCSI write-same command for use with VSphere 4.1.
. And best of all.... The firmware update will be free of charge... just download it and install it... . 16/10/2010 update - added some extra comments and improved format.
IBM is announcing a set of remarkable new storage products and enhancements.
Quite simply, the focus is on delivering three things:
announcements show not only IBMs significant investment in storage but also
IBMs tremendous depth of knowledge and experience.
rapidly see that the focus is on our new Midrange Storage product, the Storwize
V7000.However this is only one of
four major releases that you will see (plus many more other incremental
releases).From a product perspective
the big new announcements are:
XIV. The XIV
will support the VMWare VAAI API by updating the firmware to version
10.2.4.To remind you what I am talking
about, check out my earlier blog on this subject here.
is a major new product offering in the midrange space. It takes the
intelligence and history of the SVC; brings in some disk controller technology
from the DS8000; adds SAS version 2 disk enclosures; provides the sub-LUN
performance benefits delivered by Easy Tier; uses a simplified GUI influenced
strongly by XIV and has a simplified licensing structure.This is all put into a 2U modular form
factor.Because the Storwize V7000 uses
the same code base as the SAN Volume Controller (SVC), it brings all the smarts
of SVC including virtualized disk (using both internal SAS disks and external
storage controllers), thin provisioning, transparent data migration and
mirroring (including Metro and Global Mirror). Right now there is no RACE technology in
the Storwize V7000 (despite IBM using the Storwize brand).But I think you can take the name as a hint
of things to come.
DS8800.This is a fantastic incremental new
development in the DS8000 family.It
takes the long history of DS8000 development and combines it with small form
factor (2.5”) SAS version 2 disks connected via 8 Gbps host adapters and 8 Gbps
device adapters.The performance numbers,
the environmental and floor-space requirements are all improved by a
significant factor. It positions the
DS8000 for many years of new functions and features.
SVC. For SVC we are releasing SVC version
6.1.This is a major software update to
the SVC code.It delivers a remarkable
new GUI with Easy Tier and a whole raft of functional improvements.
announcements will include enhancements to TPC, IBM Director, the DS3500 and
as I have the announce letter URLS, I will post them.There is clearly plenty more to come.
This year is the year IBM brought Easy Tier to the market place. Easy Tier is not a product.... or a piece of hardware: Easy Tier is a software technology... and it is smart technology. It comes out of IBM's Almaden Research Centre and is designed to fit into any storage product that has a modular software architecture (think internal Unix operating systems like AIX or Linux). The first product to use and deliver Easy Tier is the DS8700. It is real technology and you can buy it and start using it right now. Here are some salient points which differentiate Easy Tier from other vendors offerings: .
Easy Tier provides benefits for both read and write I/O, not just read I/O.
Easy Tier can scale up by adding extra SSDs. We are not held back by how much RAM or Flash we can jam into limited slots.
Easy Tier doesn't add or require any specialised hardware.
Easy Tier is not just extra cache.
Easy Tier is "non-disruptive Bidirectional smart placement of data.
Easy Tier is a no-charge add-on. So we can order the Easy Tier feature for $0.
. Want to learn more? . Check out the IBM Redpaper. Or the Youtube video. Or the Product Page Or this commentary from the Mesabi Group . And remember... right now you can get it in DS8700.... but the Easy Tier design is well suited to a wider range of products. Watch this space.
So its that time of the month again. Rob Jackard from the ATS group does a fantastic job summarizing changes to the IBM Storage Support site and you get all the benefit of his hard work (via me!). . So cast your eyes down the list and look for issues that may affect you.... . AIX:
(2010.08.21) AIX Support Lifecycle Notice- AIX 5.3 TL9 & TL10.
NOTE-1: After November 2010, IBM will no longer provide generally available fixes or interim fixes for new defects on systems at AIX 5300-09 (applies to all Service Packs within TL9). Sometime after May 1, 2011, IBM will no longer provide generally available fixes or interim fixes for new defects on systems at AIX 5300-10 (applies to all Service Packs within TL10).
NOTE-2: As a reminder, IBM is no longer providing generally available fixes or interim fixes for new defects on systems at AIX 5300-06, AIX 5300-07 or AIX 5300-08.
(2010.08.21) AIX Support Lifecycle Notice- AIX 6.1 TL2 & TL3.
NOTE-1: After November 2010, IBM will no longer provide generally available fixes or interim fixes for new defects on systems at AIX 6100-02 (applies to all Service Packs within TL2). Sometime after May 1, 2011, IBM will no longer provide generally available fixes or interim fixes for new defects on systems at AIX 6100-03 (applies to all Service Packs within TL3).
NOTE-2: As a reminder, IBM is no longer providing generally available fixes or interim fixes for new defects on systems at AIX 6100-00 or AIX 6100-01.
NOTE: Users of MPIO storage running the 5300-12 TL- An operation to change the preferred path of a LUN could hang. A similar hang could be experienced during LPAR migration where it will try to switch the preferred paths. Install APAR IZ77907.
NOTE: Users of MPIO storage running the 6100-02 TL- An operation to change the preferred path of a LUN could hang. A similar hang could be experienced during LPAR migration where it will try to switch the preferred paths. Install APAR IZ77908.
(2010.08.17) NPIV clients of SSDPCM hosts
may experience permanent application errors during SVC concurrent code upgrade
or node reset with certain APARs and SSDPCM versions. The risk, although rare,
exists in any AIX SSDPCM host or client.
NOTE: The changes made for VIOS client
hangs in Technote SSG1S1003579 require additional AIX driver and SDDPCM code
updates for a specific SVC error condition.
Its a given today that SAN implementations have redundancy by design. When we sell SAN switches we always urge clients to use dual fabrics. Its accepted industry best practice. By having hosts with dual HBAs, each host can attach to both fabrics and survive the failure of a SAN switch. . Equally when we sell SAN Volume Controller (SVC) we always sell the SVC nodes in pairs. Each SVC node can run an SVC I/O group stand-alone, which again allows us to survive a node failure and do things like concurrent firmware updates. . So far so good. But common practice appears to be that when installing redundant hardware, that we place all the hardware into one rack. In fact I routinely look in a rack and see Switch One directly mounted above Switch Two. I see the same with SVC clusters, all the nodes often jammed into the same rack, mounted one on top of the other. . Is this a good idea? I suspect 99.9% of the time it makes no major difference. But its the 0.1% that can cause great pain. Imagine going to work on a failed switch and then accidentally powering down the working one. If each switch was in a separate rack the likelihood of doing this is significantly reduced. I recently visited a major Australian Bank where each of their Fibre Channel directors were located on opposite ends of a long row of racks, several meters apart. The separation of distance guaranteed that any event in one rack could not possibly affect the other rack. Good design... by design..... I liked it. There is no reason you cannot install SVC nodes in the same way, each node in a separate rack. Just don't separate them too far apart. When servicing a node you like to be able to see what the front panel of the other node is displaying. Having said that, SVC split cluster (where we separate the nodes across or between buildings) truly separates the nodes for the highest levels of availability.
Rob Jackard from the ATS Group has kindly supplied me with this great summation of recent updates to various parts of the IBM Support sites. Its worth just running your eyes down the list to see if there is anything that might apply to you and your environment. . For XIV users, please update to the latest GUI release, version 2.4.3a (see the link below). The release notes (which you can also find at the link below) detail an important fix that could prevent an outage when mapping new volumes to a clustered group of hosts which uses private mappings, (which you might do if you have separate dump or boot drives for clustered hosts).
DS3000 / DS4000 /
(2010.06.21) Best Practices for Running an
Oracle Database on an IBM Midrange Storage Subsystem.
(2010.06.07) All hdisks and vpath devices must be removed from host system before upgrading to SDD host attachment script 126.96.36.199 and above. All MPIO hdisks must be removed from host system before upgrading to SDDPCM host attachment script 188.8.131.52.
SDDPCM (Subsystem Device Driver Path Control Module) is the multi-pathing plug-in for AIX 5.3 and 6.1. Customers who use IBM SVC, DS6000, DS8000 and/or ESS800 use this package to allow the operating system to handle multiple paths from OS to storage. The good news is that this plug-in is supplied free of charge. The bad news is that it is not included with AIX fixpacks. What this means is that while you may be dilligent with keeping AIX up to date, you may miss SDDPCM in the process.
There are two good reasons to keep SDDPCM in mind when planning updates:
1) Planning an upgrade from AIX 5.3 to 6.1
Before the AIX OS is upgraded, SDDPCM must be uninstalled and then reinstalled after the upgrade. There are cases when the host attachment script must also be uninstalled and reinstalled. This is explained in the SDD Users Guide found here:
If you have already upgraded from AIX 5.3 to 6.1 but you are still using the AIX 5.3 version of SDDPCM, you may need help from IBM before you can upgrade your SDDPCM to the AIX 6.1 version. This will come in the form of some special scripts. 2) General SDDPCM maintenance
As I noted in my previous blog entry, there are quite a few SDDPCM flashes out there right now. You need to check these out and ensure you are not exposed to the issues that are corrected by later versions of SDDPCM. Check out the flashes listed here (or read my previous blog entry): http://www-01.ibm.com/support/search.wss?rs=540&tc=ST52G7&dc=D600&dtm
What about SDD (Subsystem Device Driver) for AIX? Prior to AIX 5.2 FP5, AIX did not offer native multi-pathing (MPIO). This meant that each hardware vendor had to offer their own third-party software to handle multiple paths. To achieve this with the ESS (Shark), IBM released a product called DPO (Data Path Optimiser). This product became SDD and was made available for a wide variety of operating systems.
When AIX offered MPIO, IBM then also offered a vendor plug-in (Path Contol Module) for native AIX MPIO which IBM called SDDPCM. This means you have two choices with AIX: SDD or SDDPCM. If your considering which is best, SDDPCM is my preference. This is because it is native to the operating system and also better supports the possibility of co-existence of multiple PCMs. Note that migrating from SDD to SDDPCM is not supported by the VIOS at this time, so if your running VIOS you will need to stay put for now.