Because if a product uses a 32 bit counter to record uptime, and that counter records a tick every 10 msec, then that 32-bit counter will overflow after approximately 497.1 days. This is because a 32 bit counter equates to 2^32, which equals 4,294,967,296 ticks. If a tick is counted every 10 msec, we create 8,640,000 ticks per day (100*60*60*24). So after 497.102696 days, the counter will overflow. What happens next depends on good programming: normally the counter just starts again, but worst case a function might stop working or the product might even reboot.
Fortunately we are seeing less and less of these issues but just occasionally one still slips out. Recently IBM released details of a 994 day reboot bug in the ESM code of some of their older disk enclosures (EXP100, EXP700 and EXP710). Details about this bug can be found here. What I find interesting is the number of days it takes to occur, since 994 is actually 497 times two. This suggests that this product records a tick every 20 msec. This meant we got past 497 days without an issue but hit a problem after exactly double that number. So if you still have these older storage enclosures, you will need to reboot the ESMs (after checking the alert).
I googled 497 to see what images that number brings up and was amazed to find the M-497 jet powered train. More details on this rather interesting attempt at speeding up the commute home can be found here and here. It adds a whole new meaning to keeping behind the yellow line.
Many of the preventable issues that occur in a SAN fabric can be avoided by using the right management and monitoring software. One way to get this software is to create or adapt open source packages. While I really like the idea (and price) of roll-your-own solutions, it is not always practical. Apart from the fact that you need to have staff with the relevant skills to do this, long-term maintenance can prove difficult when key people move on. Unfortunately the other extreme (which is far more common) is that many shops actually do nothing at all, ending up without any overall SAN management and monitoring methodology.
An ideal off the shelf solution alternative in a Brocade SAN fabric is to use IBM Network Advisor, the successor product to Data Center Fabric Manager (DCFM). IBM Network Advisor actually has its heritage in a great product called EFCM (Enterprise Fabric Connectivity Manager), that Brocade picked up when they bought McData . I loved working with EFCM and McData switches, especially the McData 6140, which was truly a great SAN director. When Brocade purchased McData they combined EFCM with their own Fabric Manager to create DCFM. They have since combined it with their Network switch management software to create Network Advisor, bringing things to a whole new level. The IBM announcement letter for this software is here.
Now the first thing you may be wondering is: OK so this software sounds great, but how much will it cost? The good news is that trying it out wont cost you anything. It's free to download and trial for 75 days. You can find the download site here.
To demo it, you can spin up a Windows 2008 guest from a template in your favorite Hypervisor. This means you don't even need to request separate hardware to do this trial.
So what benefits should you expect to see? Well first up I am talking about preventing issues like these:
Mistakes made when performing zoning updates
Failure to create regular configuration backups (which especially hurts after a switch failure)
Difficulties upgrading firmware or simply too many upgrades to get through
Poor (or no) switch and performance monitoring
Poor (or no) error notification (including notification back to IBM)
Difficulty collecting log data
Lack of report creation software
In some ways you can sum up the benefits of the software quite easily by looking at the three central menus of IBM Network Advisor: Configure, Monitor, Report.
To give you a view of some of the menu choices, you can see just how rich the options are:
From a configuration perspective you can manage the zonesets of all your fabrics from the one place. This means you don't need to jump between switches. More importantly it gives you a clear indication of what a zoning update is adding AND removing. Accidental removal of a required zone is a very common cause of zoning related SAN issues:
Do you mean to remove that zone?
It can automatically backup your switch configurations. Backing up your configs is frankly a mandatory task that is routinely never done. If a switch fails, then any customization and zoning (if it is a single switch fabric) is lost. This can be a major issue, especially if a business partner or former employee set the switch up. If we schedule a regular backup you won't need to remember, because IBM Network Advisor will do it for you:
Firmware updates also become a far simpler affair. IBM Network Advisor has a built-in FTP server and happily acts as a firmware repository. If you're facing a set of Kangaroo hops, this is a great way to make the whole process very very simple. It will perform compatibility checks before you start and also act as a repository for both firmware and release notes (which is a really nice touch).
From a monitoring perspective, the ability to set up call home to IBM is a huge advantage and a vital step in building a SAN with the highest levels of availability. The added bonus is that you can use IBM Network Advisor to generate a supportsave (a log offload file that you will invariably be asked for during trouble shooting) off every switch in your fleet in one go (you can also set it up to perform this on a regular basis), significantly boosting productivity and aiding in trouble shooting. You can also set up Fabric Watch across the entire fleet of switches, all from a single interface.
If you own DCFM already, then you are eligible for a free upgrade. If after trialing the software you feel that the significant availability benefits this software will give you are worth achieving, talk to your IBM Sales Rep or Business Partner to get a price. I personally think you will find it very reasonable, plus I guarantee that it will not be shelfware and will prove to be a vital tool in getting the most from your SAN.
But... if after trialling IBM Network Advisor you're still determined to try to avoid paying for software, then you could always consider the open-source alternative (rather than do nothing). Check out this document written by Andy Loftus and Chad Kerner from the National Center for Supercomputing Applications at the University of Illinois. It's a great example of a lessons learning document that describes how they built their own monitoring solution. You will find all of their documents and scripts here. As I said, roll-your-own might avoid vendor costs, but they have costs all of their own. Does your team have the skills, willpower and time to do this and maintain it? I would love to hear about your experiences either way.
I love PuTTY, it is one of my favourite pieces of open source software. For those who don't know what PuTTY is, it is a free implementation of Telnet and SSH for Windows and Unix platforms. Personally I use PuTTY for connecting to Storwize V7000s, SVCs, SAN switches and Unix servers in my lab and at clients.
Being someone who does implementation services, I want to always be sure about what I do, what I did and what affect it had. So keeping logs of every time I connect to any device is very important to me. I want to be able to go back in time to any point at any client I have ever worked with (sounds like time travel, but hopefully you get the idea).
I achieve this by changing the default behaviour of Putty so that every new Session I save uses my preferred default settings. Note all the screen captures below use version 0.61 of PuTTY (which came out July 2011).
First start up PuTTY. We are going to change the Default Settings (which I have highlighted):
Now select Logging from the left hand panel and change all the indicated settings (2, 3 and 4). The folder I use is C:\PuttyLogs but you could of course choose a different one. Note the &H_&Y&M&D&T means the session file name will be the host name, year, month, day and time for that session.
Now go back to Session, highlight Default Settings and hit Save:
Now every new session you open and/or save will automatically log your session output to the specified folder. Existing Saved Sessions will not be changed. You will need to update each of these separately.
When your buying a car it's worth opening the boot (that's the trunk for my American friends) and looking at the spare to see what kind it is. Car manufacturers sometimes choose to sacrifice a full spare to save space (or cost). I can understand the motivation, but there is nothing worse than having a flat and then finding that your spare is half the tyre you expected it to be.
With low end fibre channel switches there is a similar challenge when working to achieve the lowest cost and smallest footprint in a 1U form factor. The main way it expresses itself is with power supplies.
The IBM SAN24B-4 (a 24 port switch) is a case in point as it only has one fixed power supply. This of course means that if that power supply fails, then that switch will be down and the entire unit will have to be replaced. If you have dual fabrics (two switches) with dual pathed hosts, then this should not cause an application outage, but it may not be what you expected. You also need to ensure that each switch is connected to a different power rail (and/or UPS) to cater for building power issues.
IBM SAN24B-4 with power supply highlighted
How to avoid this? Purchase the slightly larger IBM SAN40B-4 (a 40 port switch) which comes with two hot swap power supplies as standard. It's a little more expensive (which makes sense as it has more ports and more hardware) but also offers redundant power and greater scaleability.
IBM SAN40B-4 Front View
IBM SAN40B-4 with one power supply being removed
Of course in the end you need to select the switch which matches your budget. The SAN24B-4 starts at only 8 ports active while the SAN40B-4 starts at 24 ports active, so the SAN24B-4 will always be a cheaper purchase. The SAN24B-4 is also much smaller, lighter (it weighs 4.35 kg versus 9.34 kg) and uses less power (48 W versus 84 W).
My preference? Well I would always choose to use a switch with dual redundant hot-swappable power supplies, but then I am not the person signing the cheques. What I would suggest is that if you choose the SAN24B-4 then you need to ensure your backing up your switch config (especially if your running single switch fabrics). You could look atSimply-Save-Your-SAN as one way to do this.
And no... that is not my car. By shear synchronicity I was thinking about this issue when I spotted this car in the carpark at Southgate. Timing is everything.
IBM have offered Brocade switch modules for their BladeCenters for many years. One common question I get asked regards which of these Switch Modules are supported with the IBM Storwize V7000 or IBM XIV.
Your first port of call is the System Storage Interoperation center or SSIC. However that will only list the switch modules by the IBM Feature Part Number (for example 26K5601), which may confuse you. So to help you determine which switch module you possess, here is a history of Brocade SAN switch modules for IBM BladeCenter:
2 Gbps Brocade Switch Modules for IBM BladeCenter
There were two switches 26K5601 and90P0165 but they are actually the same switch with different software features. The Enterprise version had extra licenses like Trunking and Extended Fabrics. These products were withdrawn from marketing on May 26, 2006.
4 Gbps Brocade Switch Modules for IBM BladeCenter
IBM offered two models which were physically identical. You could upgrade from the 10 port to the 20 port with a software activation key. These products were withdrawn from marketing on December 31, 2010.
IBM Product Name
Brocade 10-port SAN Switch Module for IBM eServer™ BladeCenter
Brocade 20-port SAN Switch Module for IBM eServer BladeCenter
3 external 7 internal
6 external 14 internal
IBM Feature Part Number
Brocade Product Number
Brocade Switch Type Numer
IBM FRU Number
Brocade ASIC Type
Minimum FOS Version
Maximum FOS Version
8 Gbps Brocade Switch Modules for IBM BladeCenter
There are actually three models available but I have collapsed them down to two. The 20 port switch is offered as both Enterprise and non-Enterprise. The 42C1828 switch module is the Enterprise version that has more licensed software features such as Trunking, Fabric Watch and Extended Fabrics (and is indicated with an *).
IBM Product Name
Brocade 10-port 8Gb SAN Switch Module for IBM BladeCenter
Brocade 20-port 8Gb SAN Switch Module for IBM BladeCenter
3 external 7 internal
6 external 14 internal
IBM Feature Part Number
Brocade Product Number
Brocade Switch Type Numer
IBM FRU Number
Brocade ASIC Type
Minimum FOS Version
Maximum FOS Version
At the moment the SSIC does not list every switch module. However support is available for many configurations via the IBM SCORE request system (sometimes called an RPQ). Your IBM pre-sales storage specialist can raise one of these. Depending on your request you may get a support statement as quickly as overnight.
If your wondering what an IBM FRU number is: a FRU or Field Replacement Unit is the part number used in the IBM spare parts system to replace your part under warranty or maintenance agreement.
If you want the information above in a spreadsheet format, you can find it here.
IBM has been selling IBM branded Brocade switches since 2001 when we announced the 8-port 2109-S08 and 16-port 2109-S16. These were classic switches that ran at 1 Gbps. They had a front operator panel with a small keypad (a feature which in the rush to fit in more SFPs, did not appear in future models). Since then IBM has gone on to sell many of Brocades switches and directors.
Sometimes you need to convert a Brocade model name to an IBM model name (or the other way around). One way to assure yourself with scientific accuracy which type of switch you are working on, is to telnet or SSH to a switch and issue a switchshowcommand. You will get a switchType value. In this example, my switch is a switchtype 27.2.
Or if you are using the Web GUI, you can also see the switch type on the opening screen. In this example the switch is a type 34.0.
Having scientifically determined the type of switch, we can now use my decoder ring to determine the IBM machine type, IBM model name and the Brocade model name. I have ordered the switches by Type number. There are three things to note:
Brocade have dropped the Silkworm branding, so I have dropped it too.
Each switch type has sub-types, for example 34.0 and 34.1. The difference is a sub-version number which is normally not published or documented.
IBM announced 16 Gbps SAN switches on August 16, 2011 so I updated the chart on that date.
If you use Data Center Fabric Manager (DCFM), it actually displays the Switch Type using Brocade model names. Here is an example report from the DCFM we are running in my lab. This level of information is very helpful.
If you have Brocade fibre channel switches in your SAN, you need to be aware of the method which Brocade use to manage firmware releases. All 4 and 8 Gbps Brocade SAN switches use a Linux based firmware which Brocade call Fabric Operating System or FOS. Updates to this firmware are released in families. This started with version 4, then version 5 and then version 6. Each family has had a series of updates. Version 5.0.x went to 5.1.x, 5.2.x and 5.3.x. Version 6.0.x went to 6.1.x, 6.2.x, 6.3.x and currently 6.4.x.
The good news is that you can non-disruptively update firmware on Brocade switches: So you can move to higher releases without an outage (note there may be exceptions to this, always read your release notes to be sure). However you need to be aware of a rule regarding thefromandto versions. Since FOS 6.0.0 Brocade have a one-release migration policy. This allows more reliable and robust migrations for customers. By having fewer major changes in internal databases, configurations, and subsystems, the system is able to perform the upgrade more efficiently, taking less time and ensuring a truly seamless and non-disruptive process for the fabric. The one-release migration policy also reduces the large number of upgrade/downgrade permutations that must be tested, allowing Brocade to spend more effort ensuring the supported migration paths are thoroughly and completely verified.
Disruptive upgrades are allowed, but only for a two-level migration (for instance from 6.2 to 6.4, skipping 6.3).
So why should you care?
Well your upgrade philosophy may be: If it aint broke, don't fix it. Or you may have the policy: We do fix on fail, apart from that, we don't update firmware. Much as I can understand the attraction of this, when you finally do perform an update, you may find yourself having to do many upgrades orkangaroo hopsas I call them.
Lets document some possible kangaroo hops from an old to what is currently the newest release:
As you can see from the steps above, you may have a very long change window if you are choosing to not perform updates on a regular basis. There are also lots of caveats and restrictions based on the hardware of the switch you are running. It is very important you consult the release notes that can be found at the following links:
If your looking for a recommended version to install, the Version 6 release notes page above gives advice on this. Currently it says: IBM recommends that Open System customers that currently use FOS 6.1 or earlier limit migration to FOS 6.2.2b, 6.3.0d, 6.3.1a, or 6.4.0c or later only.
Please notes that the release notes published on IBM's website are Brocade documents. While this is good (since it means they come straight from the manufacturer), you need to decode which Brocade hardware model is which IBM machine type. Theversion 6 URL above also contains a product cross-reference which lets you convert Brocade product names to IBM product names. I am also working on a post which will help you with this, so watch this space.
Edit July 19, 2011. The original post suggests 6.4.0a as a go-to level, this has since been removed. It has also been pointed out to me that some blade type switches may not be capable of hot-code-load (HCL). You need to check your vendor release notes to be certain.
I thought I would write a quick post about an issue that's not new, but is certainly worth being aware of....
One of the interesting tricks with the change to 8 Gbps Fibre Channel is that it required a change to the way the switch handles its idle time... the quiet time when no one is speaking and nothing is said. In these periods of quiet contemplation, a fibre channel switch will send idles. When the speed of the link increased from 4 Gbps to 8 Gbps, the bit pattern used in these idles proved to not always be suitable, so a different fill pattern was adopted, known as an ARB. All of this came to intrude on our lives when it became apparent that some 8 Gbps storage devices were having trouble connecting to IBM branded 8 Gbps capable Brocade switches because of this change. This led to two things:
IBM released several alerts regarding how to handle the connection of 8 Gbps capable devices to 8 Gbps capable fibre channel switches.
Brocade changed their firmware to better handle this situation.
An example of what was said?
"Starting with FOS levels v6.2.0, v6.2.0a & v6.2.0b, Brocade introduced arbff-arbff as the new default fillword setting. This caused problems with any connected 8Gb SVC ports and these levels are unsupported for use with SVC or Storwize V7000.
In 6.2.0c Brocade reintroduced idle-idle as the default fillword and the also added the ability to change the fillword setting from the default of idle-idle to arbff-arbff using the portcfgfillword command. For levels between 6.2.0c and 6.3.1 the setting for SVC and Storwize V7000 should remain at default mode 0.
From FOS v6.3.1a onwards Brocade added two new fillword modes with mode 3 being the new preferred mode which works with all 8Gb devices. This is the recommended setting for SVC and Storwize V7000"
So there are several tips that I will point you to, depending on your product of interest:
Brocade Release Notes
For most environments, Brocade recommends using Mode 3, as it provides more flexibility and compatibility with a wide range of devices. In the event that the default setting or Mode 3 does not work with a particular device, contact your switch vendor for further assistance. IBM publishes all the release notes for Brocade Fabric OS here.
The XIV Gen3 comes with 8 Gbps capable Fibre Channel connections. It does not support idle Fill Words meaning that the portCfgFillWord value should not be set to 0.
When an IBM System z server attaches an 8 Gbps capable FICON Express-8 CHPID to a Brocade switch with 8 Gbps capable SFPs, you should upgrade your switches or directors to Fabric OS (FOS) 6.4.0c or 6.4.2a and set the fill word to 3 (ARBff).
LTO5 and TS1140
IBM have two tape drives that are capable of 8 Gbps, the LTO-5 drive and the TS1140. Setting the fill word to 3 can actually cause issues with these drives. To avoid issues do one of the following (you only have to do one of these, not all three):
Load the tape drive with firmware that has the access fairness algorithm fix for loop: - LTO5 drives should be on BBN0 and beyond (you may need to contact IBM support to get this code. - TS1140 drives should be on drive firmware 5CD or beyond.
Change the Fibre Channel topology to point-to-point (N port) (as opposed to L or NL). This is my preferred option.
Change the Fibre Channel speed to 4Gbps. This sounds slightly retrograde, but it is very rare for an individual drive to sustain a speed above 400 MBps (unless your data is very very compressible).
**** UPDATED 28 Feb 2012 - Added System z FICON and Tape info ****
Its not hard to do. All you is need is a moment of inattention combined with a massive assumption. In fact assumptions can bring you undone at any time. A former manager of mine introduced me to the saying: To assume is to make an ass of you and me.
So what was the assumption this time?
One of our business partners sold a client two new XIVs and 4 new IBM SAN40Bs (40 port fibre channel switches). So far so good. When you order the SAN switches you have a choice of ordering 4 Gbps capable SFPs (SFPs are the fibre optic sub assemblies that you plug your cables into) or 8 Gbps capable SFPs. There was a time when the 8 Gbps SFPs were much more expensive than the 4 Gbps, but today they are about 75% of the price of the 4 Gbps. So it makes sense to buy the faster SFPs. But you need to ensure that all the HBAs at the client site are at least 2 Gbps capable, because 8 Gbps SFPs are tri-rate and can only go at 2, 4 or 8 Gbps. Sure enough an assumption was made that this was not an issue... but it was. The client has WDMs that run at 1 Gbps and upgrading those WDMs would be a significant expense.
So I got to thinking... could I force the SFP to 1 Gbps?
If I display the 8 Gbps SFP it reports it is capable of 200, 400, 800 MBps which is code for 2, 4 or 8 Gbps.
So what to do? We could not just move the old SFPs into the new switch, as the new 8 Gbps capable Brocade switches only accept Brocade approved SFPs. The only solution was to make it right and swap four of the Brocade 8 Gbps SFPs with Brocade 4 Gbps SFPs. Fortunately as we needed only four, I was able to swap them with little expense or hassle (I contacted our local Brocade rep who happily helped us out).
The end point was a happy client and a lesson re-learnt..... 1 into 8 does not go.
I am curious though... is there much 1 Gbps gear still out there? Is this a common issue?
So something I truly hate to see when visiting computer rooms is fibre cables hanging in the breeze with no dust covers, their precious glass connectors exposed to the world. Even worse are fibre patch panels and HBAs without dust covers. . When new equipment arrives, every HBA, every patch panel port and every fibre optic cable will have a dust cover. So what to do with these little guys once you remove them? Keep them! When you unplug a cable later you need to immediately re-install those precious covers both onto the cable and into the HBA or patch panel port, to protect the fibre optics from contamination. I recommend storing dust covers in sealed plastic bags, preferably kept in the relevant rack so they are close to hand. The picture below (taken from the rear of an XIV) is cute in that it shows a clever re-use of the XIV power cable covers, but the dust covers are now exposed to contamination from the open air. . Since we are on the subject, take note of the colour coded power cables in the XIV. Its another example of clever design to make power redundancy visually obvious. The ends of the power cords are red, yellow and green to indicate which of the three UPSs these cables come from. The unused cords at the top of the image are free because this machine is not fully populated with modules.
Its a given today that SAN implementations have redundancy by design. When we sell SAN switches we always urge clients to use dual fabrics. Its accepted industry best practice. By having hosts with dual HBAs, each host can attach to both fabrics and survive the failure of a SAN switch. . Equally when we sell SAN Volume Controller (SVC) we always sell the SVC nodes in pairs. Each SVC node can run an SVC I/O group stand-alone, which again allows us to survive a node failure and do things like concurrent firmware updates. . So far so good. But common practice appears to be that when installing redundant hardware, that we place all the hardware into one rack. In fact I routinely look in a rack and see Switch One directly mounted above Switch Two. I see the same with SVC clusters, all the nodes often jammed into the same rack, mounted one on top of the other. . Is this a good idea? I suspect 99.9% of the time it makes no major difference. But its the 0.1% that can cause great pain. Imagine going to work on a failed switch and then accidentally powering down the working one. If each switch was in a separate rack the likelihood of doing this is significantly reduced. I recently visited a major Australian Bank where each of their Fibre Channel directors were located on opposite ends of a long row of racks, several meters apart. The separation of distance guaranteed that any event in one rack could not possibly affect the other rack. Good design... by design..... I liked it. There is no reason you cannot install SVC nodes in the same way, each node in a separate rack. Just don't separate them too far apart. When servicing a node you like to be able to see what the front panel of the other node is displaying. Having said that, SVC split cluster (where we separate the nodes across or between buildings) truly separates the nodes for the highest levels of availability.
As a follow up to my previous blog, there are also distance considerations for longwave 8 Gpbs SFPs.
To provide some background, there are three things that control our ability to reliably send data over longer distances than 300 meters.
The intensity (brightness) of the transmitter (for long distances this is normally a laser as opposed to an LED). The transmit value is measured in dBm. There is a concept called a link budget that is arrived at by adding the number of joins and the length of the fibre to determine if the Tx (transmit) value will fall below the minimum Rx (receive) value of the SFP at the receiving side. If it does, we will have an optical quality issue. If its too bright on the other hand, we will need a device called an optical attenuator to dim the light. There is a good Wikipedia article here: Link Budget
The wavelength of the laser. Traditional longwave is 1310nm (that value effectively being the 'colour' of the light). For real longhaul (like WDM and CWDM SFPs) we use SFPs in the 1500-1550nm range. The main point is that the SFPs at each end of a link need to use the same wavelength, or they won't be able to communicate with each other. Wikipedia again: Optical variants
The number of BB (buffer to buffer) credits available to the link, especially if this is a cross site Inter Switch Link (ISL). This is not a big issue for director class switches, but can be a major gotcha if your using small/midrange switches. If you have a Brocade switches and dont have the Extended Fabrics license, you could be in trouble if your link is more than 10km long. The bad news is that this license is not free. The good news is that Brocade can provide an evaluation license so you can test to see if purchasing the license will really help. Cisco doesn't need an equivalent license, but the 4 Gbps capable MDS9124 and MDS9134 are limited in the maximum number of buffers that can be dedicated to one port (there are only 64 buffers per group of 4 ports). At 2 Gbps you need to have at least 1 buffer per kilometer of fibre. At 4 Gbps you need 2 buffers per km and at 8 Gbps you need 4 buffers per km. The 8 GBps capable Cisco MDS9148 has 128 buffers per port group meaning one port at 8 Gbps could utilize 32 km of fibre. This great article is about FICON but its an excellent read: BB Credits and FICON
Standard Longwave versus Extended Longwave
When you see a 10km SFP versus a 25km longwave SFP , the main difference between them is that one uses a more intense (brighter) transmit signal than the other.
Without intending to indicate a vendor preference, I will use the specs of Brocades SFPs to make a point. You can visit the specifications page here:
The 10km SFPs transmits between -8.4 and 0.5 dBm and needs to receive a signal of at least -15.4dBm. The 25km SFPs transmits between 0 to 5.0 dBm and needs to receive a signal of at least -13.8dBm. The higher transmit value gives you a bigger link budget, which is why the SFP can go 15 extra km.
The great thing is that both Cisco and Brocade switches use SFPs that report diagnostic information that can be displayed using the relevant management GUI. In the screen capture below you can see the output from a shortwave length (850 nm) 4 Gbps capable SFP installed in an IBM R18 (Brocade Silkworm 7500) switch. Because this is a very short connection, the Rx level is very close to the Tx level.
In the screen capture below you can see the output from a CWDM long haul (1530 nm) 4 Gbps capable SFP installed in a Cisco MDS9509. This screen capture is more interesting because it shows that the Rx level is too low (which is why I got the screen capture in the first place - the link wasn't working!) So while its a good example of failed link, it doesn't give an example of seeing how close to reality your link budget calculations were.
Its now becoming far more common to see 8 Gbps Fibre Channel switches and HBAs. IBM offer a very comprehensive range of 8 Gbps capable switches manufactured by both Brocade and Cisco. These range from top of the line SAN Directors like the IBM SAN768B (Brocade DCX) and the Cisco MDS9513 to 1U switches like the MDS9148 and the IBM SAN40B-4 (Brocade Silkworm 5100).
The transition to 8 Gbps switches is not without some potential 'gotchas' which its worth being aware of.
1) The tyranny of distance
Its important to realise that as optical speeds increase, the effective distance that we can safely transmit these signals decreases. If your doing a drop and replace upgrade where existing fibres are moved from an older 1, 2 or 4 Gbps switch to a new 8 Gbps capable switch, if the link is now able to run at 8 Gbps, perhaps you may have unintentionally exceeded recommended cable lengths. These numbers for shortwave (850 nm) SFPs are sourced from an IBM Announcement letter, but I have seen some subtle differences between vendors, I suspect based on which SFPs are being used. The lesson to learn here is that as the speed of the link doubles, the recommended length of the cable decreases. If your 50 micron cables are 300 meters long, they will work well at 1 and 2 Gbps, but may well prove unreliable at 8 Gbps.
1 Gbps 50 micron 500 meters (1640 feet)
1 Gbps 62.5 micron 300 meters (984 feet)
2 Gbps 50 micron 300 meters (984 feet)
2 Gbps 62.5 micron 175 meters (574 feet)
4 Gbps 50 micron 150 meters (492 feet)
4 Gbps 62.5 micron 70 meters (230 feet)
8 Gbps 50 micron 50 meters (164 feet)
8 Gbps 62.5 micron 21 meters (69 feet)
In the case of longwave SFPs (that operate at 1310 nm) I will write a second article because there are complicating factors due to the increasing use of extended distance SFPs.
2) Fillwords at 8 Gbps
I have received quite a few reports of clients having issues connecting DS5000s to Brocade switches running at 8 Gbps.
You can find this tip explaining how to diagnose and correct this issue on the IBM Support Site.