Anthony's Blog: Using System Storage - An Aussie Storage Blog
So something I truly hate to see when visiting computer rooms is fibre cables hanging in the breeze with no dust covers, their precious glass connectors exposed to the world.
Even worse are fibre patch panels and HBAs without dust covers.
When new equipment arrives, every HBA, every patch panel port and every fibre optic cable will have a dust cover.
So what to do with these little guys once you remove them?
When you unplug a cable later you need to immediately re-install those precious covers both onto the cable and into the HBA or patch panel port, to protect the fibre optics from contamination.
I recommend storing dust covers in sealed plastic bags, preferably kept in the relevant rack so they are close to hand.
The picture below (taken from the rear of an XIV) is cute in that it shows a clever re-use of the XIV power cable covers,
but the dust covers are now exposed to contamination from the open air.
Since we are on the subject, take note of the colour coded power cables in the XIV.
Its another example of clever design to make power redundancy visually obvious.
The ends of the power cords are red, yellow and green to indicate which of the three UPSs these cables come from.
The unused cords at the top of the image are free because this machine is not fully populated with modules.
anthonyv 2000004B9K Tags:  cisco systems fibre center nxos. system ficon z ibm data channel switch 13,136 Views
Right now I am working on giving a client a recommended version of firmware for their Cisco MDS Fibre Channel switches. For FICON, the recommendations are easy, but for Open Systems there are so many choices. So what am I going to recommend?
FICON Switches and Directors
For FICON switches, sticking to the FICON (IBM Mainframe Fibre Connection) recommended versions (which are determined by the IBM System z Mainframe team), is a very good strategy. The best place to get these is here (standard IBM logon is required). Just look along the right hand column for the release letters.
The SAN-OS and NX-OS release notes found on the Cisco website also show recommended versions for FICON. For instance have at the look at the FICON recommendations table in the releases notes for version 5.2.2a that you can find here. The upgrade path is just below the table I have linked to. This link will get outdated over time (as newer versions come out), but you can list all the release notes here.
If you are using a IBM TS7700 you should also be aware of this page on the IBM Techdocs site.
So based on current versions, if you are running SAN-OS 3.3.1c or below you need to move to 4.2.7b (as per the non-disruptive upgrade path). I strongly recommend you get to at least version 4.2.7b and start planning to move to release 5.2.2 (provided your hardware supports it).
For open systems attached Fibre Channel switches there are a number of versions to choose from. There are five things to consider:
So what this mean is that for open systems as at April 2012, I recommend you install 3.3.5b for Gen1 hardware, 4.2.7e for Gen2 and Gen3 and 5.2.2a for Gen4 hardware.
For more details on when things are going end of life, check the following websites:
End-of-Sale and End-of-Life for the Cisco MDS 9000 SAN-OS Software Release 3.x
Finally it is well worth bookmarking the following links to help you with any updates (the middle links needs a Cisco CCO login):
Cisco Release Notes
And thanks to Glen Routley and Filiph Westman for proof reading this post.
anthonyv 2000004B9K Tags:  cisco nxos vendors ficon z australia nx-os system systems ibm 10,352 Views
I got a great question recently:
We just updated our Cisco MDS9509s to NX-OS 4.2.7b (from Cisco SAN-OS 3.3.1c) and now we are getting emails from this source: GOLD-major.
The actual message looks like this:
Time of Event:2012-03-05 15:07:21 GMT+00:00 Message Name:GOLD-major Message Type:diagnostic System Namexxxx Contact Namexxx@xxx.com Contact Emailxx@xxx.com Contact Phone:+61-3-xxxx-xxxx Street Addressx Road, xxxx, VIC, Australia Event Description:RMON_ALERT WARNING(4) Falling:iso.220.127.116.11.18.104.22.168.1.10.18366464=2401032512 <= 4680000000:135, 4 Event Owner:ifHCOutOctets.fc4/5@w5c260a03c162 ThresholdType:FallingThreshold
So who is GOLD-major?
GOLD actually stands for Generic OnLine Diagnostics. From Cisco's website:
So in our example GOLD is actually reporting a major event (to do with exceeded thresholds, in this example utilisation on interface fc4/5).
Most clients using Cisco MDS switches are now moving to NX-OS (over SAN-OS, the name Cisco used for MDS firmware between version 1 and version 3) so this question will become more common. I am working on a post that discusses recommended versions (and the sunsetting of SAN-OS), so expect something soon. If on the other hand you are thinking.... how do I setup call home on a Cisco MDS switch? The information for NX-OS is here.
anthonyv 2000004B9K Tags:  brocade of 497 cisco the storage ibm and beast. 32-bit day 994 tagged number days 10,902 Views
Last year I blogged about 497 being the IT number of the beast.
Because if a product uses a 32 bit counter to record uptime, and that counter records a tick every 10 msec, then that 32-bit counter will overflow after approximately 497.1 days. This is because a 32 bit counter equates to 2^32, which equals 4,294,967,296 ticks. If a tick is counted every 10 msec, we create 8,640,000 ticks per day (100*60*60*24). So after 497.102696 days, the counter will overflow. What happens next depends on good programming: normally the counter just starts again, but worst case a function might stop working or the product might even reboot.
Fortunately we are seeing less and less of these issues but just occasionally one still slips out. Recently IBM released details of a 994 day reboot bug in the ESM code of some of their older disk enclosures (EXP100, EXP700 and EXP710). Details about this bug can be found here. What I find interesting is the number of days it takes to occur, since 994 is actually 497 times two. This suggests that this product records a tick every 20 msec. This meant we got past 497 days without an issue but hit a problem after exactly double that number. So if you still have these older storage enclosures, you will need to reboot the ESMs (after checking the alert).
I googled 497 to see what images that number brings up and was amazed to find the M-497 jet powered train. More details on this rather interesting attempt at speeding up the commute home can be found here and here. It adds a whole new meaning to keeping behind the yellow line.
As a follow up to my previous blog, there are also distance considerations for longwave 8 Gpbs SFPs.
To provide some background, there are three things that control our ability to reliably send data over longer distances than 300 meters.
Standard Longwave versus Extended Longwave
When you see a 10km SFP versus a 25km longwave SFP , the main difference between them is that one uses a more intense (brighter) transmit signal than the other.
Without intending to indicate a vendor preference, I will use the specs of Brocades SFPs to make a point. You can visit the specifications page here:
Compare this 8 Gbps capable 10km range SFP:
With this 8 Gbps capable 25km range SFP:
The 10km SFPs transmits between -8.4 and 0.5 dBm and needs to receive a signal of at least -15.4dBm.
The 25km SFPs transmits between 0 to 5.0 dBm and needs to receive a signal of at least -13.8dBm. The higher transmit value gives you a bigger link budget, which is why the SFP can go 15 extra km.
Remember you can increase these distance with WDM or CWDM technology, but you need to ensure the available BB credits on the ISL will be able to fully utilize it.
For Cisco switches check out the specs here:
Displaying actual transmit and receive values
The great thing is that both Cisco and Brocade switches use SFPs that report diagnostic information that can be displayed using the relevant management GUI.
In the screen capture below you can see the output from a shortwave length (850 nm) 4 Gbps capable SFP installed in an IBM R18 (Brocade Silkworm 7500) switch. Because this is a very short connection, the Rx level is very close to the Tx level.
In the screen capture below you can see the output from a CWDM long haul (1530 nm) 4 Gbps capable SFP installed in a Cisco MDS9509. This screen capture is more interesting because it shows that the Rx level is too low (which is why I got the screen capture in the first place - the link wasn't working!) So while its a good example of failed link, it doesn't give an example of seeing how close to reality your link budget calculations were.
anthonyv 2000004B9K Tags:  ds5100 brocade ds5020 ds3950 arb 8gbps fillwords ds5300 cisco 10,910 Views
Its now becoming far more common to see 8 Gbps Fibre Channel switches and HBAs. IBM offer a very comprehensive range of 8 Gbps capable switches manufactured by both Brocade and Cisco. These range from top of the line SAN Directors like the IBM SAN768B (Brocade DCX) and the Cisco MDS9513 to 1U switches like the MDS9148 and the IBM SAN40B-4 (Brocade Silkworm 5100).
You can check them all out here: http://www-03.ibm.com/systems/storage/san/index.html
Of course we should not forget 8 Gbps capable IBM BladeCenter modules from Brocade and Qlogic, which you can check out here:
The transition to 8 Gbps switches is not without some potential 'gotchas' which its worth being aware of.
1) The tyranny of distance
Its important to realise that as optical speeds increase, the effective distance that we can safely transmit these signals decreases. If your doing a drop and replace upgrade where existing fibres are moved from an older 1, 2 or 4 Gbps switch to a new 8 Gbps capable switch, if the link is now able to run at 8 Gbps, perhaps you may have unintentionally exceeded recommended cable lengths. These numbers for shortwave (850 nm) SFPs are sourced from an IBM Announcement letter, but I have seen some subtle differences between vendors, I suspect based on which SFPs are being used. The lesson to learn here is that as the speed of the link doubles, the recommended length of the cable decreases. If your 50 micron cables are 300 meters long, they will work well at 1 and 2 Gbps, but may well prove unreliable at 8 Gbps.
In the case of longwave SFPs (that operate at 1310 nm) I will write a second article because there are complicating factors due to the increasing use of extended distance SFPs.
2) Fillwords at 8 Gbps
I have received quite a few reports of clients having issues connecting DS5000s to Brocade switches running at 8 Gbps.
You can find this tip explaining how to diagnose and correct this issue on the IBM Support Site.
To check out details of why this happened, visit the release notes for Brocade FOS 6.2.0e firmware found here:
You should also have a read of these release notes for v6.3.0c that can be found at the same URL above.
These suggest that for the following devices at 8 Gbps, the fill words should be set to IDLE: