Comments (2)
  • Add a Comment
  • Edit
  • More Actions v
  • Quarantine this Entry

1 ExtensionArchitect commented Permalink

Seb, <div>&nbsp;</div> My name is Mark Detrick and I’m a Principal Engineer in Brocade’s Solutions Engineering and specialize in Extension. I have a few comments in reply to your post about the Brocade 7800 (IBM SAN06B-R)… <div>&nbsp;</div> There is an 8G FC switching ASIC named Goldeneye2 (GE2), which was developed by Brocade and is used in various products. This is the FC switch part of the 7800. The other ASIC is a Broadcom Ethernet switch, which provides the Ethernet connectivity for the 6 interfaces. There is a Cavium network processor, and lastly there is an FPGA used to connect it all together and perform HW based high-speed and high-rate functions like compression and IPsec. The FPGA sits in the middle of everything and can be firmware reprogrammed if functionality needs to be enhanced. I believe you are referring to the FC communications between the GE2 and the FPGA. <div>&nbsp;</div> The communications between the GE2 and the FPGA (named “Blaster”) has 5x 4 Gbps connections. 4 Gbps connections were used because at the time the 7800 was designed 4 Gbps was the fastest PHY that Xilinx had for their FPGAs. There is an aggregate of 20 Gbps available to supply data to the 6 Gbps of FCIP WAN interfaces. You mentioned most of this. If you were to get 2:1 data compression, which is typical for the “standard” mode of compression, you would only need 12 Gbps coming from the GE2. If you got 3:1 compression it would still work. <div>&nbsp;</div> There is no bottleneck within the 7800, even with only 4 BBC. Extension Engineering disagrees with your claim and there is no risk of HoLB between the GE2 and the Blaster. The assumption that you need more BBC because “there are multiple reasons why frames need to be touched” is not correct in this specific case. The BBCs used between the GE2 and the FPGA are merely used to offload the data from the GE2 into the data processor (DP) for FCIP processing and 4 BBC is more than adequate for that purpose. The Brocade 7800/FX has a large number of resources available for this offloading of data. Only when resources are totally consumed will Storage-Optimized TCP (SO-TCP) apply back pressure by closing the rwnd (receive window) causing BBCs to be withheld to this offloading process. Yes, a slow drain device on the remote side can cause data to back up all the way through to the local side. This is the nature of FC flow control within a fabric and is considered normal operation, nonetheless, there is no HoLB going on within the 7800 itself. <div>&nbsp;</div> SO-TCP consists of 4 TCP sessions for each circuit, one for each of: Class-F, high, med &amp; low. The default is medium. There may be multiple flows placed onto one of those TCP sessions from the same priority. Each of these TCP sessions run independently and if there is a problem with a slow drain flow, it is best practice to place that type of device or that particular device onto a different QoS priority, hence, a different TCP session. Since the TCP sessions are autonomous they will not interfere with each other. By using a different QoS you gain separate TCP sessions for these various flows while staying with the same VF LS and using the same VE_Port which is important in some environments. <div>&nbsp;</div> There is another method that can be used, Virtual Fabrics (VF) Logical Switches (LS). Segregated devices will not be able to interfere with the performance of other devices. By using a different VE_Port in a different VF LS you gain an entirely different set of TCP sessions, each with their own queues. On Brocade Extension devices, using different VF LS does not mean you have to use a different Ethernet interface. Ethernet interfaces can be shared across different VF LS. For example, two 500 Mbps circuits could share a single 1 Gbps interface. The Ethernet interfaces are abstracted. <div>&nbsp;</div> FCIP Trunking uses a single VE_Port permitting protocol optimization like FastWrite, OSTP and FICON Acceleration to operate while utilizing multiple circuits over disparate paths. No need for Traffic Isolation Zones. Each circuit aggregates more bandwidth, provides for redundant and resilient paths, has Lossless Link Loss (LLL) and always delivers data in-order which is perfect for mainframe and Global Mirror applications. <div>&nbsp;</div> I guess the proof is in the pudding, as the saying goes... I have not heard anything from the thousands of Brocade customers concerning their 7800s or FX blades pertaining to poor performance. This includes customers with multiple flows across their FCIP connection. I must point out however that it is key to construct the right architecture to achieve optimal operation. The 7800 provides all the features and functionality to accomplish this mission. <div>&nbsp;</div> P.S. Why does IBM continue to call Extension devices “routers”? FCR is built into all Brocade’s FC switching ASICs now and there is no designation as “router” anymore. You turn on the FCR feature using the IR license on any Condor2/GE2 or later ASIC switch/Director. The FCIP products are properly referred to as “Extension”, even the “SAN” part of SAN Extension has been dropped in modern times. I don’t want to pick on IBM because we love IBM, but IBM is the only OEM that remains still doing this. <div>&nbsp;</div> Best Regards, <br /> Mark Detrick <br /> BCFP, BCNE, BCAF, CISSP, CCIE <br /> Principal Engineer

2 seb_ commented Permalink

Mark, thank you for your very helpful feedback and please excuse the bad formatting. I did not find out so far how to get HTML tags working inside comments again. Your explanation contains some real eye-openers and I have to think about them, as they might work very well for most of our customers. I think we meanwhile found a good compromise for the environment I had in mind while writing this blog post. Reg. "routers vs Extension devices": It's interesting to hear that nobody else calls them routers today. Usually everybody I know from post-sales and defect support still calls them so in the "tradition" of the old multiprotocol routers. For many of our clients I even see the word "router" in the switchnames. Maybe "router" just trips off the tongue much easier than "extension device". :o) Cheers seb