Prerequisites

You should have familiarity with:

• Cisco Unified Computing Systems (UCS) VIC
• Cisco UCS B-Series and C-Series servers
• Cisco UCS I/O Modules (IOMs) and Fabric Interconnects (FIs)
• Cisco UCS Manager (UCSM) / Intersight Management Mode (IMM)
• Basic concepts of cut-through vs. store-and-forward switching
• CRC "stomping" behavior

Components Used

• Cisco UCS Manager version 2.x and later
• Cisco UCS Fabric Interconnects: 6200, 6300, 6400, 6500 Series
• Cisco UCS I/O Modules: 2200, 2300, 2400 Series
• Cisco UCS VIC Adapters: 1200, 1300, 1400, 1500 Series

All devices in the referenced lab started from default configurations. If troubleshooting a live environment, be mindful that changes can impact production traffic.

Background Information

• Cut-through Switches (e.g., UCS Fabric Interconnects, Cisco Nexus 5000): Forward frames before the entire frame is read. They do not verify the CRC prior to forwarding, so corrupted frames can be passed along.
• Store-and-forward Switches (e.g., Cisco Nexus 7000): Validate the entire frame, including the CRC, before forwarding. They drop frames that fail CRC checks.

Because of cut-through behavior, an interface seeing CRC errors is not always the actual source of those errors. You often need to trace CRC errors back to the original source.

Reasons for Bad Frames and CRC Errors

• Physical issues: Bad transceiver, cable (copper/fiber), adapter, or port.
• MTU violations: Frames larger than the configured MTU can become corrupted or stomped.
• CRC stomped frames: A neighboring cut-through switch might have stomped the CRC on outgoing frames it already began forwarding.

Forwarding Mode Behavior (Cut-through or Store-and-Forward)

UCS Fabric Interconnects can operate in cut-through or store-and-forward mode. The forwarding mode depends on the ingress and egress data rates, as shown in the table below. Generally, cut-through happens only if the ingress rate is equal to or faster than the egress rate.

Main Forwarding ASICs: Commands for UCS FIs, IOMs, and VIC Cards

Certain commands let you view internal counters, identify drop reasons, and see whether frames are being stomped or dropped.

Commands for UCS Fabric Interconnects and IOMs

Cisco UCS 6200 Series (Carmel ASIC)

show hardware internal carmel …
show platform fwm info asic-errors X
show platform fwm info pif e1/X | grep asic
show platform fwm info pif e1/X | grep drop

Cisco UCS 6300 Series (Trident2 ASIC)

show hardware internal bcm-usd info port-info
show hard internal interface indiscard-stats front-port X

Cisco UCS 6324 (Mini FI, Monticello ASIC)

show hardware internal mtc-usd port-status

Cisco UCS 6400 / 6500 Gen5

show hardware internal interface asic counters module 1
attach module 1; show hardware internal tah drop-reason counters module 0

Cisco UCS 2200 / 2300 / 2400 Series IOMs

show platform software woodside sts
show platform software tiburon sts
show platform software woodside|tiburon rmon 0 [NIx/HIx]
show platform software woodside|tiburon drops 0 [NIx/HIx]

Commands for UCS VIC Adapters (1200, 1300, 1400, 1500)

attach-mcp
macstats <port>
uifportstatus
pcie_links

Examples:

• Gen 2 VIC (e.g., 1225, 1240): adapter (mcp):3# dcem-macstats 0
• Gen 4 VIC (e.g., 1440, 1480): adapter (macd):1# macstats 0
• Gen 5 VIC (e.g., 15420, 15425): adapter (mcp):1# uifportstatus

In Summary:

• Check Physical Health First: Many CRC or drop issues can stem from faulty cables, SFPs, or adapters.
• Identify Cut-through vs. Store-and-Forward: Cut-through devices can pass corrupted frames onward, so the actual error source might be upstream.
• Use ASIC-Specific Commands: Each generation of FI or IOM has specialized commands to reveal drops and error counters.
• VIC Adapter Debugging: Connect to the adapter and use attach-mcp or attach-macd to see real-time counters (like macstats).
• Correlate Counters: If one side shows RX_CRC_STOMPED, look at the connected device’s counters to find the actual cause (MTU mismatch, cable issue, etc.).
• Refresh Counters Repeatedly: Checking “Diff” columns or re-running commands can show if counters are actively increasing.