White Papers
Abstract
This documents explains tim_txcrd_z counters in Broadcom switches.
Just because this counter has high value doesn't mean there is an problem.
Content
Time at zero credits (tim_txcrd_z) is generally considered to be the number of times that a
port was unable to transmit frames it had in queue because the transmit BB_Credit
counter was zero. But you must realize that the zero credit counter is incremented
when the last buffer credit is allocated. A buffer credit is allocated just prior to frame
transmission, so, technically, the switch continues to transmit the last frame while at zero
buffer credits. In long-haul links this is insignificant, but in local data centers over short
pieces of cable you need to look at link performance as well to understand whether a
buffer credit problem exists or not. Theoretically, a short cable link can be at zero buffer
credits 100 percent of the time, and the link will continue transmitting 100 percent of the
time.
counter was zero. But you must realize that the zero credit counter is incremented
when the last buffer credit is allocated. A buffer credit is allocated just prior to frame
transmission, so, technically, the switch continues to transmit the last frame while at zero
buffer credits. In long-haul links this is insignificant, but in local data centers over short
pieces of cable you need to look at link performance as well to understand whether a
buffer credit problem exists or not. Theoretically, a short cable link can be at zero buffer
credits 100 percent of the time, and the link will continue transmitting 100 percent of the
time.
The purpose of this statistic is to detect congestion or a device affected by latency. This
parameter is sampled on every port at intervals of 2.5 microseconds (μsec) (400,000
times per second), and the tim_txcrd_z counter is incremented if the condition is true.
Each sample represents 2.5 μsec of time with zero transmitter BB_Credits and a frame
waiting to be transmitted. An increment of this counter means that the frames could
not be sent to the attached device for 2.5 μsec, indicating the potential of some level of
degraded performance.
parameter is sampled on every port at intervals of 2.5 microseconds (μsec) (400,000
times per second), and the tim_txcrd_z counter is incremented if the condition is true.
Each sample represents 2.5 μsec of time with zero transmitter BB_Credits and a frame
waiting to be transmitted. An increment of this counter means that the frames could
not be sent to the attached device for 2.5 μsec, indicating the potential of some level of
degraded performance.
In a high-performance environment, it is normal for buffer credits to reach zero at
times, and buffer credits reaching zero does not mean that there is a congestion problem.
Optimization does not mean that the port can never run out of buffer credits, but rather
that the port will not run out of buffer credits 99.999 percent of the time. That .0001
percent when it does run out of buffer credits will not harm performance or utilization on
the link. Also, in mixed speed environments, where the switch is attached to 4 Gbps,
times, and buffer credits reaching zero does not mean that there is a congestion problem.
Optimization does not mean that the port can never run out of buffer credits, but rather
that the port will not run out of buffer credits 99.999 percent of the time. That .0001
percent when it does run out of buffer credits will not harm performance or utilization on
the link. Also, in mixed speed environments, where the switch is attached to 4 Gbps,
8 Gbps, 16 and 32 Gbps devices, this counter typically increments at a higher rate on
the lower speed devices than on the higher speed devices.
the lower speed devices than on the higher speed devices.
However, if the buffer credits reaching zero is a high value in regard to total frames
sent, it could indicate a congestion problem. The absolute value is there in the counter
(since the last POR or statsClear CLI command), but the relationship between number
of buffer credits reaching zero and the total number of transmitted frames should be of
interest. It is recommended that if the tim_txcrd_z counter shows a value around
10 percent or more of the total frames sent, you should take a real interest in it. A
percentage much higher than 10 percent might indicate a fabric congestion problem.
Use the following to calculate percentage of BB_credit zero .
portstats64show <port> — This gives the amount of frames send (stat64_ftx) and
amount of BB_credit zero occurrences (tim64_txcrd_z).
amount of BB_credit zero occurrences (tim64_txcrd_z).
Example:
portstats64show 40
stat64_ftx 115 top_int : Frames transmitted
161688329 bottom_int : Frames transmitted
tim64_txcrd_z 1 top_int : Time BB_credit zero
240621272 bottom_int : Time BB_credit zero
161688329 bottom_int : Frames transmitted
tim64_txcrd_z 1 top_int : Time BB_credit zero
240621272 bottom_int : Time BB_credit zero
The total amount of frames sent is (2³²)*<top_int>+<bottom_int>.
The total amount of BB_credit zero ( is (2³²)*<top_int>+<bottom_int>.
The percentage of BB_credit zero is (<BB_credit zero>/<frames sent>)*100.
According to Brocade some BB_credit zero is allowed, but no more than 10%
The total amount of BB_credit zero ( is (2³²)*<top_int>+<bottom_int>.
The percentage of BB_credit zero is (<BB_credit zero>/<frames sent>)*100.
According to Brocade some BB_credit zero is allowed, but no more than 10%
using values from example above
Total Frames transmitted = 115(2^32) + 161688329 = 115 * 4294967296 + 161688329 = 493921481569
Total Time BB_credit zero = 1(2^32) + 240621272 = 4294967296 + 240621272 = 4535588568
Total Frames transmitted = 115(2^32) + 161688329 = 115 * 4294967296 + 161688329 = 493921481569
Total Time BB_credit zero = 1(2^32) + 240621272 = 4294967296 + 240621272 = 4535588568
So percentage of BB_credit zero is ~ 4535588568 / 493921481569 ≈ 0.009174 ~ 0.91%
So considering this though tim_txcrd_z is high (161688329) it should be of no concern as its well below 10%.
[{"Type":"MASTER","Line of Business":{"code":"LOB26","label":"Storage"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"STQPPK","label":"IBM Storage Networking SAN512B"},"ARM Category":[{"code":"a8m0z000000bqLAAAY","label":"Brocade IBM B-Type->Reference"}],"Platform":[{"code":"PF025","label":"Platform Independent"}]}]
Was this topic helpful?
Document Information
Modified date:
22 May 2023
UID
ibm16983837