Small tips for analyse the fabric stats. I mean packets stats (IN/OUT/Drops) between each PFE and the Fabric.
Usually fabric stats can be observe by using the following cli command. Remember, that there are only 2 streams at fabric level: High and Low (aka. Priority Drop).
sponge@bob>show class-of-service fabric statistics summary
Destination FPC Index: 0, Source FPC Index: Summarized from all source FPC's
Total statistics: High priority Low priority
Packets: 448626385 1365903575
Bytes : 598687578257 1861601724122
Pps : 40 3996
Bps : 20912 5470976
Tx statistics: High priority Low priority
Packets: 448626385 1365903575
Bytes : 598687578257 1861601724122
Pps : 40 3996
Bps : 20912 5470976
Drop statistics: High priority Low priority
Packets: 0 0
Bytes : 0 0
Pps : 0 0
Bps : 0 0
The above command gives the total statistics from all FPC to a given Destination FPC (here the FPC 0). To collect the stats from one given FPC to another one use:
sponge@bob> show class-of-service fabric statistics destination 0 source 3
Destination FPC Index: 0, Source FPC Index: 3
Total statistics: High priority Low priority
Packets: 42445974 439161456
Bytes : 71944097695 604147953760
Pps : 0 3995
Bps : 0 5470681
Tx statistics: High priority Low priority
Packets: 42445974 439161456
Bytes : 71944097695 604147953760
Pps : 0 3995
Bps : 0 5470681
Drop statistics: High priority Low priority
Packets: 0 0
Bytes : 0 0
Pps : 0 0
Bps : 0 0
sponge@bob> show chassis fabric statistics rates detail 2 | match "DPC3PFE|put"
Input Link State vld/sec req/sec gnt/sec drops/sec
DPC3PFE0->CB1F0_03_0 up 0000000000 0000000000 0000000000 0000000000
DPC3PFE1->CB1F0_03_1 up 0000007601 0000007601 0000000000 0000000000
DPC3PFE2->CB1F0_03_2 up 0000000000 0000000000 0000000000 0000000000
DPC3PFE3->CB1F0_03_3 up 0000014374 0000014373 0000000000 0000000000
Output Link State vld/sec req/sec gnt/sec drops/sec
CB1F0_03_0->DPC3PFE0 up 0000000000 0000000000 0000000000 0000000000
CB1F0_03_1->DPC3PFE1 up 0000000000 0000000000 0000007600 0000000000
CB1F0_03_2->DPC3PFE2 up 0000000000 0000000000 0000000000 0000000000
CB1F0_03_3->DPC3PFE3 up 0000000000 0000000000 0000014373 0000000000
The stats are in cells/sec (JCELL is, if i'm remember, a 64 bytes size container).
For the traffic DPCxPFEx --> CBxFx (F for SF chip hosted by the sCB) :
- vld/sec : valid cells per second sent by the PFE.
- req/sec : valid cells sent to the fabric
- gnt/sec : valid cells received from the fabric.
- drops/se : cells drops (in or out)
- vld/sec : valid cells per second sent by the SF chip.
- req/sec : valid cells sent to the PFE
- gnt/sec : valid cells received from the PFE
- drops/se : cells drops (in or out)
NB.: The traffic from and to a PFE should be equally distributed over each fabric plane or sib.
For T series the output is a little bit different:
star@patrick> show chassis fabric stats rates detail 3 | match "FPC2_|put"
Input Link State vld/sec req/sec gnt/sec drops/sec
FPC2_T->SIB3_F0,04 UP 0000020570 0000020573 0000001413 0000000000
FPC2_B->SIB3_F0,05 UP 0000000000 0000000000 0000000002 0000000000
Output Link State vld/sec req/sec gnt/sec drp/sec
SIB3_F0,02->FPC2_B UP 0000000000 0000000002 0000000000 0000000000
SIB3_F0,03->FPC2_T UP 0000001411 0000001413 0000020588 0000000000
NB. :
The _T means PFE 0 of the given FPC
The _B means FPE 1 of the given FPC
All of those commands are very useful when troubleshooting packet drops issue, gongestion problem, HW issue or during stress tests of new HW or SW. I'm currently written an op script, for MX, that should aggregate in only one command all the stats from all fabric planes but it's not so easy because it's an hidden command that doesn't have an XML output...
David.
