Broadcom Tomahawk 101
Juniper recently launched their Tomahawk-based switch (QFX5200) and included a lot of information on the switching hardware in one of their public presentations (similar to what Cisco did with Nexus 9300), so I got a non-NDA glimpse into the latest Broadcom chipset.
You’ll get more information on QFX5200 as well as other Tomahawk-based switches in the Data Center Fabrics Update webinar in spring 2016.
Here’s what I understood the presentation said:
- Each 100 GE port can be channelized into 4 x 10GE, 4 x 25GE, 2 x 50GE or 1 x 40GE. It seems like each port can run 4 lanes at either 10 Gbps or 25 Gbps;
I may be totally wrong, but the way I understand the specs the 100GE ports use 100GBASE-SR4 (802.3bm) standard and would thus be incompatible with switches using older 100GBASE-SR10 (802.3ba) standard, although they would work with all 40 Gbps switches using 40GBASE-SR4.
- Similar to Trident-2, Tomahawk becomes line-rate (3.2 Tbps) at packet sizes above 250 bytes;
- Presentation claims overlay routing (VXLAN-to-VXLAN or VXLAN-to-VLAN) is not supported, which is a bit surprising as the forwarding pipeline includes tunnel termination before L2 and L3 lookup, which should be good enough;
- The switching silicon has 10 queues per port (nice!);
- Switching latency is approximately 500 ns and can be reduced to 300 ns if the chipset is reconfigured into doing only simple L2 switching;
- Unified forwarding table (UFT; 128K entries) is split in memory banks that can be allocated to L2 entries, ARP entries and L3 LPM entries;
- One of the printouts in the presentation hinted at 1K LPM IPv6 prefixes longer than /64;
- Tomahawk support exact matching of ACL entries in UFT (not TCAM). UFT split with filter-mode profile can have 64K ACL entries, 16K IP LPM entries and 8K ARP/MAC entries;
- There are 43 queues between the switching silicon and CPU, and you can configure control-plane policing parameters on each queue;
- The hardware supports 16K MPLS labels (must be a separate MPLS lookup table, not TCAM tricks);
- TCAM slicing is too tricky for me to understand, but it seems you’ll get between 512 and 6K TCAM entries based on the complexity of the matching conditions. Based on the matching length used by Junos you get up to 512 port- or VLAN ACL entries or up to 1024 IP ACL entries;
- TCAM is not wide enough for all possible IPv6 matching conditions, so the hardware uses address compression. It seems you can have at most 128 source and destination IPv6 addresses in all filters deployed on the box;
Have I missed or misunderstood something? Please write a comment!
Btw, HP´s A5510 HI switches (== H3C S5560-EI) supports multiple table capacity modes:
0: 32000 MAC, 16000 ARP or 8000 ND, 8000 MPLS labels
1: 64000 MAC, 16000 ARP or 8000 ND, 4000 MPLS labels
2: 32000 MAC, 32000 ARP or 16000 ND, 8000 MPLS labels
It would be interesting to know what Broadcom ASIC they use on those
"16K to 128K MAC addresses (The number is configurable...)"
"VLAN mapping 64K entries (configurable)"
"ND 32K entries (configurable)"
"64 OpenFlow instances"
"6K extensibility flow entries"
"MAC-IP flow table"
But what ASIC?
P.S.: The problem with HP is they don´t know their products very well to answer those questions, which is sad...
Jim Warner, Univ Cal Santa Cruz
-jim warner, UCSC
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-3000-series-switches/datasheet-c78-736608.html
Tw pair and SFP+ models. No 100 Gb/s ports. It will be interesting to see if this is the start of a raft of products from the usual suspects.
UFT split with filter mode will in theory provide 64K ACL entries, but how does this translate to only 512 to 1K with complex matches? This part is not clear, wondering if there are any datasheets that can shed more light on this.
It would be a miracle if you'd manage to find more specific publicly available information ;) in which case please post a link!
H3C S6520-EI => http://download.h3c.com.cn/download.do?id=2485742 => 12.8Tbps
H3C S5560X-EI => http://download.h3c.com.cn/download.do?id=2697173 => 5.98Tbps