Category: switching

I've found the following "gem" in the Catalyst 3000 LANE module data sheet:

The module "provides legacy LANs with access to ATM-based services in an ATM campus backbone".

The legacy LAN was switched Ethernet (which is still around after 15 years) and ATM campus backbones have joined the dinosaurs.

In case you've never seen a Catalyst 3000: it was a switch that Cisco got through one of its first acquisitions and although it was a good Ethernet switch, it was a nightmare to configure and the later additions (for example, the LANE module) were a disaster. Luckily, it was allowed to die a quiet death a few years later.

If you’re old enough to remember the days when switches were still called bridges and were used to connect multiple sites over WAN links, you’ve probably experienced interesting network meltdowns caused by a single malfunctioning network interface card. Some of you might have had the “privilege” of encountering another somewhat failed attempt at WAN bridging: ATM LAN Emulation (LANE) service (not to mention the “famous” Catalyst 3000 switches with LANE uplink).

It looks like some people decided not to learn from others’ mistakes: years later the bridging-over-WAN idea has resurfaced in the VPLS clothes. While there are legitimate reasons why you’d want to have a bridged connection across the Service Provider network, VPLS should not be used to connect regular remote sites to a central site without on-site routers, as I explained in the VPLS: A secure LAN cloud solution for some, not all article I wrote in 2009 (republished below).

read more see 7 comments

read more see 10 comments

One of my readers sent me a question that triggered one of my old grudges:

In my experience, when you first add a new switch (having a NULL domain) on an existing VTP Domain, it inherits the domain name, regardless of it being a VTP Server. I was wondering if this is a feature (i.e. has proved to be a solution in most cases) or a bug (i.e. has proved to cause problems in most cases). I know it's proved to be the latter for us!

In my personal opinion Cisco at one point in time wanted too much plug-and-play and someone had a great idea that you can just plug another switch into your network and it would autoconfigure itself. We've been suffering because of that "insight" ever since (and the CCIE written test has material for a few more interesting questions :).

I strongly believe that VTP should be turned off by default and should generate a warning before being enabled, but it will probably not happen. What do you think?

Disclaimer: I am not a switching person and have no idea about anything below or above layer 3.

The control and management planes in a network device run numerous mission-critical processes, including routing protocols and network management services (SNMP, telnet or SSH access to the router, web access to the router), and is thus the most vulnerable part of any network device.

A determined attacker can quickly overload the CPU of any router (or switch) with a targeted denial-of-service attack, either by sending IP packets that are propagated from the switching fabric (or interrupt code on software-only platforms) to the control plane processes or by targeting individual services running on the router. The situation is becoming worse with widespread use of high-speed hardware switching platforms that are connected to an underpowered CPU over a PCI bus; getting enough traffic to a network device to saturate the ASIC-to-CPU connection, or the CPU, is becoming trivial.

I was very anxious to get my copy of Cisco Secure Firewall Services Module (FWSM) from Cisco Press, as I’m a purely router-focused person, and I wanted to understand the capabilities of the Firewall Services Module (PIX/ASA-like blade for the Catalyst 6500 switching system with virtual firewall capability). I have a good background in IOS-based firewalls and network address translation (NAT), so the book was a perfect fit for me. However, if you’re looking for “best practices for securing networks with FWSM,” you’ve been misled by the subtitle.

The Thinking problem management! blog had an interesting article on The Leaky VLANs myth, quoting a test report from SANS Institute that documents how you can inject frames into other VLANs even if you're not connected to a trunk port. The report is eight years old (so one would hope this issue has been fixed in the meantime), but there's another question you should ask yourself is: what happens when you lose the configuration of the switch (and I've seen devices losing configuration after a power glitch). If you're using a router to perform L3 switching, no harm is done; a router with empty configuration forwards no packets. But if you're using a low-end switch, you're in deep trouble; by default, a switch forwards packets between all ports ... and if you use static IP addresses on all subnets, you won't even notice they're connected. If you want to be very safe, you're better off having a different set of switches for the inside and the outside zones of your firewall.

Peter Weymann sent me a really intriguing question:

A few days ago I started reading the Ciscopress book End-to-End Network Security: Defense-in-Depth and stumbled over the scheduler command. This one could be used to allocate time that the cpu spends on fast switching packets or process switching packets, if I understand it correctly. They also mention interrupting CPU processes but honestly I don't really understand how it works.

Cisco routers support (at least) three forms of layer-3 switching (formerly known as routing). CEF switching and fast switching are performed entirely within the interrupt context (I/O adapter interrupts a process the CPU is currently executing and all the work is done before the process resumes). Process switching is performed in two steps: packet is briefly analysed within the interrupt context and requeued into the IP Input process where it's eventually switched. Almost all I/O adapters used these days use a concept of RX/TX rings to communicate with the CPU, meaning that the CPU potentially has to handle more than one packet for each interrupt.

Fast switching is gone starting with IOS release 12.4(20)T.

Under very high load, the packet arrival rate could be so high that the router would constantly service packets within the interrupt context without ever returning back to the IOS processes.

You can check the CPU load incurred by the interrupt context and IOS processes with the show process cpu command. The second number in the five seconds part of the first line tells you the amount of interrupt context activity in the last five seconds.

To prevent the starvation of IOS processes (which could result in keepalive and routing protocol problems, eventually leading to loss of routing protocol neighbors), the scheduler allocate command limits the amount of time that can be spent in the interrupt context and allocates some guaranteed time to the IOS processes. Very probably the routers have a mechanism to mask the requests from the I/O adapters during that period so that the CPU is not interrupted (BTW, this slightly increases the jitter).

A similar command is the scheduler interval command. IOS has high- and low priority processes. Whenever the CPU has to decide what process to run (usually following an interrupt or when a process decides it's done with its work), it will run a high-priority process if one is ready. This could lead to starvation of low-priority processes and the scheduler interval command specifies the maximum amount of time the higher-priority processes can consume before a low-priority process is given a chance to run.

Unless you have serious (and I mean __serious__) problems in your network, don't play with these commands. They are a last-resort things you can do if you're under very heavy load and still need access to the exec to reconfigure the router. In most cases, you should not have to worry ... and anyhow, if the CPU load is close to 100%, you have other problems anyway.

Harold Arley Morales has asked an interesting question:

What's the difference between Cisco Express Forwarding and Cisco MLS? Is Cisco's implementation of MLS standardized?

CEF is a routing table lookup mechanism. Instead of doing a lookup in the main IP routing table (displayed with the show ip route), the router does a lookup in a fully computed non-recursive version of the IP routing table (Forwarding Information Base - FIB) with layer-2 next-hop information attached to it (adjacency table).

MLS is a caching mechanism (similar to Netflow) that offloads layer-3 processing from the routing component into layer-2 ASICs that cannot perform full-blown layer-3 switching. When the layer-2 engine detects a single IP packet traversing multiple VLANs, the MLS populates the cache with the flow details and the subsequent packets belonging to the same flow (same source/destination IP addresses and port numbers ...) are switched without going through all the layer-3 mechanisms (for example, access lists). The Multilayer Switching Overview document gives you additional details.

The MLS uses a proprietary protocol (MLSP) through which the layer-2 switches identify routers.

This article is part of You've asked for it series.

Update 2008-12-08: Ofer Granit sent me the following information: according to Troubleshooting IP Multilayer Switching document, Supervisor Engine 2 and Supervisor Engine 720 no longer use MLS but rely exclusively on CEF to perform layer-3 forwarding.

CCIE Pursuit writes about the usage of MAC addresses on SVI (VLAN) interfaces on various Catalyst switches. Some switches might reuse the same MAC address on multiple VLANs, leading to interesting problems if another box bridges those VLANs (for example, a transparent firewall).