Blog Posts in March 2009

Generating layer-2 broadcast from a regular IP packet

The Wake-on-LAN discussion we had a while ago brought us nowhere; there's simply no way to generate UDP packets on the router. I thought I could use Application Performance Monitor's Tcl scripts to generate the packet, but it looks like APM has been removed from recent IOS releases (and it's not clear whether you can use APM without a peer router).

The discussion nonetheless had an interesting side effect. Robert Turnšek sent me an interesting trick: with static ARP you can generate layer-2 broadcasts with a layer-3 unicast packet.

read more see 3 comments

Ah, the wonderful quoting process

Network World just published an article on LISP, including a quote attributed to Geoff Huston:

LISP relies on tunneling, and tunneling is not 100% perfect. At times, the tunnel passes a packet that's too big and it disappears without a trace to the sender or the recipient…That's really bad.

We all know what can go wrong with the tunnels: a combination of:

  • MTU setting on a core link that is too low to accomodate tunnel envelope+payload with
  • Tunnel headend that cannot respond to Fragmentation needed ICMP message.
read more add comment

Rate-limiting Inbound Traffic on DSL

Julian is faced with an interesting challenge:

In the real world, many customers using DSL solutions have their Internet connection disrupted by one internal user performing a large download. On a typical DSL solution, implementing quality of service on outbound traffic is trivial (you can use PQ, CBWFQ, policing or shaping). However, how does one rate-limit inbound traffic in a sensible fashion? Turnkey solutions like packeteer allow inbound classes of traffic like HTTP to be rate limited per flow by dynamically changing window sizes.

Cisco IOS has three basic QoS mechanisms: queuing, shaping and policing. It cannot intercept a TCP session and slow it down by reducing its window size (like PacketShaper).

read more see 5 comments

ADSL overhead

Yesterday I’ve described the difference between line rate and bit rate (actually physical layer gross bit rate and physical layer net bit rate). Going to the other extreme, we can measure goodput (application-level throughput), which obviously depends on multiple factors, including the TCP window sizes and end-to-end delays. There are numerous tools to test the goodput from/to various locations throughout the world ( worked quite nicely for me) and you’ll soon discover that the goodput on your DSL line differs significantly from what the ISP is advertising.

read more see 10 comments

Line rate and bit rate

Rajendra had an interesting problem:

Recently I got confused with the term line-rate. Is it the packets being switched across the switch fabric or control packets destined to the protocol tasks or both or something else?

The line rate is a physical layer term that has nothing to do with the line cards or switching fabrics. It indicates the actual speed with which the bits are sent onto the wire (and is thus also known as physical layer gross bit rate). The data transfer rate (commonly known as bit rate) is the transfer rate offered by the physical layer to the data link layer. If you want to be precise, you should call it physical layer net bit rate.

Two well-known physical layer technologies with different line rate and data transfer rate are ISDN (actually the I.430 recommendation) with 160192 kbps line rate and 144 kbps data transfer rate and Gigabit Ethernet (the 802.3z recommendation) with 1.25 Gbps line rate (due to 8b/10b encoding).

This article is part of You've asked for it series.

see 3 comments

Bandwidth allocation with class-based weighted fair queuing (CB-WFQ)

Sebastian sent me an interesting question:

I have read that we can only use 75% of the bandwidth for the custom queues as 25% is reserved for the keepalives and routing protocol updates. If I want to set 50% of bandwidth for a particular queue should it 50% of the total bandwidth or 50% of the available 75% of the bandwidth?

Before going into the details, it’s important to remember that the WFQ (CB-WFQ is only a mechanism to sort packets into output queues) uses relative ratios (percentages) between queues to determine which packet to send (the absolute bandwidths are used just to compute the ratios).

read more see 7 comments

Over a million page views

Two weeks ago Douglas Crockford wrote that his blog generated more than a million page views. Since he's an icon in the mainstream Web industry with his own Wikipedia listing, I simply had to check how my blog compares to that. Not bad, claims I had 1.2 million page loads in two and a half years.
If you'll remain as loyal readers as you've been in the first three months of this year, we'll probably reach the million-a-year mark in 2009.
see 4 comments

Blocking rogue DHCP servers

The reader who was concerned about making a loop while connecting a switch to itself was also facing “customer-installed” DHCP servers in his LAN. He wrote:

Some users have installed their own Linksys routers and plug our cable in router's LAN ports, so there is DHCP servers fight in our LAN. How can I sort this out (I cannot physically find the location of the Linsys routers)?

The ideal solution is DHCP snooping (assuming your switch supports it), well documented on The basic configuration takes only a few minutes:

read more see 7 comments

Network Migration with BGP Local-AS Feature

I’ve always thought that Cisco introduced the BGP Local-AS feature into IOS to support complex MPLS VPN design scenarios. Obviously I was wrong, the early documentation always describes an ISP AS merging scenario. Unfortunately, all the articles I’ve found skip some important details: they describe the basics and the configuration commands, but forget to mention the impact on the AS paths received by the ISP customers.

I described those details in an article that has disappeared from the Internet sometime in 2019, but fortunately retained a copy of it.

Would you like me to migrate that article to Send me a message and I just might do it...

see 2 comments

Last-resort password recovery

Pappyar has sent me an interesting password recovery technique, which can be used in those weird circumstances when you cannot force the router to go to ROMMON (for example, you’ve configured no service password-recovery and the break signal does not work as expected). Unfortunately, his trick works only if you can remove the flash memory from the router (it’s soldered in low-end models):

  1. Turn off the router.
  2. Take out the flash.
  3. Turn on the router.
  4. This time router will take you to ROMMON as it cannot find an IOS image.
  5. Set the configuration register with confreg 0x2142.
  6. Reset (this will change the stored value of the configuration register).
  7. Turn off the router.
  8. Reinsert the flash.
  9. Turn on the router and you are done.
see 10 comments

EBGP Load Balancing with a Multihop EBGP Session

Multihop EBGP sessions are the traditional way to implement EBGP load balancing on parallel links. EBGP session is established between loopback interfaces of adjacent routers (see the next diagram; initial router configurations are included at the bottom of the article) and static routes (or an extra instance of a dynamic routing protocol) are used to achieve connectivity between loopback interfaces (BGP next-hops). The load balancing is an automatic result of the recursive route lookup of BGP next hops.

The following text written by Ivan Pepelnjak in 2009 was originally published on CT3 wiki. That web site became unreachable in early 2019. We retrieved the original text from the Internet Archive, cleaned it up, updated it with recent information if necessary, and republished it on blog on March 13, 2009
read more see 4 comments

Fat finger follow-up: use the key labels

Karsten Iwen made an interesting comment to my “Don't let a lab rat anywhere near a production box” post: you should avoid the SSH/VPN key generation mistakes by using key labels. He also wrote a post explaining the concept but since it’s in German, let me rephrase it in English.

Cisco IOS release 12.2(8)T added the label parameter to the crypto key generate rsa command. You can use this parameter to assign a label to your VPN key, for example

Rtr(config)#crypto key generate rsa label VPN modulus 2048

To use the labeled key to generate your certificate, use the rsakeypair command in the CA-trustpoint configuration mode:

crypto pki trustpoint
enrollment retry count 100
enrollment mode ra
enrollment url
rsakeypair VPN
see 1 comments

Build a VPN Across Your IP Network with Multi-VRF Feature

One of our customers had to provide end-to-end IP transport across their enterprise network for an outsourced video surveillance solution. We implemented a true VPN solution for them (the hosts in the enterprise network cannot access the surveillance equipment and vice versa) using the Multi-VRF feature available in all recent Cisco IOS releases.

I described the solution Add a VPN to an Enterprise Network with Multi-VRF Functionality article; you’ll find it somewhere in this list.

add comment

Don't let a lab rat anywhere near a production box

I tried to do a few simple NETCONF tests yesterday (I wanted to see how the router's configuration looks like when it's encoded in XML). I didn't want to start a lab for such a simple task and decided to use my home router. SSH was the only reasonable transport (you can't run BEEP with standard Linux tools), but it was not working on the router.

Obviously I've configured too many lab devices in the past. As soon as I've realized I had SSH problems, my fingers automatically typed crypto key generate rsa. A few milliseconds after I've hit ENTER it dawned on me that my router uses PKI certificates for the VPN connection to our network ... and I've managed to invalidate the router's certificate, which is one of the few things that a reload will not solve. Although our IT guru was fantastic and approved my (router's) certificate request late in the evening, I still felt bad about the whole experience.

And the moral? Don't let a lab wizard with fast fingers too close to a production box :)

see 1 comments

Four byte AS number support in Cisco IOS

Last week IOS release 12.4(24)T appeared on CCO. One of the significant improvements in that release (I can’t manage to get enthusiastic about new kludges to support the SIP morass) is the support for 4-byte BGP AS numbers.

Finally an enterprise network that uses Cisco routers to connect to the Internet can use the new AS numbers distributed by the regional registries (assuming you’re brave enough to run 12.4(24)T on your production gear). The Service Providers using 7600 routers will have to wait … corresponding 12.2SR release is not yet available.

Hat tip to Tassos @ CCIE in 3 months: he was the first to write about this feature.

see 2 comments

BGP AS-Path Prepending: Technical Details

I thought I knew all there is to know about the AS-path prepending before the February 2009 incident, which prompted me to focus on this particular Cisco IOS feature.

For example, did you know you could do inbound AS-path prepending? I didn’t, until Rodney Dunn from Cisco mentioned it in an e-mail exchange. Did you ever wonder whether the AS-path prepending affects inbound or outbound AS-path filters? I had a hunch it doesn’t, but was never sure. Time to figure out all the gory details…

read more add comment

AS-path incident: the bowdlerizer strikes

A post by CiscoSubnet has pointed me to the official IntelliShield alert describing the AS-path incident. While the alert is very well written (I wouldn’t expect less) and the associated Protecting Border Gateway Protocol for the Enterprise is excellent, I was highly amused by the following text:

Cisco Security Intelligence Operations has identified a method through which administrators can modify device configurations to mitigate the effects of the AS path processing issue. Administrators can limit the amount of AS path segments that are associated with any route by using the bgp maxas-limit feature …

The bgp maxas-limit command was suggested by some members of the NANOG mailing list as early as few hours after the incident (which happened on February 16th; the IntelliShield alert has been released on February 20th) and I’ve released a detailed article on February 17th. I know Cisco’s engineers did a great job in this particular case; why did someone have to run their results through the /usr/local/marketese | /usr/bin/bowdlerize?

see 1 comments