Building network automation solutions

9 module online course

Start now!

Blog Posts in July 2009

Aggressive BGP fall-over behavior

Soon after I wrote the Designing Fast Converging BGP Networks article, one of my regular readers sent me an interesting problem: BGP sessions would be lost in his (IS-IS based) core network if he would use fall-over on IBGP neighbors and the BGP router would have a primary and a backup path to the IBGP neighbor.

We’ve identified the source of the problem (delayed SPF run) and I decided to document it. It took “a while” (and a kind nudge from Mateusz following a discussion on Cisco-NSP mailing list) to find time to set up the test lab and document the behavior. You can find the (undesired) results in the Aggressive BGP fall-over behavior in the CT3 wiki.

Just in case someone from Cisco wants to put this on a wish list: please add a configurable delay to the neighbor fall-over command like you did with the bgp nexthop trigger delay command.

see 2 comments

What is a CTunnel interface?

You might have noticed that your IOS release supports a ctunnel interface (hint: your image has to support CLNS) and wondered what it could do. Well, it’s a GRE tunnel between a pair of NSAPs, so you can transport IP traffic across your well-engineered CLNS network without ever exposing the core routers to the dangers of IP.

But wait, it gets better: starting with IOS releases 12.3(7)T and 12.2(33)SRA, you can transport IPv6 across the ctunnel interface. Unfortunately, they haven’t implemented MPLS over GRE over CLNS yet (the mpls ip command is present, but does not work).

It looks like there's at least one potentially very large-scale application that could use this feature.

see 4 comments

When 4 x 0 != 0

As we all know, OSPF area ID is a 32-bit quantity that can be written as a decimal number or as a dotted decimal (although I never really understood the need for dotted decimal support). You would expect both representations to behave identically (after all, it’s a 32 bit number we’re talking about) and most often they do. However, Murali reported an interesting bug: some show commands do not understand that 4 times zero still equals zero.

read more see 4 comments

Netflix summary

Many thanks to those of you that responded with Netflix details (special thanks to Volcker for sending me the packet capture). Immediately after someone mentioned firewalls, I knew what the most sensible answer should be: to get across almost anything, use HTTP. No surprise, Netflix chose to use it. However, they’ve managed to deploy streaming video over TCP, which is not a trivial task. So, how did they do it?

read more see 2 comments

Summer schedule

I’m switching to the “traditional” summer schedule: until mid-August, I’ll post two to three shorter articles per week. I don’t want to spend too much of my vacation time writing, but I also don’t want to see you bored with dormant blogosphere.

Some of my projects will simply have to wait for the temperatures to drop, including a few selected Service Provider issues I’ve started writing about in the last weeks and the ADSL QoS topics (don’t worry, I haven’t forgotten them).

add comment

Wimax: the next disruptive technology?

Fifteen years ago, the focus of the “true” service provider was on voice traffic and data offerings based on virtual circuits, implemented with a plethora of semi-compatible technologies slowly developed within the ITU organization: X.25, ISDN, Frame Relay and the all-encompassing ATM.
In the meantime, some relatively small companies (including Cisco, Wellfleet and 3Com) were producing so-called “routers” that supported two technologies nobody took seriously: Ethernet and IP.

read more see 2 comments

Followup: VLAN interface status

Thanks to my readers, I often learn something completely new about the intricacies of Cisco IOS. The “VLAN Interface Status post resulted in a comment about the SVI autostate concept, which is (not surprisingly) a somewhat muddy topic:

  • In most cases, the SVI interface tracks the state of access and trunk ports using the VLAN. The details are well explained in the Understanding SVI Autostate section of the Cisco IOS documentation.

The important part of the SVI autostate calculation is the “port is in STP forwarding state for the VLAN” requirement. If a VLAN is not carried in a trunk port (for example, due to switchport trunk allowed configuration command), the trunk port’s status does not influence the autostate.

  • In some IOS releases, you can exclude the individual physical ports from the autostate calculation with the switchport autostate exclude interface configuration command. Most commonly you’d want to exclude uplink ports on access switches.
  • In some unspecified IOS releases (including 12.4T), you can use the (currently undocumented according to Command Lookup Tool) no autostate VLAN interface configuration command, which disables the autostate algorithm and makes the SVI interface permanently active.
see 5 comments

PE-to-PE IPSec: do you have creative ideas?

Ying would like to have a PE-to-PE IPSec protection for traffic within a single VRF. For example, all traffic in VRF-A sent between PE-1 and PE-2 should be protected with IPSec and the PE-routers should be the endpoints of the IPSec session (CE-to-CE IPSec is trivial).

My first response was “hard to do”, then I started hallucinating about MPLS-over-GRE-over-IPSec-over-IP-over-MPLS tunnels between the PE-routers with tunnel-specific IGP and per-VRF BGP next hops. It can be done (we’ve implemented numerous large-scale MPLS/GRE/IPSec designs), but is there a simpler alternative? Please share your ideas in the comments.

see 20 comments

Quick tip: VLAN interface status

Vijay sent me this question a while ago:

I have configured a L3 VLAN interface on a Cisco 3750 switch and assigned an IP address to it. I haven't assigned any ports to this VLAN. Why am I not able to ping the IP address of the VLAN interface from the switch itself?

The VLAN interface (like any other interface) has layer-1 and layer-2 state.

The layer-1 state is displayed in the Status column of the show ip interface brief command, the layer-2 state in the Protocol column.

A VLAN interface is always up, but its line protocol state tracks the state of attached ports: if at least one port is operational, the line protocol of the VLAN interface is up, otherwise it’s down. With no ports assigned to a VLAN, the line protocol of the VLAN interface is down, its IP address is not in the IP routing table and thus you cannot ping it.

This article is part of You've asked for it series.

see 18 comments

Goodbye, Blogger comments

I received numerous complaints about the inability to write comments to my blog in the last few weeks. I had comment-related problems for months, but the situation was getting out of control: initially only the Internet Explorer users had (manageable) problems, then it spread to some Firefox users, later it became impossible to submit comments at all and the last marvel in this buggy saga was the “automatic” conversion from Publish request to preview screen. I had enough, Google has yet again proven that you usually get what you pay for.

Today I stumbled across a blog post shared by one of my Facebook friends that mentioned JS-Kit services. I also happened to have a few spare creative hours (one of my appointments was pushed forward) and tried the JS-Kit comments on one of my rarely-visited blogs. After the initial crashes I got it to work and implemented it on the IOS Hints blog. If you’ll experience any problems with the new system, please let me know (get in touch with me via my e-mail web form if you can’t leave a comment).

Notice to NOSCRIPT users: you have to allow scripts from JS-KIT.COM if you want to leave a comment; you should have no problems seeing the already posted ones.

see 1 comments

Question everything

In one of our discussions, Stretch provided an excellent graph illustrating that the ISP competition seems to reduce prices almost linearly and asked me in a later comment to justify the inverse relation between subscription charges and consumer choice.

You might consider this debate to be purely between Stretch and myself, but it’s an interesting example of what you might need to do in daily your job. If you want to be a great networking engineer, you have to be prepared to question everything, including common wisdoms, “well-known truths”, “common practices” and facts that look too good to be true. Ready? Let’s go …

read more see 2 comments

Looking for additional information on Netflix video streaming

I'm looking for details on how Netflix streams videos over the Internet. I've found their description of encoding and bit rates, but was not able to find lower-layer details (I can only assume they use UDP, but I would like to verify that with someone who's actually using the service).

I would also appreciate any information on whether they work with Service Providers (for example, using local direct peering) to ensure the upstream Internet connections are not clogged with streamed video.

Thanks in advance for your responses!
Ivan

see 8 comments

Drawing the diagrams

Every so often, someone asks me what tools I use to draw the diagrams. Years ago I was perfectly happy with Visio, but since Microsoft bought it, it became so bloated that I’ve been forced to drop it (it would take minutes to start on my laptop) and revert back to PowerPoint.

Cisco provides great icon libraries (including the visionary “space router” icon shown on the right) in Visio and PowerPoint format and I’m lucky enough to have an older version where the colors of the devices are not light blue but a darker shade of blue/green/gray. Drawing connections between the devices is obviously easier in Visio than in PowerPoint, but if you keep the diagrams simple, you can work around the limitations.

Export from PowerPoint to JPEG/PNG has always been a nightmare with dubious results (although it looks like the copy/paste from PowerPoint to Paint Shop Pro produces reasonable quality in some cases). To work around this, I’m using SnagIT from Techsmith to capture the screen (Paint Shop Pro also has a screen capture utility, but SnagIT is so much easier to use) and trim/resize the images.

After the image is trimmed and resized, I need to add the final touch: replace the white background with PNG transparency so the diagrams look good in our Wiki, where the images are shown in a light gray frame. I use Paint Shop Pro as I happen to have it installed (SnagIT does not have this functionality), but any other decent image manipulation tool or even a PERL script with ImageMagick (which I am too lazy to write) would do.

see 11 comments

PPPoE testbed

Following the ADSL QoS discussions, I decided to test “a few” things in the lab. I didn’t want to build a huge lab with DSL modems and DSLAM and decided to emulate an end-to-end DSL network with routers.

Step 1: Create a PPPoE session between a client (SOHO router) and a server (NAS). I’d never configured it before, so I’ve visited uncle Google first. It gave me tons of useless hits (no wonder) and a few somewhat useful ones. All of them used the old VPDN-centric PPPoE syntax. It seems I’ve stumbled across another “niche spot”: PPPoE testbed implemented with recent IOS configuration syntax (bba-group).

Read more in the “PPPoE Testbed” article in the CT3 wiki.

see 2 comments

When the “passive-interface” command is missing from OSPF

A few days ago a member of the cisco-nsp mailing list asked an interesting question: “the passive-interface command is not available in a VRF OSPF process. What can I do?”

It turned out he stumbled across CSCeb86068, which is already fixed in a later software release for his platform.

The passive-interface command tells the routing process to ignore packets received from the specified interface. In case of OSPF, the relevant packets are the hello packets, as an OSPF router will not exchange routing updates without an established adjacency. You can get the same results by deploying an inbound access list on the interface (which is the functionally equivalent workaround for this bug), although this method generates more configuration overhead than the OSPF-specific solution.

read more see 1 comments

Trigger EEM applet with SNMP

Anderson sent me an interestion question:

I'd like to use the snmpset command to get my router to execute an EEM script. Are there OIDs that are associated with EEM scripts that could help me achieve this?

Although EEM has associated MIB, it has a single read-write variable: the size of the history table. It's thus not possible to use EEM MIB to trigger EEM events. However, EEM 2.4 added support for SNMP notification events, which you can use to trigger EEM applets based on incoming SNMP traps/informs.

You can therefore use the event snmp-notification command on a router and the snmptrap command on a Linux host to remotely trigger EEM applets.

Read more in the Trigger EEM applets with SNMP Informs article in the CT3 wiki.

This article is part of You've asked for it series.

add comment

Followup: All-I-can-eat

The “All-I-can-eat-mentality” article has triggered (as expected) numerous responses. Some of them provided useful data, links to more information or informative perspectives – many thanks to those readers. A few others were unfortunately following the “I-am-right” line without considering facts. Most of the readers from the Service Provider community decided to stay anonymous (when you read all the comments, it becomes obvious they made a wise decision) or respond off-line.

Whatever your position in this issue, I would like to ask you to keep your comments focused on the topic. Although you were all infinitely more polite than the usual forum/blog crowd and provided some really good arguments, writing angry replies does not help. What’s happening with Internet is (like it or not) our common problem … or you could take the blue pill and continue bashing the other side.

I particularly liked the summary of our discussion posted on Slashdot (where someone included the link to my blog):

Whoa, whoa, whoa, that article seems to be promoting a balanced viewpoint that denies a) that telcos are totally evil and b) that we should all be allowed to have as much bandwidth as we want and not have to pay for it. We'll have none of that nonsense on /.
read more see 9 comments

What went wrong: end-to-end ATM

Red Pineapple was kind enough to share his 15-year-old ATM slides. They include interesting claims like:

ATM has the potential to displace all existing internetworking technologies
One single network handles all traffic types: Bursty data and Time-sensitive continuous traffic (voice/video).

All these claims are still true if you just replace »ATM« with »IP«. So what went wrong with ATM (and why did the underdog IP win)? I can see the following major issues:

  • ATM is a layer-2 technology that wanted to replace all other layer-2 technologies. Sometimes it made sense (ADSL), sometimes not so much (LAN … not to mention LANE). IP is a layer-3 technology that embraced all layer-2 technologies and unified them into a single network.
  • ATM is an end-to-end circuit-oriented technology, which made perfect sense in a world where a single session (voice call, terminal session to mainframes) lasted for minutes or hours and therefore the cost of session setup became negligible. In a Web 2.0 world where each host opens tens of sessions per minute to servers all across the globe, the session setup costs would be prohibitive.
  • Because of its circuit-oriented nature, ATM causes per-session overhead in each node in the network. Core IP routers don’t have to keep the session state as they forward individual IP datagrams independently. IP is thus inherently more scalable than ATM.

The shift that really made ATM obsolete was the changing data networking landscape: voice and long-lived low-bandwidth data sessions which dominated the world at the time when ATM was designed were dwarfed by the short-lived bursty high-bandwidth web requests. ATM was (in the end) a perfect solution to the wrong problem.

see 3 comments

Not all interfaces are created equal

Two days ago I’ve managed to write aGenuineStupidity™ (OK, maybe I cannot get a trademark on this concept): the MQC shaping actions cannot be attached to a Dialer interface; they have to be specified on the underlying physical interface (in case of PPPoE link, the outside Ethernet interface).

The reason for my stupidity (apart from the obvious one: writing without testing) is the difference between true logical interfaces and dialer templates. A tunnel interface or a VLAN interface is a true logical interface; it behaves like any other interface (with a few exceptions; for example, tunnel interface does not have an output queue) and you can use most QoS actions (including shaping) on it. A dialer interface is even more “conceptual”. It can never be operational on its own – as soon as the link is established, it’s bound to a physical (for example, BRI0:1) or virtual access interface (which is yet again bound to a physical interface) and the shaping is performed on the final physical interface.

This behavior (on top of being unexpectedly inconsistent) results in interesting quirks. For example, you have to shape PPPoE packets (based on their IP characteristics) on the physical Ethernet interface which usually doesn’t even have an IP address.

… and let’s hope that the late hour hasn’t resulted in another blunder.

see 1 comments
Sidebar