Blog Posts in July 2009
Soon after I wrote the Designing Fast Converging BGP Networks article (you’ll find it somewhere in this list, one of my regular readers sent me an interesting problem: BGP sessions would be lost in his (IS-IS based) core network if he would use fall-over on IBGP neighbors and the BGP router would have a primary and a backup path to the IBGP neighbor.
It turned out to be an interesting side effect of aggressive route table purge following a link failure: the route to BGP neighbor was removed from the routing table before IS-IS ran SPF and installed an alternate route, and BGP decided it’s time to give up and terminate the session.
For more details, read the What Exactly Happens after a Link Failure blog post.
You might have noticed that your IOS release supports a ctunnel interface (hint: your image has to support CLNS) and wondered what it could do. Well, it’s a GRE tunnel between a pair of NSAPs, so you can transport IP traffic across your well-engineered CLNS network without ever exposing the core routers to the dangers of IP.
But wait, it gets better: starting with IOS releases 12.3(7)T and 12.2(33)SRA, you can transport IPv6 across the ctunnel interface. Unfortunately, they haven’t implemented MPLS over GRE over CLNS yet (the mpls ip command is present, but does not work).
It looks like there's at least one potentially very large-scale application that could use this feature.
Many thanks to those of you that responded with Netflix details (special thanks to Volcker for sending me the packet capture). Immediately after someone mentioned firewalls, I knew what the most sensible answer should be: to get across almost anything, use HTTP. No surprise, Netflix chose to use it. However, they’ve managed to deploy streaming video over TCP, which is not a trivial task. So, how did they do it?
I’m switching to the “traditional” summer schedule: until mid-August, I’ll post two to three shorter articles per week. I don’t want to spend too much of my vacation time writing, but I also don’t want to see you bored with dormant blogosphere.
Some of my projects will simply have to wait for the temperatures to drop, including a few selected Service Provider issues I’ve started writing about in the last weeks and the ADSL QoS topics (don’t worry, I haven’t forgotten them).
Fifteen years ago, the focus of the “true” service provider was on voice traffic and data offerings based on virtual circuits, implemented with a plethora of semi-compatible technologies slowly developed within the ITU organization: X.25, ISDN, Frame Relay and the all-encompassing ATM.
In the meantime, some relatively small companies (including Cisco, Wellfleet and 3Com) were producing so-called “routers” that supported two technologies nobody took seriously: Ethernet and IP.
Thanks to my readers, I often learn something completely new about the intricacies of Cisco IOS. The “VLAN Interface Status” post resulted in a comment about the SVI autostate concept, which is (not surprisingly) a somewhat muddy topic:
- In most cases, the SVI interface tracks the state of access and trunk ports using the VLAN. The details are well explained in the Understanding SVI Autostate section of the Cisco IOS documentation.
The important part of the SVI autostate calculation is the “port is in STP forwarding state for the VLAN” requirement. If a VLAN is not carried in a trunk port (for example, due to switchport trunk allowed configuration command), the trunk port’s status does not influence the autostate.
- In some IOS releases, you can exclude the individual physical ports from the autostate calculation with the switchport autostate exclude interface configuration command. Most commonly you’d want to exclude uplink ports on access switches.
- In some unspecified IOS releases (including 12.4T), you can use the (currently undocumented according to Command Lookup Tool) no autostate VLAN interface configuration command, which disables the autostate algorithm and makes the SVI interface permanently active.
Ying would like to have a PE-to-PE IPSec protection for traffic within a single VRF. For example, all traffic in VRF-A sent between PE-1 and PE-2 should be protected with IPSec and the PE-routers should be the endpoints of the IPSec session (CE-to-CE IPSec is trivial).
My first response was “hard to do”, then I started hallucinating about MPLS-over-GRE-over-IPSec-over-IP-over-MPLS tunnels between the PE-routers with tunnel-specific IGP and per-VRF BGP next hops. It can be done (we’ve implemented numerous large-scale MPLS/GRE/IPSec designs), but is there a simpler alternative? Please share your ideas in the comments.
Vijay sent me this question a while ago:
I have configured a L3 VLAN interface on a Cisco 3750 switch and assigned an IP address to it. I haven't assigned any ports to this VLAN. Why am I not able to ping the IP address of the VLAN interface from the switch itself?
The VLAN interface (like any other interface) has layer-1 and layer-2 state.
The layer-1 state is displayed in the Status column of the show ip interface brief command, the layer-2 state in the Protocol column.
A VLAN interface is always up, but its line protocol state tracks the state of attached ports: if at least one port is operational, the line protocol of the VLAN interface is up, otherwise it’s down. With no ports assigned to a VLAN, the line protocol of the VLAN interface is down, its IP address is not in the IP routing table and thus you cannot ping it.
In one of our discussions, Stretch provided an excellent graph illustrating that the ISP competition seems to reduce prices almost linearly and asked me in a later comment to justify the inverse relation between subscription charges and consumer choice.
You might consider this debate to be purely between Stretch and myself, but it’s an interesting example of what you might need to do in daily your job. If you want to be a great networking engineer, you have to be prepared to question everything, including common wisdoms, “well-known truths”, “common practices” and facts that look too good to be true. Ready? Let’s go …
I'm looking for details on how Netflix streams videos over the Internet. I've found their description of encoding and bit rates, but was not able to find lower-layer details (I can only assume they use UDP, but I would like to verify that with someone who's actually using the service).
I would also appreciate any information on whether they work with Service Providers (for example, using local direct peering) to ensure the upstream Internet connections are not clogged with streamed video.
Thanks in advance for your responses!
Every so often, someone asks me what tools I use to draw the diagrams. Years ago I was perfectly happy with Visio, but since Microsoft bought it, it became so bloated that I’ve been forced to drop it (it would take minutes to start on my laptop) and revert back to PowerPoint.
Cisco provides great icon libraries (including the visionary “space router” icon shown on the right) in Visio and PowerPoint format and I’m lucky enough to have an older version where the colors of the devices are not light blue but a darker shade of blue/green/gray. Drawing connections between the devices is obviously easier in Visio than in PowerPoint, but if you keep the diagrams simple, you can work around the limitations.
Export from PowerPoint to JPEG/PNG has always been a nightmare with dubious results (although it looks like the copy/paste from PowerPoint to Paint Shop Pro produces reasonable quality in some cases). To work around this, I’m using SnagIT from Techsmith to capture the screen (Paint Shop Pro also has a screen capture utility, but SnagIT is so much easier to use) and trim/resize the images.
After the image is trimmed and resized, I need to add the final touch: replace the white background with PNG transparency so the diagrams look good in our Wiki, where the images are shown in a light gray frame. I use Paint Shop Pro as I happen to have it installed (SnagIT does not have this functionality), but any other decent image manipulation tool or even a PERL script with ImageMagick (which I am too lazy to write) would do.
The “All-I-can-eat-mentality” article has triggered (as expected) numerous responses. Some of them provided useful data, links to more information or informative perspectives – many thanks to those readers. A few others were unfortunately following the “I-am-right” line without considering facts. Most of the readers from the Service Provider community decided to stay anonymous (when you read all the comments, it becomes obvious they made a wise decision) or respond off-line.
Whatever your position in this issue, I would like to ask you to keep your comments focused on the topic. Although you were all infinitely more polite than the usual forum/blog crowd and provided some really good arguments, writing angry replies does not help. What’s happening with Internet is (like it or not) our common problem … or you could take the blue pill and continue bashing the other side.
I particularly liked the summary of our discussion posted on Slashdot (where someone included the link to my blog):
Whoa, whoa, whoa, that article seems to be promoting a balanced viewpoint that denies a) that telcos are totally evil and b) that we should all be allowed to have as much bandwidth as we want and not have to pay for it. We'll have none of that nonsense on /.
ATM has the potential to displace all existing internetworking technologies
One single network handles all traffic types: Bursty data and Time-sensitive continuous traffic (voice/video).
All these claims are still true if you just replace »ATM« with »IP«. So what went wrong with ATM (and why did the underdog IP win)? I can see the following major issues:
- ATM is a layer-2 technology that wanted to replace all other layer-2 technologies. Sometimes it made sense (ADSL), sometimes not so much (LAN … not to mention LANE). IP is a layer-3 technology that embraced all layer-2 technologies and unified them into a single network.
- ATM is an end-to-end circuit-oriented technology, which made perfect sense in a world where a single session (voice call, terminal session to mainframes) lasted for minutes or hours and therefore the cost of session setup became negligible. In a Web 2.0 world where each host opens tens of sessions per minute to servers all across the globe, the session setup costs would be prohibitive.
- Because of its circuit-oriented nature, ATM causes per-session overhead in each node in the network. Core IP routers don’t have to keep the session state as they forward individual IP datagrams independently. IP is thus inherently more scalable than ATM.
The shift that really made ATM obsolete was the changing data networking landscape: voice and long-lived low-bandwidth data sessions which dominated the world at the time when ATM was designed were dwarfed by the short-lived bursty high-bandwidth web requests. ATM was (in the end) a perfect solution to the wrong problem.
Two days ago I’ve managed to write aGenuineStupidity™ (OK, maybe I cannot get a trademark on this concept): the MQC shaping actions cannot be attached to a Dialer interface; they have to be specified on the underlying physical interface (in case of PPPoE link, the outside Ethernet interface).
The reason for my stupidity (apart from the obvious one: writing without testing) is the difference between true logical interfaces and dialer templates. A tunnel interface or a VLAN interface is a true logical interface; it behaves like any other interface (with a few exceptions; for example, tunnel interface does not have an output queue) and you can use most QoS actions (including shaping) on it. A dialer interface is even more “conceptual”. It can never be operational on its own – as soon as the link is established, it’s bound to a physical (for example, BRI0:1) or virtual access interface (which is yet again bound to a physical interface) and the shaping is performed on the final physical interface.
This behavior (on top of being unexpectedly inconsistent) results in interesting quirks. For example, you have to shape PPPoE packets (based on their IP characteristics) on the physical Ethernet interface which usually doesn’t even have an IP address.
… and let’s hope that the late hour hasn’t resulted in another blunder.