Category: BGP

Following my IBGP or EBGP in an enterprise network post a few people have asked for a more graphical explanation of IBGP/EBGP differences. Apart from the obvious ones (AS path does not change inside an AS) and more arcane ones (local preference is only propagated on IBGP sessions, MED of an EBGP route is not propagated to other EBGP neighbors), the most important difference between IBGP and EBGP is BGP next hop processing.

SK left a long comment to my More OSPF-over-DMVPN Questions post describing a scenario I find quite often in enterprise networks:

• Primary connectivity is provided by an MPLS/VPN service provider;
• Backup connectivity should use DMVPN;
• OSPF is used as the routing protocol;
• MPLS/VPN provider advertises inter-site routes as external OSPF routes, making it hard to properly design the backup connectivity.

If you’re familiar with the way MPLS/VPN handles OSPF-in-VRF, you’re probably already asking the question, “How could the inter-site OSPF routes ever appear as E1/E2 routes?”

I got the following question from one of my readers:

I recently started working at a very large enterprise and learnt that the network uses BGP internally. Running IBGP internally is not that unexpected, but after some further inquiry it seems that we are running EBGP internally. I must admit I'm a little surprised about the use of EBGP internally and I wanted to know your thoughts on it.

Although they are part of the same protocol, IBGP and EBGP solve two completely different problems; both of them can be used very successfully in a large enterprise network.

In the next few days, I'll write about some of the interesting topics we’ve been discussing during the last week’s fantastic on-site workshop with Ian Castleman and his team. To get us started, here’s a short video describing BGP/IGP network design principles. It’s taken straight from my Building IPv6 Service Provider Core webinar (recording), but the principles apply equally well to large enterprise networks.

A while ago someone asked me to help him troubleshoot his Internet connectivity. He was experiencing totally weird symptoms that turned out to be a mix of MTU problems, asymmetric routing (probably combined with RPF checks on ISP side) and non-routable PE-CE subnets. While trying to figure out what might be wrong from the router configurations, I was surprised by the amount of complexity he’d managed to introduce into his DMZ design by following recipes and best practices we all dole out in blog posts, textbooks and training materials.

John (not a real name for obvious reasons) sent me an interesting challenge after attending my Enterprise MPLS/VPN Deployment webinar. He’s designed an MPLS/VPN network approximated by the following diagram:

If you’re building a DMVPN network with large spoke-to-hub ratio, BGP is one of the better options – it has no scalability limitations associated with multicast flooding; the only parameter you have to consider is the number of BGP sessions the hub router can handle (and according to this presentation, ASR can handle 2000+ spokes).

You’re probably familiar with the April fat fingers incident in which Chinanet (AS 23724) originated ~37.000 prefixes for about 15 minutes. The incident made it into the annual report of US Congress’ U.S.-China Economic and Security Review Commission (page 243 of this PDF) and the media was more than happy to pick it up (Andree Toonk has a whole list of links in his blog post). We might never know whether the misleading statements in the report were intentional or just a result of clueless technical advisors, but the facts are far away from what they claim:

I’ve been mentioning LISP several times during the last months. It seems to be the only viable solution to the global IP routing table explosion. All other proposals require modifying layers above IP and while that’s where the problem should have been solved, expecting those layers to change any time soon is like waiting for Godot.

If you’re interested in LISP, start with the introduction to LISP I wrote for Search Telecom, continue with the LISP tutorial from NANOG 45 and (for the grand finale) listen to three Google Talks from Dino (almost four hours).

Read my article @ SearchTelecom.com

If you’re in the Service Provider business, this is (hopefully) old news: on Friday, RIPE decided to experiment with the Internet causing routers running IOS-XR to hiccup. They stopped the experiment in less than half an hour and only 2% of the Internet was affected according to Renesys analysis (a nice side effect: Tassos had great fun decoding the offending BGP attribute from hex dumps).

My first gut reaction was “something’s doesn’t feel right”. A BGP bug in IOS-XR affects only 2% of the Internet? Here are some possible conclusions:

Every now and then an incident reminds us how vulnerable BGP is. Very few of these incidents are intentional (the Pakistan vs. YouTube is a rare exception) and few of them are propagated far enough to matter on a global scale (bugs in BGP implementations are scarier). Most of these incidents could be prevented with either Secure BGP or Secure Origin BGP but it looks like they will not be implemented any time soon.

Jerry sent me an interesting question:

I was wondering if there's a way to modify an as-path access-list much like we do with regular access lists, simply by adding/ removing lines according to their sequence numbers.

I'm not aware of any such mechanism in Cisco IOS (but then maybe I’m missing something), but his question made me wonder: if you’re maintaining large AS-path access lists, do you edit them on the router (I guess not) or off-line (on a NMS platform) and download them when they need to be changed?

One of the decades-long grudges most people have with BGP is that it’s so easy to insert bogus routing information into the Internet if your upstream ISP happens to be a careless idiot (as Google discovered when Pakistan decided to use blackhole routing for Youtube and leaked the routes). There are two potential solutions that use X.509 certificates to authenticate BGP information: Secure BGP (which uses optional transitive attributes) authenticates the originator as well as the whole AS-path (using AS-by-AS certificates), while the significantly simpler Secure Origin BGP (which uses new BGP messages) authenticates only the originator of the routing information.

A post in the My CCIE Training Guide pointed me to the GoogleTechTalk given by Yakov Rekhter (one of the fathers of BGP) in 2007. You should watch the whole video (it helps you understand numerous BGP implementation choices), but its most important message is undoubtedly the Design by Pragmatism approach:

• They had a simple, manageable problem (get from a spanning-tree Internet topology to a mesh topology).
• They did not want to solve all potential future problems; they left that marvelous task to IDRP (which still got nowhere the last time I've looked).
• They started with simple specifications (three napkins), had two interoperable implementations in a few months, and wrote the RFC after BGP was already in production use.
• They rolled it out, learnt from its shortcomings and fixed it.
• They gradually made it easily extensible: TLV encoding, optional attributes, capabilities negotiations. This approach made it possible to carry additional address families in BGP and use it for applications like MPLS VPN and VPLS.

One could only hope that the IPv6 architects had used the same approach ... but as Yakov said in his talk, that’s “water under the bridge”.

TL&DR: QPPB works in MPLS VPNs… with a few limitations (at least in Cisco IOS implementation).

And now for the long story: A while ago I’ve noticed that my LinkedIn friend Joe Cozzupoli changed his status to something like “trying to get QPPB to work in MPLS VPN environment”. I immediately got in touch with him and he was kind enough to send me working configurations; not just for the basic setup, but also for Inter-AS Option A, B and C labs.

Knowing that QPPB relies on CEF, I doubted it would work as well on VRF interfaces as it does in pure IP environments, so I decided to do a few tests of my own. Here are the limitations I found: