Category: BGP

Here’s a weird requirement that you could get on a really hard CCIE preparation lab (and hopefully never in a live network): redistribute external OSPF routes from selected ASBRs into BGP without using a route map on the redistribution router.

For example, assuming R1 and R2 insert external routes into OSPF, you want only routes from R1 to be redistributed into BGP on R3, but you cannot use route maps on R3.

One of the readers of my BGP Route Reflectors article spotted an “obvious deviation from how we always though the route reflectors work”:

An IBGP route received from a route-reflector client is sent to all IBGP peers, including the client from which it was received.

A quick lab test confirmed the validity of my claims: a BGP route reflector does send an update back to the client from which it was received (and it’s perfectly legal according to the updated BGP Route Reflector RFC).

I’ve always thought that Cisco introduced the BGP Local-AS feature into IOS to support complex MPLS VPN design scenarios. Obviously I was wrong, the early documentation always describes an ISP AS merging scenario. Unfortunately, all the articles I’ve found skip some important details: they describe the basics and the configuration commands, but forget to mention the impact on the AS paths received by the ISP customers.

I described those details in an article that has disappeared from the Internet sometime in 2019, but fortunately archive.org retained a copy of it.

Would you like me to migrate that article to ipSpace.net? Send me a message and I just might do it...

Multihop EBGP sessions are the traditional way to implement EBGP load balancing on parallel links. EBGP session is established between loopback interfaces of adjacent routers (see the next diagram; initial router configurations are included at the bottom of the article) and static routes (or an extra instance of a dynamic routing protocol) are used to achieve connectivity between loopback interfaces (BGP next-hops). The load balancing is an automatic result of the recursive route lookup of BGP next hops.

I thought I knew all there is to know about the AS-path prepending before the February 2009 incident, which prompted me to focus on this particular Cisco IOS feature.

For example, did you know you could do inbound AS-path prepending? I didn’t, until Rodney Dunn from Cisco mentioned it in an e-mail exchange. Did you ever wonder whether the AS-path prepending affects inbound or outbound AS-path filters? I had a hunch it doesn’t, but was never sure. Time to figure out all the gory details…

Numerous articles describing the widespread routing instabilities caused by sloppy parser of a small router vendor (including posts at BGPmon, Renesys, Arbor Security and my blog) hinted that the unusual BGP update caused so many problems because the ISPs were using outdated Cisco IOS releases. This is definitely not the case; all classic IOS releases were affected.

Rodney Dunn from Cisco and myself were quickly able to reproduce so far unknown bug in Cisco IOS that occurs only when the inbound AS-path contains close to 255 AS numbers and the router does inbound or outbound AS-path prepending. The new bug is tracked as CSCsx73770 and affects downstream EBGP or IBGP sessions as follows:

The “BGP experiment” a small European ISP performed in February 2009 has generated quite a splash: Cisco has discovered a new BGP bug that can be triggered only if you have a long enough AS-path and do outbound AS-path prepending (and a few of us learned more BGP intricacies we never wanted to know), lots of people have (hopefully) discovered the importance of the bgp maxas-limit configuration command and at least some ISPs have implemented inbound prepending filters that I wrote about almost a year ago. However, most of us thought that the original problem arose due to inexperienced operators of a leaf AS.

Mikael Abrahamsson was the first to notice that the number of prepends matches the low-order 8 bits of the offending AS number. Further contributors to NANOG mailing list confirmed that two autonomous systems with very long prepends are using BGP routers from Mikrotik. You configure those boxes with commands that have syntax deceptively close to Cisco's, but expect the number of AS numbers to prepend, not the AS-path. Obviously no range checking is done on the configuration parameter and the high-order 8 bits are ignored.

So it looks like the incident started with a box that accepts invalid configuration parameter used in an AS with very high value in low-order 8 bits (quite improbable, but obviously not impossible). Numerous ISPs that did not limit the BGP updates they were propagating and an IOS bug did the rest.

In February 2009, a greenhorn ISP (they joined RIPE less than four months before the incident) in central Europe managed to generate a BGP update with too many AS numbers in the AS path, confusing older routers. You can find the details in an old Renesys blog post; at the peak of the instability, they were receiving over 100.000 BGP updates per second.

It’s very easy to protect yourself (and your downstream neighbors) from an operational error like this one. Cisco has implemented the AS-path length limiting code in IOS release 12.2. One would hope that the major ISPs would have started using this feature years ago; obviously that’s not the case, so here’s how you do it… just to make sure everyone understands what the bgp maxas-limit command does and hopefully implements it in this millennium.

BGP allows numerous attributes (including AS-path, metrics, local preference and communities) to be attached to every advertised IP prefix. The total length of BGP attributes attached to a single IP prefix can be very large (up to 64K bytes). IP prefixes with excessive amount of attribute data residing in the BGP table can results in significant memory utilization and trigger software bugs.

Cisco IOS can limit the maximum length of the AS-path attribute with the bgp maxas-limit length router configuration command. It’s highly advisable that you use this command together with other BGP security measures to reduce the potential impact of oversized AS-path attributes on the operation of your network.

The maximum sensible length of the AS-path attribute depends on your position within the Internet. Core operators observe lower AS-path lengths than the edge points. Due to CSCdr54230, accepting AS-paths having more than approximately 100 AS numbers was best avoided; reasonable values are usually much lower… and as always, Geoff Huston published a measurement giving you the answer to that question.

Configuring the bgp maxas-limit command does not impact the regular BGP operation. The maxas-limit is checked during the inbound update processing. Prefixes with oversized AS-path length are simply ignored; BGP sessions are not disrupted.

Test bed description

The bgp maxas-limit functionality can be easily demonstrated in a test bed consisting of only two routers.

hostname R1
!
ip cef
!
interface Loopback0
 ip address 10.0.1.1 255.255.255.255
!
interface Serial1/0
 description Link to R2 s1/0
 ip address 10.0.7.13 255.255.255.252
 encapsulation ppp
!
router bgp 65000
 no synchronization
 bgp log-neighbor-changes
 network 10.1.1.0 mask 255.255.255.0
 network 10.1.2.0 mask 255.255.255.0
 neighbor 10.0.7.14 remote-as 65100
 neighbor 10.0.7.14 route-map prepend out
 no auto-summary
!
ip classless
!
ip route 10.1.1.0 255.255.255.0 Null0
ip route 10.1.2.0 255.255.255.0 Null0
!
ip prefix-list prepend seq 5 permit 10.1.2.0/24
!
route-map prepend permit 10
 match ip address prefix-list prepend
 set as-path prepend 65000 65000 65000 65000
!
route-map prepend permit 20
!
line con 0
 exec-timeout 0 0
 privilege level 15
 logging synchronous
 transport preferred none
 stopbits 1
!
ntp logging
end 

hostname R2
!
ip cef
!
interface Loopback0
 ip address 10.0.1.2 255.255.255.255
!
interface Serial1/0
 description Link to R1 s1/0
 ip address 10.0.7.14 255.255.255.252
 encapsulation ppp
!
router bgp 65100
 no synchronization
 bgp log-neighbor-changes
 bgp maxas-limit 3
 neighbor 10.0.7.13 remote-as 65000
 no auto-summary
!
line con 0
 exec-timeout 0 0
 privilege level 15
 logging synchronous
 transport preferred none
 stopbits 1
!
ntp logging
end 

Exception logging

The bgp maxas-limit functionality does not impact the regular BGP operation. Whenever an inbound BGP update is received with an oversized AS-path attribute, the router logs a warning message and ignores the update.

%BGP-6-ASPATH: Long AS path 65000 65000 65000 65000 65000
 received from 10.0.7.13: More than configured MAXAS-LIMIT 

The AS-path length limiting functionality can also be observed with any of the debug ip bgp update commands. A sample printout is included below:

BGP(0): 10.0.7.13 rcv UPDATE w/ attr: nexthop 10.0.7.13, origin i,
  metric 0, originator 0.0.0.0, path 65000 65000 65000 65000 65000,
  community , extended community , SSA attribute
BGPSSA ssacount is 0
BGP(0): 10.0.7.13 rcv UPDATE about 10.1.2.0/24 -- DENIED due to:
  AS-PATH length over 4072;
BGP(0): 10.0.7.13 rcvd UPDATE w/ attr: nexthop 10.0.7.13, origin i,
  metric 0, path 65000
BGP(0): 10.0.7.13 rcvd 10.1.1.0/24...duplicate ignored 

When I was discussing the details of the BGP troubleshooting video with one of my readers, he pointed out that I should mention the need for CEF switching in EBGP multipath scenario. My initial response was “Why would you need CEF? EBGP multipath is older than CEF” and his answer told me I should turn on my gray cells before responding to emails: “Your video as well as Cisco’s web site recommends CEF for EBGP multipath design… but interestingly, it does work without CEF”.

The real reason we need CEF in EBGP load sharing designs is the efficacy of load distribution. Without CEF, the router will send all traffic toward a single BGP prefix over one of the links (fast switching performs per-destination-prefix load sharing). With CEF, the load is distributed based on the source-destination IP address pair combinations. Even if multiple clients send the traffic toward the same server, the load is spread across available links.

Just stumbled across this: Amazon is offering the MPLS and VPN Architectures, MPLS and VPN Architectures, Volume II and Internet Routing Architectures (2nd Edition) (Networking Technology) (from Sam Halabi) for a total of $160.

read more see 3 comments

A week ago I've asked whether anyone has a good use for BGP aggregation. Out of 1300 daily visitors of my blog and 2200 subscribers to the RSS feed,  four (4) people found the time and energy to reply. Thank you for your thoughts!

Let me conclude that BGP aggregation is not widely used (so I will spend much energy covering it in my future posts) ... all the other potential conclusions are too pessimistic.

One of the presentations at the recent Defcon 16 event demonstrated how you can use the very common laziness of the Internet Service Providers to hijack any prefix you want (just ask YouTube). Nothing new so far, but the part where they fake the AS path in the hijacked announcement to create a safe (hijack-free) conduit back to the destination is brilliant ... and the TTL manipulation is the icing on the cake.

Of course this is a well-known BGP vulnerability (actually, implementation sloppiness on the part of ISPs) that we've been writing about for a long time, but the Defcon presentation is probably the first documented step-by-step recipe for a realistic MITM attack.

Hat tip to Jeremy Stretch; I found the link to the Defcon presentation on his blog.