Oversized AS paths: Cisco IOS bug details

Numerous articles describing the widespread routing instabilities caused by sloppy parser of a small router vendor (including posts at BGPmon, Renesys, Arbor Security and my blog) hinted that the unusual BGP update caused so many problems because the ISPs were using outdated Cisco IOS releases. This is definitely not the case; all classic IOS releases were affected.

Rodney Dunn from Cisco and myself were quickly able to reproduce so far unknown bug in Cisco IOS that occurs only when the inbound AS-path contains close to 255 AS numbers and the router does inbound or outbound AS-path prepending. The new bug is tracked as CSCsx73770 and affects downstream EBGP or IBGP sessions as follows:

  • When you do inbound AS-path prepending and receive a BGP update where the total length of inbound AS-path and prepending exceeds 255, the AS-path in your BGP table is completely mangled. The path is sporadically advertised to IBGP or EBGP peers and kills downstream BGP sessions (remote BGP peer sends BGP notification due to invalid UPDATE message).
  • The results of inbound AS-path prepending are very hard to reproduce, so they might be completely random

  • When you do outbound AS-path prepending and send a BGP update where the total length of the AS-path in your BGP table, prepended AS-path and your own AS-number exceed 255, the outbound EBGP update is incorrect and the downstream EBGP peer sends BGP notification, resulting in BGP session reset. IBGP peers are not affected, as IOS does not perform AS-path prepending on IBGP sessions.
  • If you don’t prepend, you’re safe. IOS marks BGP paths with AS-path length greater or equal to 254 as invalid and does not propagate them, so the AS-path length in the outbound update can never exceed 255 without prepending.

All the problems associated with CSCsx73770 can be avoided if you limit the AS-path length with the bgp maxas-limit global configuration command.

Obviously, if a downstream device is running a very old IOS release that is vulnerable to CSCdr54230, it has no chance of surviving incoming BGP update.

When I was testing the IOS behavior in my lab, I was not aware how long the offending AS-path was, so my initial analysis was incorrect. With the input from NANOG mailing list supplying to the actual length of the AS-path, this is the most probably scenario: the offending AS-path length was close enough to 255 that the outbound AS-path prepending performed between Tier-1 providers (or their peers) that did not limit the AS-path length caused EBGP sessions to reset (if someone performs inbound AS-path prepending, the IBGP sessions would suffer). Those providers that implemented AS-path length limiting successfully protected their downstream peers and their customers (unless, of course, the customers were running really old IOS releases).

3 comments:

  1. Mister Pepelnjak your blog is just amazing ...

    Thanks for sharing, kind regards
    Laurent

    ReplyDelete
  2. CCIE-Routing&Switching17 December, 2009 23:33

    How Many people Know´s Logicalis in Latin America?

    ReplyDelete
  3. I am working at Logicalis
    what´s your question?

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.