Prevent bridging loops without BPDUs?

Anton sent me an interesting question:

Most IP phones have a network facing port and a port for user to connect the PC. Today a user plugged in both of these ports into the switch. It looks like phone filters out BPDUs, so the switch did not catch this loop. Do you know of a feature or design that would be able to catch/prevent this type of event?

My answer would be “no, there’s nothing you can do if you have a broken device that acts like a STP-less switch” but you know I’m not a switching or IP telephony guru. Any ideas?

30 comments:

  1. Its not perfect, but port security applies for situations like this.

    ReplyDelete
  2. Youssef El Fathi04 January, 2012 09:03

    Exact ianh, had the same issue and applied port-security in order to limit the number of hosts per port with storm-control. Not perfect !!!

    ReplyDelete
  3. We have seen this a few times so far. Catalyst 3560 access layer switches, Cisco IP Phones.
    Switchports are configured with a voice vlan, Portfast is enabled with BPDU guard.
    It seems the phone bridges the data and the voice vlan.
    The switches so far were not able to catch it.
    Brings down pretty much everything including the distribution layer switches, pretty nasty.

    ReplyDelete
  4. I use this as well. Limit each port to 4 mac addresses or so and the port will be disabled as soon as there is a loop.

    switchport port-security
    switchport port-security maximum 4
    switchport port-security aging time 10
    switchport port-security violation <protect> (depending on your situation) </protect>

    ReplyDelete
  5. Let's not forget the usefulness of stormcontrol as well.

    ReplyDelete
  6. Curious, but how would this have helped this situation?

    ReplyDelete
  7. Limiting the number of MAC addresses using port security should help in this case...

    ReplyDelete
  8. Something like MAC-based or 802.1x authentication has the nice property of blocking (or at very least, significantly alleviating) loops.

    If you have MAC-based authentication enabled, then a loop causes MAC addresses to appear, and be authenticated, on different ports. Since this process is usually mediated by a radius server, you can do two things - rate-limit it, and watch for well-known MACs like your routers, HSRP/VRRP vMACs and so forth, and "detect" the loop - then deny all authentication (and thus, learning) of new MACs on that switch/port combo for a period of time.

    If you have 802.1x it works even better; the phone supplicant only runs on the "network" facing port. The PC-facing port doesn't have an 802.1x supplicant, so the upstream switch port never goes to "forwarding".

    Both have other benefits (RADIUS-based vlan assignment, proper logging of who is where, accounting, etc.)

    We have observed that loops are much less harmful to the upstream network when being "gated" by mac/802.1x auth. You get local weirdness, but it's rare for it to take the network out, and you can find the location of the weirdness from the radius logs, and fix the problem.

    ReplyDelete
  9. The loop would cause >$maximum MACs to be learnt on a port - thus, the port-security "violation" action kicks in, and breaks the loop.

    ReplyDelete
  10. An "STP-less" switch is no problem - the upstream switch should see its own BPDUs and stop the loop.

    A "BPDU-filtering, STP-less" switch is a catastrophe.

    Rich Seifert addressed this problem in "The All New Switch Book"

    I don't have the citation handy, but it boils down to this: A device that filters BPDUs, and doesn't proactively prevent loops is spectacularly broken. Send them back to the manufacturer.

    ReplyDelete
  11. I just found the following in my outbox, from a conversation about SOHO boxes along these same lines. A relevant Seifert quote is here too.
    -----------------

    Bridges usually fall into one of two categories: Those that support 802.1D, and those that don’t.

    Those that support 802.1D do all of the right things STP-wise, so no problem.

    Those that don't support 802.1D don't generate or understand BPDUs, but they also don't know about the 802.1D requirement to filter frames in the range 01:80:C2:00:00:00-01:80:C2:00:00:0F. This means that they'll pass BPDUs generated by the rest of the network. The SOHO box won't prevent loops on its own, but the rest of the network should detect and correct the problem*

    In 'The All-New Switch Book', Rich Seifert said: "if you're going to violate 802.1D, you have to violate it *all*the*way*"

    ReplyDelete
  12. "Today a user plugged in both of these ports into the switch."

    I think it is more a policy issue than a feature issue. Why would you let your user connect network devices to the switch directly? Shouldn't the switch be secure somewhere?

    ReplyDelete
    Replies
    1. simple! when the user desk place has 2 faceplate ports that led to the internal cabling to the patch panels, and eventually to the switch ports.

      Delete
  13. Brocade FastIron series supports "loop detect" on a port or a VLAN. It sends out probes and watches for those probes to come back. (yes, it sounds like bpdus, but they're not). Depending on how the loop detect is configured, it can err disable the port. This feature was created for just this scenario, where people don't want to (or can't) use spanning-tree.

    ReplyDelete
  14. HP procurve have the same feature. Extreme have it too - though they call it ELRP. I believe both of them work by sending a packet to a multicast MAC periodically and then listening to see if it comes back.

    ReplyDelete
  15. Anton Yurchenko05 January, 2012 18:17

    Hey all,

    I was the person who originally asked the question(thanks for posting Ivan).

    Some background, this happened during an office move, when movers showed initiative and plugged in everything that fits into the jack. Explaining that movers brought down corp network leaves you feeling pretty stupid.

    Thank you all for your suggestions, I think limiting the number of MACs per port and 802.1x look promising. Though 802.1x on IP phones is a pain in the butt.

    ReplyDelete
  16. I agree - as long as you have syslog/SNMP monitoring and you monitor it.

    However, it's important not to use port security auto recovery on it, otherwise the loop will break and restart a while later.

    Restrict will be more permissive, but may make the issue harder to pinpoint/detect. I'd use shutdown, as it will surely make something break.

    ReplyDelete
  17. true, but this is funny how to protect ourself :) or switchself

    ReplyDelete
  18. storm control on the user ports could help to keep the loop under control.

    ReplyDelete
  19. In Extreme, there is loop detection protocol (ELRP) with action (in the later versions).
    Storm control is a must.

    ReplyDelete
  20. Who's the manufacturer of the phone in question, so I can be sure never to buy them?

    ReplyDelete
  21. Yes I´m think so you and i have other question, who long is the cable :-D =-O

    ReplyDelete
  22. Some simple ways to limit damage is by making sure port configuration has the following:

    * storm control set to the lowest bandwidth needed to get the job done.
    * explicit maximum allowed number of source MACs that can be learned over the port (port security feature). This limits looping packets to the configured maximum of source macs. Set it to the lowest number needed to get the job done.
    * unused ports should always be disabled and also have the above two settings "just in case".

    Ideally the IEEE would have standardized a UNI to exchange source MACs a long time ago so such bandaids wouldn't be necessary and so many unnecessary outages avoided.

    ReplyDelete
  23. It sounds like a very simple solution. If the port is not supposed to be receiving BPDUS, then configure the switch to use root protection on those ports. Set bpdu-block-on-edge with a timeout value if you are suing Juniper switches, I should say.

    ReplyDelete
  24. You have indicated some useful port security protection features, but none do not address the question posed. The question seem not to be about a device sending bpdus to the switch, which means it would be able trigger a STP topology change so it would become part of the STP. So it would require bpdu protection so that if the switch gets bpdu on those ports, it would take the configured action.

    ReplyDelete
  25. You have indicated some useful port security protection features, but none do not address the question posed. The question seem not to be about a device sending bpdus to the switch, which means it would be able trigger a STP topology change so it would become part of the STP. So it would require bpdu protection so that if the switch gets bpdu on those ports, it would take the configured action.

    ReplyDelete
  26. enable bpdu protection on those edge ports

    ReplyDelete
  27. Hi all,

    You can try to do this in order to help to mitigate the issue.

    On interface configuration mode:

    storm-control broadcast level 25.00 (this is the percentage of bandwidth that will be the threshold of broadcast, if th limit is exceeded, the action is defined as follows)
    storm-control action shutdown
    switchport port-security maximum 5
    switchport port-security
    switchport port-security aging time 1 (will flush the MAC addresses each minute, it can be adjusted at your convenience)
    switchport port-security aging type inactivity

    In global configuration mode

    errdisable recovery cause psecure-violation
    errdisable recovery interval 300

    Sorry for my english and hope this help.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.