Detect DOS attacks with EEM

Someone sent me an interesting question a while ago: “is it possible to detect DOS flooding with an EEM applet?” Of course it is (assuming the DOS attack results in very high load on the Internet-facing interface) and the best option is the EEM interface event detector.


Interface event detector is just one of the many topics covered in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop or we can organize a dedicated event for your networking team.

The interface event detector is more user-friendly than the SNMP event detector. You can specify interface name and parameter name in the interface event detector; with SNMP event detector you have to specify SNMP object identifier (OID). The interface event detector stores the interface name, measured parameter name and its value in three convenient environment variables that you can use to generate syslog messages or alert the operators via e-mail.

Notes:

  • You must use the bandwidth command to set the interface bandwidth to the actual line speed.
  • Set the bandwidth to the access speed of your Internet service on Ethernet uplinks.
  • The range of the rxload and txload parameters is between 0 and 255.
  • Interface load is computed as 256 * input-or-output-rate / configured-bandwidth.
  • The input-or-output-rate is a weighted average computed over the load-interval.

5 comments:

  1. I am a bit confused. This is a DOS attack detector or a high utilization notifier? The latter is not the same as the former, other than maybe sharing that symptom.

    The DOS attacks that I've seen that kill routers and switches have been a high rate of minimally sized packets. They won't necessarily reach the bandwidth limit of an interface (especially on switch ports).

    Also, in my view, direct DOS attacks are like a serial killer (no purpose except to kill and exceedingly rare). If your router/site gets hit with a direct bandwidth attack you probably have a P.O.'d former employee more than likely. Just sayin.

    Much more common are poorly conceived virus/worms whose goal is to create an army of zombie PC's.The poorly written ones are written to send packets so fast they kill the networking equipment in the middle, which of course suddenly shines a spotlight on them by the network managers. The effective ones remain undetected at layer 3 as they don't stress the network enough to be detected by network tools.

    Cisco routers and switches can easily handle high utilization streams if they are single flows and if you have 100% utilization it doesn't mean anything is wrong.

    That said, EEM is one of those technologies waiting for enough features to finally be a killer app.

    My question on this article, I'll give the benefit of the doubt on detection efficacy, but what I want to see is auto-mitigation. Then you have something useful!

    I don't know EEM *that* well, so could it be done? If so, I'll dive into EEM and write the script.

    ReplyDelete
  2. I'll just focus on the last two paragraphs: if you can define what you want to do as "auto-mitigation", it can almost always be done.

    ReplyDelete
  3. Auto-mitigation for a typical old-school "blaster" worm, for example. It is easily identifiable in show ip accounting and show ip cache flow. Infected PC's, during their probing phase, rapidly send out single packets to a wide range of IP addresses.

    Each packet is handled as a separate flow, and at enough speed this will spike the CPU to 100% and you officially have a DoS. (One PC can easily take down an access layer switch, and sadly I've seen 6500's also fall to this kind of infection.)

    A basic and effective mitigation is to apply a deny input ACL on the closest interface to the infected PC for that PC.

    PS: I don't see this type of infection very often anymore. It was much more common earlier this decade (mainly due to lax anti-virus), but if EEM could do this kind of classification and mitigation, you should be able to make an Anti-DoS for IOS EEM suite. A concept like your original post but with more sophisticated detection.

    The basic idea might be, analyze live netflow data (or ip accounting), count # of flows per source-ip, if # > "too many", apply ACL or policing for that source-ip to the correct interface. The variable values can be debated, but the question is, does EEM have this capability?

    ReplyDelete
  4. Yes, it does, but not with EEM applets. With periodically triggered Tcl scripts (or you could use CPU utilization as the trigger) you can dump the Netflow table, process it (Tcl is a decent language when it comes to data structures) and then change the router/switch configuration.

    While this is doable, it's miles beyond what I'm considering writing about (for free) ;)

    ReplyDelete
  5. I would recommend also monitoring the PPS rate. Many DDOS attacks are actually detected when the PPS rate is high, while BPS rate stays relatively normal.

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.