Category: network management

After the initial explosion of OpenFlow/SDN hype, a number of people made claims that OpenFlow is not the tool one can use to make SDN work, and NETCONF is commonly mentioned as an alternative (not surprisingly, considering that both Cisco IOS and Junos support it). Unfortunately, considering today’s state of NETCONF, nothing can be further from the truth.

Recently-published RFC 5837 describes additions to ICMP messages that would allow you to gather more information (including interface ifIndex, IP address and name). Two obvious applications are enhanced traceroute and path MTU discovery where the new ICMP extensions could indicate which interface is the MTU bottleneck.

The RFC authors come from BT, Juniper and Cisco, so there’s a non-zero chance it will actually get implemented where it’s most needed.

Every now and then, a really interesting question appears on the cisco-nsp mailing list. A while ago I’ve seen this one:

I’ve heard that Cisco devices handle ICMP at a low priority. I found one post describing it handled in process-switching and not fast-switching. Does anyone have an article that explains that process and is it configurable?

Most packets sent to the router are handled in process switching (the packet is queued in the input queue of one of the IOS processes), the obvious exceptions being GRE and IPSec packets (unless they’re fragmented).

Packets sent to the router can also be rate-limited with a control plane policy.

The IOS processes perform their job between interrupts (packets being CEF- or fast switched). A reply to an ICMP packet is therefore a lower-priority task than regular packet forwarding.

read more see 8 comments

It all started with a innocuous question: can you detect voice traffic with EEM? Looks simple enough: create a QoS class-map that matches voice calls and read the cbQosClassMapStats table in the CISCO-CLASS-BASED-QOS-MIB. The first obstacle was finding the correct indexes, but a Tcl script quickly solved that; I was ready to create the EEM applet. The applet failed to work correctly and after lots of debugging I figured out the counters in the cbQosClassMapStats table change only every 10 seconds.

I couldn’t believe my eyes and simply had to test other MIB variables as well. As expected, the IF-MIB (standard interface MIB) counters increase in real-time, but obviously someone had the bright idea that we need to detect changes in traffic profile only every now and then. Although I've received numerous suggestions from my readers, none of them works on a Cisco 1800 or a Cisco 7200. Oh, well, Cisco developers from the days when I started working with routers would have known better…

Jared Valentine sent me a really interesting problem: he would like to detect voice traffic and start shaping TCP traffic for the duration of the voice call. The ideal solution would be an EEM applet reacting to the changes in the CISCO-CLASS-BASED-QOS-MIB; one of its tables contains the amount of traffic for each class configured in a service policy.

The MIB navigation looks simple: you just read the values from the cbQosClassMapStats table, indexed by policy ID and class ID. The real problem is finding the correct index values. I could walk the MIB manually with a MIB browser or snmp_getnext TCL calls, but this approach is obviously not scalable, so I wrote a script that walks through the cbQosServicePolicy, cbQosObjects, cbQosPolicyMapCfg and cbQosClassMapCfg tables and prints the index values you need.

This script traverses the Class-based QoS MIB and displays service policies and classes attached to individual interfaces. The policy index and class index values are printed next to the policy/class name to help the operator fetch the desired SNMP variable from the statistics tables of the CISCO-CLASS-BASED-QOS-MIB.

Installation

• Download the source file into flash:cbindex.tcl
• Configure alias exec cbindex tclsh flash:cbindex.tcl
• Configure persistent CBQoS indexes with the snmp mib persist cbqos (otherwise the indexes will change after the router reload).

Usage guidelines

Usage: cbindex community

Command line parameters:

• Community: SNMP community with R/O access to the CISCO-CLASS-BASED-QOS-MIB

Source code

#
# title:    Displays MQC class map indexes
# name:     cbindex.tcl
# desc:     The script traverses the Class-based QoS MIB and
#           displays service policies and classes attached to 
#           individual interfaces. The policy index and class
#           index values are printed next to the policy/class
#           name to help the operator fetch the desired SNMP 
#           variable from the statistics tables of the 
#           CISCO-CLASS-BASED-QOS-MIB.
#

proc snmpInit { oid } {
  global snmpCommunity
  set getResult [ snmp_getnext $snmpCommunity $oid ]
  if { [ regexp {snmp error} $getResult ] } { 
    puts "SNMP calls with community $snmpCommunity fail"; return 0 
  }
  if { [ regexp {oid='(.*)'} $getResult ignore nxtoid ] } {
    if { [string first $oid $nxtoid] == 0 } { return 1 }
  }
  puts "MIB $oid not implemented in this IOS release"; return 0;
}
  
proc snmpGet { oid result } {
  global snmpCommunity
  upvar $result r
  if { [info exists r] } { unset r }

  set getResult [ snmp_getone $snmpCommunity $oid ]
  if { [ regexp {snmp error.*text='(.*)'} $getResult ignore errtxt ] } { 
    error "snmpGet - $errtxt"; return 0 
  }
  if { [ regexp {oid='(.*)'.*val='(.*)'} $getResult ignore oid result ] } {
    if { ! [ string equal $result "NO_SUCH_INSTANCE_EXCEPTION" ] } {
      set r(OID) $oid ;
      set r(VALUE) $result ; 
      return 1;
    }
  }
  return 0;
}

proc snmpGetNext { oid result } {
  global snmpCommunity
  upvar $result r
  if { [info exists r] } { unset r }

  set getResult [ snmp_getnext $snmpCommunity $oid ]
  if { [ regexp {snmp error.*text='(.*)'} $getResult ignore errtxt ] } { 
    error "snmpGet - $errtxt"; return 0 
  }
  if { [ regexp {oid='(.*)'.*val='(.*)'} $getResult ignore oid result ] } {
    if { ! [ string equal $result "NO_SUCH_INSTANCE_EXCEPTION" ] } {
      set r(OID) $oid ;
      set r(VALUE) $result ;
      set oidSplit [ split $oid "." ]
      set r(NAME)  [ lindex $oidSplit 0 ]
      set r(INDEX) [ lreplace $oidSplit 0 0 ]
      set r(IDXLIST) [ join $r(INDEX) "." ]
      return 1;
    }
  }
  return 0;
}

proc snmpGetInTable { oid result { parentoid "" }} {
  global snmpCommunity
  upvar $result r

  snmpGetNext $oid r
  if { ! [info exists r(OID)] } { return 0 }
  if { [string equal $parentoid ""] } {
    set oidSplit [ split $oid "." ]
    set parentoid [lindex $oidSplit 0]
  }
  if { [string first $parentoid $r(OID)] != 0 } { return 0 }
  return 1;
}

proc printQosClassIndex {} {
  global snmpCommunity
  set oid "cbQosIfIndex"
  array set dirLookup { 1 in 2 out }
  set cnt 0
  while { [ snmpGetInTable $oid svcPolicy ] } {
    if { [snmpGet "ifDescr.$svcPolicy(VALUE)" ifDescr] } {
      snmpGet "cbQosPolicyDirection.$svcPolicy(INDEX)" svcDirection
      snmpGetNext "cbQosConfigIndex.$svcPolicy(INDEX)" policyObject
      snmpGet "cbQosPolicyMapName.$policyObject(VALUE)" policyName
      puts "\n$ifDescr(VALUE) ($dirLookup($svcDirection(VALUE))): $policyName(VALUE) ($svcPolicy(INDEX))"
      set coid "cbQosObjectsType.$svcPolicy(INDEX)"
      set parentoid $coid
      while { [ snmpGetInTable $coid svcClass $parentoid ] } {
        if { $svcClass(VALUE) == 2 } {
          snmpGet "cbQosConfigIndex.$svcClass(IDXLIST)" svcClassConfig
          snmpGet "cbQosCMName.$svcClassConfig(VALUE)" svcClassName
          puts "  $svcClassName(VALUE) $svcClass(IDXLIST)"
        }
        set coid $svcClass(OID)
      }
    } else { error "Cannot get interface name for service policy $svcPolicy(VALUE)" }
    set oid $svcPolicy(OID)
  }
}

set snmpCommunity [lindex $argv 0]
if { [string equal $snmpCommunity ""] } { set snmpCommunity "public" }
if { ! [ snmpInit "cbQosObjectsType" ] } return
printQosClassIndex

Sample usage scenario

The following QoS classes and policies have been configured on the router:

class-map match-all Mail
 match protocol smtp
!
class-map match-all Web
 match protocol http
!
class-map match-all SecureWeb
 match protocol secure-http
!
class-map match-any Surfing
 match class-map Web
 match class-map SecureWeb
!
class-map match-all Files
 match protocol ftp
!
policy-map Internet
 class Web
    bandwidth 128
 class SecureWeb
    priority 64
 class Mail
    bandwidth 32
!
policy-map MailOrFtp
 class Mail
  set ip precedence 0
 class Files
  set ip precedence 0
 class Surfing
    police 16000
 class class-default
   police cir 8000
     exceed-action drop 
!
interface Serial1/0
 service-policy input MailOrFtp
 service-policy output Internet
!
interface Serial1/1
 service-policy output MailOrFtp

The cbindex script reported the following SNMP indexes:

c7200#cbindex Test

Serial1/0 (in): MailOrFtp (48)
  Web 48.383777
  Surfing 48.1970017
  Mail 48.4297921
  Files 48.13110129
  class-default 48.14779377
  SecureWeb 48.15077857

Serial1/0 (out): Internet (50)
  Mail 50.10516033
  Web 50.14007809
  SecureWeb 50.14520625
  class-default 50.15008753

Serial1/1 (out): MailOrFtp (66)
  Web 66.383777
  Surfing 66.1584993
  Files 66.4236097
  Mail 66.11615889
  SecureWeb 66.15077857
  class-default 66.15082481

Based on these indexes, you could monitor the bit rate of the Web class in outbound policy configured on Serial 1/1 with SNMP variable cbQosCMPrePolicyBitRate.66.383777.

c7200#tclsh
c7200(tcl)#snmp_getone Test cbQosCMPrePolicyBitRate.66.383777
{<obj oid='cbQosCMPrePolicyBitRate.66.383777' val='0'/>}

Dear readers! This time I really need your help (uncle Google and his relatives gave me only one relevant hit and even that doesn't work on 7200 or 1800).

I'm trying to implement an EEM applet that would detect traffic rate change using CISCO-CLASS-BASED-QOS-MIB. Everything would work perfectly ... if only IOS wouldn't update the MIB counters approximately every 10 seconds, not in real-time. Is anyone aware of a configuration command that would force the router to update these counters any faster?

Someone recently asked me how to get the physical location of an IP address. One of the better (free) services available on the Internet is the IP2Location (demo) service.

This feature might come handy if you're trying to figure out who's attacking your application servers (when the TCP session has already been established). Denial-of-service attacks commonly use fake source IP addresses.

read more see 16 comments

Ralf sent me a SNMPv3 question:

If I create a SNMPv3 user which has a password (snmp-server user userthree groupthree v3 auth md5 user3passwd), this user does not appear in the running- or startup-config. Cisco even documents this if you know what to look for.

I strongly suspect (although I did not test this) that these users are also missing from configuration exported to TFTP servers. What would be the recommended way to make usable config backups of routers with such users?

Like certificates, the SNMPv3 users are stored in private-config and thus never appear in the router configuration. If you want to have a backup of the user data, create a text file on one of your NMS servers, add SNMPv3 usernames and passwords in the text file and use the copy somewhere running-config to configure SNMPv3 users on the routers.

When I was trying to test how the router running IOS release 12.4(15)T5 classifies inbound IP packets into various CPPr subinterfaces, I wanted to use the log action in the MQC classes I've defined. This approach worked perfectly for the host and cef-exception interface (I've even seen ARP packets logged), but the packets classified as transit generated no log messages. While this makes perfect sense (after all, all punted packets are processed by the transit service-policy), the IOS should generate a warning when you apply a policy-map with the log option as service-policy on the control-plane transit interface.