Blog Posts in March 2008
Tcl-based IOS backdoor
Track the DHCP Default Route
Cisco has published a series of documents describing how you can connect a SOHO site to two ISPs.
Their configuration also includes a nice trick: the ip dhcp client route track number command is a convenient replacement for a static default route with the track option if one of the upstream interfaces uses DHCP and the router generates the default route based on DHCP replies.
NAT activates NBAR
A few days ago I had an “interesting” experience on a router that was running low on memory: when I enabled NAT, it immediately ran out of memory although it had over 4 MB free memory before that (and since I was doing the tests in a lab, I wasn't worried about that … in a production network, 4 MB of free memory is something to worry about).
It took me a while to figure out what was going on: the moment you enable NAT in IOS release 12.4, it activates Network Based Application Recognition (NBAR) even when CEF is disabled (and supposedly NBAR requires CEF to run).
Detect routers operating in process-switching mode
resource policyAnd here are some more ERM usage guidelines:
policy HighProcCPU type iosprocess
system
cpu process
critical rising 40 falling 25
major rising 20 falling 10
!
!
!
user group IPInput type iosprocess
instance "IP Input"
policy HighProcCPU
- This time, we're monitoring a group of processes, so the policy definition is no longer global but has a type (iosprocess is the only type defined at the moment).
- As in the previous ERM example, we're monitoring CPU utilization of the main CPU (system), but this time we're interested in the process utilization.
- The policy is applied to a user group of resources of the type iosprocess (translated into English: a group of IOS processes).
- The only process in this group is the IP Input process (and the "magic keyword" is an instance of the group).
The quotes in the instance configuration command are required, as the command accepts only a single word as the process name.
Predefine your own Tcl functions
If you want to have your own Tcl functions available when you start tclsh, you could use the scripting tcl init file configuration command that I've briefly mentioned in one of the previous posts. This command specifies a source file that is executed every time you start Tcl shell. The source file can contain function definitions, package declarations or any other Tcl code.
If you need to, you can specify multiple initialization files.
For example, if you'd like to implement a comfortable Tcl-based pinger (similar to the one Ethan Banks found in the Sadikhov forums, store the following Tcl code into the file flash:pinger.tcl …
proc pinger { iplist } {
foreach ip $iplist {
if { [regexp "(!!!)" [exec "ping $ip timeout 1" ]] } {
puts "$ip"
} else { puts "$ip **** failed ***" }
}
}… and configure scripting tcl init flash:pinger.tcl. Now you can ping a number of hosts in a single operation:R1#tclsh
R1(tcl)#pinger { 10.0.0.1 10.0.0.2 10.0.0.3 10.0.0.4 }
10.0.0.1
10.0.0.2
10.0.0.3 **** failed ***
10.0.0.4 **** failed ***
Local-AS Has to Be Matched by Incoming Filter-List
In a previous post I've described how you can use neighbor local-as feature to fix AS-number mismatch between adjacent autonomous systems. However, without additional options, the local-as is inserted in the AS-path of incoming BGP updates before any inbound filters. Your inbound filters thus have to match the local-as as well.
When “copy” actually means “merge”
Marcus Jensen asked me a very interesting question:
I want to send 3 lines of configuration to a remote router, but I know the first line will kill my connection. Can I save these 3 lines of code to a text file, and then issue a Tcl command to add those to the running config?
The solution is much simpler and does not have to involve Tcl at all. The copy something system:running-config command merges the configuration commands in the source file with the current running configuration.
You can store the configuration commands you want to execute in a local file (even in NVRAM) or you could execute them directly off a file server (using HTTP, FTP, TFTP or SCP protocol).
This article is part of You've asked for it series.
SNTP will not work if you've configured NTP
SNTP multicast/broadcast client mode works in combination with NTP
NTP process could be running even if your running configuration has no NTP-related commands. It starts automatically whenever you enter NTP-related configuration (ntp logging configuration command is enough) and is not stopped when the last NTP-related configuration command is removed. You have to reload the router to kill it.
Use EEM to respond to ERM events
However, even EEM applet could solve some immediate problems. For example, if you want to store a snapshot of processes on a TFTP server every time the global CPU load crosses a policy threshold, you could use the following applet:
event manager applet ReportHighCPU
event resource policy "HighGlobalCPU"
action 1.0 cli command "show process cpu sorted 5sec | redirect tftp://10.0.0.10/highCPU$_resource_time_sent.txt"
To differentiate the snapshots, I've appended the _resource_time_sent variable set by the EEM before the applet is started to the file name, guaranteeing that the snapshot files will have unique names (at least until the router reload).
As an alternative, you could send the show process output in an e-mail:event manager environment _ifDown_rcpt [email protected]
!
event manager applet ReportHighCPU
event resource policy "HighGlobalCPU"
action 1.0 cli command "show process cpu sorted 5sec"
action 1.1 info type routername
action 2.0 mail server "mail-gw" →
to "$_ifDown_rcpt" from "[email protected]" →
subject "CPU @ $_resource_current_value" →
body "$_cli_result"
This article is part of You've asked for it series.
How Do I Detect Router Restarts?
Mike Nipp has wondered which syslog message to use to reliably detect router reload under all circumstances:
The problem I had with the SYS-5-RESTART message is I don't think you will get one if the power is suddenly pulled from the router. It does do a SNMP-5-COLDSTART and SYS-6-BOOTTIME on boot up.
I did an actual power-cycle test of a router and the SYS-5-RESTART message is reliably generated at every startup, be it from the power cycle or the reload command (I was not able to provoke an on-demand crash ;).
The Mysteries of the “Internet” BGP Community
Cisco documentation has always claimed there were four well-known communities (the Internet community being one of them), while the RFC 1997 lists three well-known values. Unfortunately, many people blindly copy the IOS documentation without asking themselves “what the heck is the Internet community”.
Detect CPU spikes with Embedded Resource Manager
The ERM syntax is a bit baroque (and not well documented), so let's work through the example: this is the configuration you need to detect high overall CPU utilization on the main CPU in the box:
resource policy
policy HighGlobalCPU global
system
cpu total
critical rising 95 falling 70 interval 10
major rising 75 falling 50 interval 10
!
user global HighGlobalCPU
And here are the usage/configuration guidelines:
- The whole ERM subsystem is configured under the resource policy section;
- You always have to configure a policy and a user to which the policy applies. In our example, the user is global (as we're measuring the global CPU load);
- The policy we're defining must have the global keyword to indicate we're measuring overall utilization (otherwise you can't attach it to the global user);
- We're measuring the load on the main CPU, so we're configuring the system subsection of the policy (on distributed platforms you could specify slot name to measure utilization on a specific linecard);
- The cpu section selects CPU load measurements. You could measure interrupt load, process load or total CPU load.
- Within each resource section in the policy (in our example, total CPU load on the main system) you can define minor, major and critical thresholds (syslog messages are generated when each threshold is crossed).
- After the policy is defined, it's applied to the global user.
With the CPU load measurement policy defined, the router will generate syslog messages (SYS-4-CPURESRISING) every time the overall CPU load exceeds the specified rising thresholds. When the utilization falls below the falling threshold, the SYS-4-CPURESFALLING syslog message is generated.
This article is part of You've asked for it series.
Display the names of the configured route-maps
I’m probably getting old … I keep forgetting the exact names (and capitalization) of route-maps I’ve configured on the router. The show route-maps command is way too verbose when I’m simply looking for the exact name of the route-map I want to use, so I wrote a Tcl script that displays the names of the route-maps configured on the router. If you add a -d switch, it also displays their descriptions (to be more precise, the first description configured in the route-map).
Copy file to an FTP server with EEM applet
event manager applet backup-crl
event timer watchdog time 86400 maxrun 4294967295
action 1.0 cli command "enable"
action 2.0 cli command "copy flash:/iosca.crl ftp://username:[email protected]/" pattern "a.b.c.d"
action 3.0 cli command "a.b.c.d" pattern "iosca.crl"
action 4.0 cli command "iosca.crl"
action 5.0 syslog msg "FTP backup successful"
Use extended access-lists to filter BGP updates
When I've included a few slides on this feature in the first BGP course I've developed for Cisco (that was probably somewhere around 1994), the results in the class were always the same: total confusion that needed an hour of whiteboard examples to dissolve. You can find a few examples that will help you understand this arcane feature in a post written by Brian Dennis.
The use of extended IP ACL as a route matching mechanism was made obsolete by the ip prefix-list command, which was introduced in 12.0T. As 12.0T reached End-of-Engineering in the previous millennium, it's a safe bet that the only place where you might still be required to use extended ACLs to match IP routes is in the CCIE lab.
Router Fragmentation Is Gone from IPv6
In response to my Never-Ending Story of IP Fragmentation, Stojanco Cavdarov made an interesting observation: routers are not allowed to fragment IPv6 packets, they have to respond back with ICMP unreachable (effectively, routers behave as if IPv6 packets would have an implicit don't fragment bit).
To make life easier for non-TCP IPv6 applications (TCP is supposed to use Path MTU Discovery), the minimum IPv6 packet size that has to be supported on all links was increased to 1280 bytes (which, incidentally, fits very nicely into GRE+IPSec envelope transported across links with 1500-byte MTU).
WAN IP addresses and subnet masks
“What I would like to know is, on my PPP negotiated ADSL connection, how the ISP assigns me a /32 ip address.”… which prompted me to test various WAN encapsulations and address assignment rules. Here are the results:
- On all WAN encapsulations you can configure subnet masks down to /31 (/30 in old IOS releases).
- The same IP address can be used on more than one interface as long as both IP address and subnet mask match.
- Two WAN interfaces can have different IP addresses but still belong to the same IP subnet. You would use this on Frame Relay when you have multiple interfaces into the same FR cloud for bandwidth reasons.
- If you configure IP address with IPCP (with the ip address negotiated command), the subnet mask becomes /32 as IPCP does not carry subnet mask (and you get the host route toward the PPP peer unless you turn off the PPP peer route option)
- If you configure IP address with SLARP (Serial Line ARP) on HDLC, the subnet mask is inherited from the peer (HDLC SLARP carries subnet mask) and the IP address is determined by flipping the low-order bits in the neighbor's IP address.
Phase 2: Upload text files through a Telnet session
The trick works flawlessly, but typing the same obscure Tcl commands gets tedious after a while, so the first time I had to use this solution to develop a Tcl script, I've quickly written another script that takes file name as the parameter and hides all the other murky details.
To use it, transfer the contents of storeFile.tcl (available from my web site) to the router's flash (using the previously described trick), follow the installation instructions in the source and you're ready to go.
Note: You can adapt the Tcl script to your needs; for example, you could add instructions to re-register EEM Tcl policy every time you upload the new code.
Debugging time-based configuration
alias exec 859 clock set 08:59:30
alias exec 900 clock set 09:00:30
Obviously, these tests are best done in a lab setup … and you have to turn off NTP or any other form of time synchronization.