Category: network management

Cisco IOS release 12.4(15)T brought (among a plethora of voice features) the logging to non-volatile storage, a nice-sounding name for the ability to write syslog messages into files on your flash memory (or an embedded disk, if you have one). To configure it, use the logging persistent [url directory] [size filesystem-size] [filesize logging-file-size] global configuration command:

• The directory argument specifies where you want the files to be stored (for example, flash:/logging).
• The filesystem-size specifies the maximum disk space the logging files can consume (once you exceed the limit, the oldest file is deleted)
• The logging-file-size parameter specifies the maximum size of each file (once the file grows too large, a new file is created).

Note: You can store the log files on the router's flash memory if it appears as a disk file system (check with the show file systems command). Wouldn't it be great if this feature would also work on USB drives ...

Contrary to what the regular reload does, the warm reload does not change the configuration register value (obviously that's done by ROMMON, which is not involved in the warm reload process). If you just did a password recovery and changed the configuration register back to a normal value, you'd thus be unpleasantly surprised when the NVRAM would be ignored (yet again) after a warm reload (I stumbled across this as I was trying a new IOS release with the reload warm file URL command).

Pete Vickers sent me a very interesting configuration sample:

To get an IOS device to upload it’s configuration periodically to an external FTP server:

ip ftp source-interface loopback 0
ip ftp username ftp_username
ip ftp password ftp_password
file prompt quiet
!
kron policy-list backup
 cli copy running-config ftp://10.20.30.40
!
kron occurrence daily-backup at 0:30 recurring
 policy-list backup

The beauty of this example is that you can use it on platforms that don't support Embedded Event Manager (which has a very similar cron functionality) as the kron commands were introduced in 12.2T and 12.3 IOS releases.

Note: You have to use the file prompt quiet configuration command as the commands executed by kron cannot supply any user input

One of the hardest troubleshooting problems within an MPLS VPN network has always been finding a broken LSP. While you could (in theory) use the IP ping or traceroute (assuming all hops support ICMP extensions for MPLS), the results are not always reliable… and interpreting them is not so easy. For example, after I've disabled LDP on an interface with the no mpls ip configuration command, the routers in the LSP path still reported outgoing MPLS labels in ICMP replies for a few seconds (until the LDP holddown timer expired on both ends of the link).

As a side note, would you deduce from the printout that the break in the LSP path happened on the router with the IP address 192.168.201.1?

The IOS documentation claims that the quiet mode the router enters after a series of login failures blocks all telnet (or ssh) sessions as well as HTTP requests. Unfortunately the latter is wrong; you can execute any HTTP request on the router during the quiet mode.

If you want to block HTTP requests during the quiet mode, you can use EEM applets to change the HTTP server configuration when the quiet mode is started and completed.
First you need to configure a standard numbered IP access list that will be used to block HTTP requests during the quiet mode (the ip http access-class command accepts only numbered ACLs), for example:

access-list 95 deny any log

Then you define two EEM applets: one that triggers when the router enters the quiet mode (matching the SEC_LOGIN-1-QUIET_MODE_ON syslog message) and another that runs when the quiet mode is finished (triggered with the SEC_LOGIN-5-QUIET_MODE_OFF). Both applets modify the router configuration, changing the access-list used in ip http access-class configuration command.

event manager applet EnterQuietMode
 event syslog occurs 1 pattern "SEC_LOGIN-1-QUIET_MODE_ON" period 1
 action 1.0 cli command "configure terminal" 
 action 1.1 cli command "ip http access-class 95" 
 action 2.0 syslog msg "Entered Quiet mode on HTTP server"
!
event manager applet ExitQuietMode
 event syslog occurs 1 pattern "SEC_LOGIN-5-QUIET_MODE_OFF" period 1
 action 1.0 cli command "configure terminal"
 action 1.1 cli command "ip http access-class 70"
 action 2.0 syslog msg "Exiting Quiet mode on HTTP server"

After you've configured the Warm Reload, you can also perform warm IOS upgrade/downgrade (assuming that you already run at least the IOS release 12.3(11)T or 12.4). The Warm Upgrade functionality loads the new IOS image into the main memory, decompresses it and starts it, significantly reducing the downtime (in my case, a 2800 router reloaded in 62 seconds as compared to 415 seconds it took to load the image from a locally-attached server).

Apart from the downtime reduction, the warm upgrade (requested with the reload warm file url command) has a number of other benefits:

• The new image does not have to be stored in flash
• You don't have to change the boot image with the boot system command
• If the new image crashes, the router will revert to the original IOS image stored in flash

In a previous post, I've described how you could generate a (almost) continuous ping from a router using the extended ping command. While that approach is extremely simple to implement, it ties up a line (and if you're working from the console, it's highly impractical). You could get the same results (as Tom has already pointed out) using IP SLA feature of Cisco IOS. Configure the ping request with commands similar to these:

ip sla 100
  icmp-echo 172.16.1.2
  timeout 500
  frequency 3

To start the ping, use the ip sla schedule 100 life forever start-time now configuration command, to stop it, no ip sla schedule 100 command. When using IP SLA ping, you can trigger Embedded Event Manager applets to report ping failure (similar to the technique described in this post) or read the ping results from SLA MIB with SNMP.

Note: In IOS release 12.3(14)T, the ip sla command replaced the rtr command. To use this technique in IOS release 12.3 or lower, use the rtr configuration command.

IOS release 12.2(13)T (integrated in IOS release 12.3) has added the capability to redirect output of an IOS show command to a file. This feature uses Unix-style pipes (similar to the include, exclude and section keywords) and adds append, redirect and tee (redirect + print) keywords.

The show output can be redirected to a local filename (in flash, on usb token or even in NVRAM) or sent to a remote server (currently only FTP and TFTP servers are supported). For example, the show ip interface brief | redirect ftp://student:[email protected]/ifstatus command will store the current interface status to an FTP server.

Note: the append (or tee /append) operation only works on destinations that support the file append operation: class-C flash file systems, local disks, USB tokens and NVRAM.

I've almost started writing a Tcl procedure to display top-10 CPU-intensive processes on a router ... and then discovered the sorted option of the show processes cpu command. Even more, starting in IOS release 12.2T, the show processes cpu history command gives you a nice CPU utilization graph.

The Warm Reload functionality introduced in IOS release 12.3(2)T significantly reduces the reload time. In my test lab, the reload time of a Cisco 2800 router booting from flash was reduced from 135 to 54 seconds as measured by the %SYS-6-BOOTTIME: Time taken to reboot after reload ... syslog message.

The theory behind warm reload is simple: the router saves initial data (as stored in IOS image) in a separate memory region and reuses saved data together with IOS code already residing in RAM to restart IOS. Of course, the IOS code (depending on platform's memory management capabilities) or saved data could get corrupted, therefore the warm reload cannot be used continuously (and the router falls back to traditional reload if the router crashes before a specified time interval).

Warm reload is configured with the warm-reboot count number uptime minutes configuration commands. After it has been configured, a router reload (or power-up) is needed to initialize the saved data region. When the warm reboot is operational (as verified with the show warm-reboot command), you can use reload warm command to start it.

One of my readers has asked an interesting question: can you reload a router when pinging a specific IP address from it fails? While there are other ways of dealing with stuck interfaces or routing processes, sometimes such a drastic measure is the only workaround, so here's how you do it:

Displaying configuration of a single interface can be a time-consuming task if your router has extremely long configuration (for example, high-end device with hundreds of interfaces, route-maps, access-lists etc.). In this case, the interface keyword of the show running-config command becomes extremely useful.

In certificate-based IPSec deployments, the router has to establish an approximately valid date and time before it can use a certificate to establish IPSec session (as most certificates were issued after March 1st 2002, which is the default initial value, they are not valid until the router has acquired an approximately correct date-and-time).

This requirement is not a problem for most router models, as they have battery-backed hardware clock that continues running even when a router is reloaded or powered down. The low-end models, though, have a problem, as they always start with the default date/time after the reload. These devices have to get their time from an NTP/SNTP server before being able to establish the IPSec session. If the (S)NTP server is only accessible across the VPN, you have a nice chicken-and-egg problem.

With IOS file system (IFS) introduced in IOS release 11.3AA (integrated in 12.0), IOS got the more command that displays any local or remote file that you could specify with IFS. The obvious use of the more command would be display of startup configuration (more nvram:startup-config), but you could also display built-in Tcl EEM policies (for example, more system:lib/tcl/http.tcl) or remote router configurations (for example, more tftp://host/cfg-file). But that's not all, you could even troubleshoot web servers and display HTML generated by the web server (for example, more http://192.168.0.2/index.html).

Note: IOS documentation claims that the show running command is obsolete and that you should use more system:running-config. This is not true, as the show running command has a number of interesting options that are not implemented with the more command.