Blog Posts in May 2007

Catching all syslog messages

If you use UDP-based syslog servers, you might have noticed that they miss a message or two generated during a router reload (particularly when the syslog server is on a directly connected LAN). The reason is simple - when the first message is sent to the syslog server, its MAC address is not yet in the router's ARP cache and the message is dropped. To prevent the message loss, you can use the logging server-arp configuration command (introduced in IOS release 12.3T), making sure that the router sends ARP request to the configured syslog server(s) before generating the first syslog messages.

Note: this issue does not occur when using TCP as the syslog transport mechanism.
add comment

The DNS configuration "challenges"

With the introduction of DNS views in IOS release 12.4(9)T, a number of additional DNS-related configuration commands were introduced. As IOS still supports all the older configuration commands (and the DHCP-acquired DNS servers), the results are not as obvious as one would hope. The IOS documentation is pretty explicit (a nice surprise :), but I still had a bit of a headache figuring it all out, so you might be in the same position.

Let's start with the DNS resolvers (the DNS name servers the router itself uses when it needs to change a name into an IP address or vice versa):
  • You can configure the DNS resolvers with the ip name-server global configuration command or the domain name-server configuration command within the ip dns view default.
  • Both lists are merged and combined with the IP addresses acquired from the DHCP reply messages to get the final list of the DNS resolvers. You can inspect the final list with the show ip dns view default command.
  • The DNS name servers from the DHCP replies never appear in the router configuration.
  • If the same IP address is specified with the ip name-server and domain name-server command, only the global (ip name-server) command will appear in the router configuration.

The rules for DNS forwarders (the DNS servers the router uses to answer incoming DNS queries) are a bit different:

  • The DNS view the current DNS query should use is determined based on view-list assigned to incoming interface (with the ip dns view-group interface configuration command) or the global view-list (specified with the ip dns server view-group global configuration command).
  • The incoming query is (sequentially) sent to the IP addresses configured as dns forwarders in the selected view. No other DNS servers are used.
  • If the selected view has no dns forwarders, but contains domain name-servers, they are used as forwarders.
  • If the selected view has no dns forwarders or domain name-servers, the query is forwarded as IP broadcast ... unless the selected view is the default view, in which case the first DNS server computed according to the the previous bullet list is used (only one DNS server is used in this case, even if you have configured multiple DNS servers with the ip name-server configuration command).
see 6 comments

Warm upgrade

After you've configured the Warm Reload, you can also perform warm IOS upgrade/downgrade (assuming that you already run at least the IOS release 12.3(11)T or 12.4). The Warm Upgrade functionality loads the new IOS image into the main memory, decompresses it and starts it, significantly reducing the downtime (in my case, a 2800 router reloaded in 62 seconds as compared to 415 seconds it took to load the image from a locally-attached server).

Apart from the downtime reduction, the warm upgrade (requested with the reload warm file url command) has a number of other benefits:
  • The new image does not have to be stored in flash
  • You don't have to change the boot image with the boot system command
  • If the new image crashes, the router will revert to the original IOS image stored in flash
read more see 1 comments

Reload EEM Tcl policy with help of Tcl shell

Testing Embedded Event Manager (EEM) Tcl policies is a convoluted process:
  • Source file is usually edited on a general-purpose workstation.
  • The file has to be downloaded to router's local storage (EEM does not register non-local policies).
  • The new version of the EEM policy has to be registered with EEM with event manager policy configuration command
  • After all these steps, the new policy can be tested.
While you can use EEM applet to automate this process, slightly more flexible approach (you can specify the policy name to be replaced) can be implemented with Tcl script:
set policy [lindex $argv 0]
set source "tftp://10.0.0.10/tcl/" # replace with your host and directory
set destination "nvram:" # replace with local storage device
if {[string equal $policy ""]} {
return -code error "expected policy name"
}
puts "replacing policy: $policy"
ios_config "file prompt quiet"
ios_config "no event manager policy $policy" ""
exec "copy $source$policy $destination$policy"
ios_config "event manager policy $policy"
ios_config "no file prompt quiet"
To use the script, follow these steps:
  • Save the script in a .tcl file (for example, changePolicy.tcl)
  • Change the script parameters (remote host and local storage)
  • Save the .tcl file to your router's local storage (you can also run it from a remote server)
  • Configure a command alias, for example alias exec eem tclsh flash:changePolicy.tcl testPolicy.tcl
Now you can replace the target EEM Tcl policy with a simple eem command.

Alternatively, if you define alias exec eem tclsh flash:changePolicy.tcl, you can specify policy name as an argument to the eem command, for example eem testPolicy.tcl.
see 1 comments

Unbundle DNS settings from DHCP client

In one of my previous posts I've been writing about the problems I had when the DHCP client on Cisco IOS was messing up the DNS name-servers I've configured manually with the ip name-server configuration command. As is quite usual in Cisco IOS, there's one more know to turn to fix this - the Configurable DHCP Client feature introduced in IOS release 12.3(8)T.

To stop the router's DHCP client from overwriting the static name-server settings, use the no ip dhcp client request dns-nameserver interface configuration command (you can also exclude a few other DHCP options).
add comment

Change the Telnet font color from a Cisco router

I've recently got an excellent question: how can I change the color of the terminal emulator font from the CLI by sending the telnet program an escape sequence?

For those of you that haven't worked with escape sequences before - you can control a lot of parameters in you terminal emulation program by sending it a special sequence starting with <ESC>[ (<ESC> begin character code 27). These sequences work even in the simplest telnet clients on Windows and Linux thanks to built-in operating system support or ANSI.SYS driver (on Windows); you can get an in-depth description and the list of all supported escape sequences from Wikipedia.

With the list of escape sequences in hand, the only remaining problem is how to send them from the router. Tcl shell was the immediate answer, but then I remembered the various banners provided by the Cisco IOS. As I wanted to change the font color as soon as possible, the login banner (or the message-of-the-day banner) seemed the right one to use.

To insert the ESC character in a banner from the configuration prompt, you have to use the Ctrl-V/Esc sequence. For example, to change the font color to bright yellow, do the following:
router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
router(config)#banner motd #
Enter TEXT message. End with the character '#'.
<Ctrl-V><ESC>[33;1mThis is a yellow router#
router(config)#
When you look at the running configuration, the router transforms the control characters into ^x sequences, so your banner will look like this:
router#show run | begin banner
banner motd ^C
^[[33;1mThis is a yellow router^C
...
Notes:
  • Saving router configurations on (T)FTP server and reloading them from a server does not pose a problem, as the control characters are preserved throughout the process, but you cannot use cut-and-paste with banners, as the terminal emulator does not transform ^x sequence into a control character.
  • With this technique, you can color-code your routers. For example, internal routers could use green font, publicly accessible ones red font and the firewall routers yellow font, so all the operators would know immediately what to expect on each device.
  • This article is part of You've asked for it series.
see 6 comments

DNS resolver in Cisco IOS is auto-configured with parameters from a DHCP reply

If you're using DHCP to get IP interface addresses on your router (using the ip address dhcp interface configuration command), the router will also inherit the DNS resolver settings included in the DHCP reply. Makes sense, but the implementation is "a bit" unexpected: if you configure the DNS name servers manually with the ip name-server address-list command, the ones matching the values in the DHCP reply packet are not included in the running configuration and thus not saved to NVRAM. Even worse, the statically-configured name-servers overwritten by a DHCP reply are lost if the DHCP-configured interface goes down.

To avoid total confusion, you thus have these options:
  • Do not use DHCP to acquire IP interface addresses
  • Make sure the DHCP server does not send DNS-related parameters (a bit hard if you're using DHCP with your ISP)
  • Rely exclusively on DHCP to provide your router with the DNS name server addresses
read more see 10 comments

Periodic execution of IOS show commands

If you want to execute IOS show commands periodically (for example, to monitor router status or take snapshots of routing tables), you can combine new output redirection features introduced in IOS release 12.2T in an Embedded Event Manager (EEM) applet. For example, to store the brief interface status into a file on an FTP server, use the following EEM applet:
event manager applet SaveInterfaceStatus
 event timer watchdog name SaveIfStat time 60
 action 1.0 cli command "show ip interface brief | redirect ftp://username@password:host/path"
 action 2.0 syslog msg "Interface status saved"
Notes:
  • The timer watchdog EEM event defines a recurring event triggered every X seconds.
  • Output of a show command can be redirected only to a TFTP or FTP server, redirection to a web (HTTP) server does not work yet.
  • The syslog action is configured for debugging purposes only and can be removed in production environment.
  • More complex functionality (for example, sending show command output in an email) can be implemented with help of Tcl EEM policies
see 2 comments

Protecting the primary DNS server on your router

In a comment to my post describing how to make a router into a primary DNS server, one of the readers noted that you could easily overload a router doing that ... and he's obviously right.

Apart from having too many valid DNS requests for the zone the router is responsible for, the observed behavior could be spam-related. Just a few days ago when I've discussed the router-based DNS server with my security engineers, they've pointed out that a lot of spammers perform regular DNS attacks trying to poison the DNS cache of unpatched open caching DNS servers.

Obviously, a router is no match in raw CPU power to a high-end server, so even when running the authoritative server on the router, it might not be a bad idea to use a DNS server of your ISP as the secondary DNS and list only the ISP's DNS server in the NS records for your zone. This would deflect most of the traffic (as nobody would know your router is acting as a DNS server), but I would still apply an inbound access-list allowing only DNS queries from the secondary name server on the Internet-facing interface.

Alternatively, you could protect the router with Control Plane Policing and drop excessive DNS request packets, but that would affect the queries you should respond to as well.
see 6 comments

Unicode IPS vulnerability: more details

Cisco has released security response acknowledging that the IPS software does not properly handle a rarely used Unicode encoding. Reading the security notice you might be left wondering what's going on. Here's the whole story.

Within an URI (web address), the ASCII characters can be encoded in one of three ways:
  • Unless they are reserved, they can be included in the URI directly (for example, you can always use the letter a in an URI).
  • You can always escape a character using its hexadecimal value. Letter a can thus be written as %61.
  • Unicode character set includes full-width form of ASCII characters, where letter a can be encoded as a two-byte value 0xFF61 (or %ff%61 in an URI)
The IPS software (standalone or integrated in Cisco IOS) does not recognize the sequence %ff%61 as letter a. It's thus possible to evade some IPS triggers by replacing ASCII characters with their full-width Unicode encoding.

The vulnerability by itself does not open new attack routes, unless you use the IPS as the only means of protection of a vulnerable system (which is a bad practice anyway).
add comment

DNS views are broken in release 12.4(11)T

The Split DNS functionality introduced in IOS release 12.4(9)T has survived a single maintenance cycle before being broken. While you can still configure the DNS views in 12.4(11)T2 (and they still work), the view names are missing from the router-generated configuration (show running, for example), making the configuration syntactically incorrect. The router will thus reboot without DNS views after you've saved the running configuration to NVRAM.
see 2 comments

Command Authorization Fails with EEM applet or Tcl policy

One of my readers asked an interesting question: „why do the commands executed within a EEM Tcl policy fail with Command authorization fails message?“ The short answer is simple: If you use AAA command authorization (which you can only do if you're using a TACACS+ server), you have to specify the username under which the EEM will execute its CLI commands with the event manager session cli username user configuration command.

read more see 8 comments

Background Continuous Ping from a Router

In a previous post, I've described how you could generate a (almost) continuous ping from a router using the extended ping command. While that approach is extremely simple to implement, it ties up a line (and if you're working from the console, it's highly impractical). You could get the same results (as Tom has already pointed out) using IP SLA feature of Cisco IOS. Configure the ping request with commands similar to these:
ip sla 100
  icmp-echo 172.16.1.2
  timeout 500
  frequency 3
To start the ping, use the ip sla schedule 100 life forever start-time now configuration command, to stop it, no ip sla schedule 100 command. When using IP SLA ping, you can trigger Embedded Event Manager applets to report ping failure (similar to the technique described in this post) or read the ping results from SLA MIB with SNMP.

Note: In IOS release 12.3(14)T, the ip sla command replaced the rtr command. To use this technique in IOS release 12.3 or lower, use the rtr configuration command.
add comment

The self zone in zone-based firewall configuration

One of my readers made an interesting observation when faced with configuring zone-based firewall on Cisco IOS: „My main issue is a confusion between when to use self and when to use in/outside.“

The rules are simple:
  • Whenever you filter traffic transiting the router, you control it with a zone-pair specifying an inside and an ouside zone.
  • The self zone controls traffic sent to the router itself or originated by the router.
  • Unless you specify a zone-pair combining self zone with another zone, all traffic from that zone sent to the router itself is allowed (the router is not protected)
  • To control traffic that the router can send into a zone use a zone-pair from self to another zone. Use inspect in the service-policy to allow the return traffic.
  • To filter the traffic that the router can accept, use a zone-pair from another zone to self. Only the packets accepted by this zone-pair's service-policy will be accepted by the router.
More information about the self zone (as well as other aspects of zone-based firewall configuration) can be found in my digital book Deploying Zone-Based Firewalls published by Cisco Press.
see 8 comments

Use Tcl script to change the interface status

During network troubleshooting or proof-of-concept testing, I often change the state of a loopback interface (to insert or remove IP prefix from the routing protocols) or flap it to test the impact on network stability. The traditional approach to this procedure should be known to everyone:
  • Enter configuration mode;
  • Select the appropriate interface with the interface loopback x command;
  • Try to remember whether you need to disable or enable it;
  • Issue the shutdown or no shutdown command;
  • Exit the configuration mode and continue your debugging/testing process.

After a particularly boring testing session I decided to write a Tcl script to automate the job. To use it, download it and store it into ifchange.tcl. Download the Tcl file to your router (Flash or NVRAM) and define an alias: alias exec ifchange tclsh flash:ifchange.tcl. Now you can use the new ifchange command to change interface status.

read more see 2 comments

IOS Tclsh resources

Before trying to write Tcl procedures to be executed by Cisco IOS tclsh command, read the following articles:

And last but not least, if you want to store Tcl procedures on your router and don't want to write into the router's Flash memory (I hate that the router prompts me whether I want to erase the flash every time I store something into it), you can store them in NVRAM.

add comment
Sidebar