Your browser failed to load CSS style sheets. Your browser or web proxy might not support elliptic-curve TLS

Building network automation solutions

6 week online course

reserve a seat

Catching all syslog messages

If you use UDP-based syslog servers, you might have noticed that they miss a message or two generated during a router reload (particularly when the syslog server is on a directly connected LAN). The reason is simple - when the first message is sent to the syslog server, its MAC address is not yet in the router's ARP cache and the message is dropped. To prevent the message loss, you can use the logging server-arp configuration command (introduced in IOS release 12.3T), making sure that the router sends ARP request to the configured syslog server(s) before generating the first syslog messages.

Note: this issue does not occur when using TCP as the syslog transport mechanism.
Add comment

The DNS configuration "challenges"

With the introduction of DNS views in IOS release 12.4(9)T, a number of additional DNS-related configuration commands were introduced. As IOS still supports all the older configuration commands (and the DHCP-acquired DNS servers), the results are not as obvious as one would hope. The IOS documentation is pretty explicit (a nice surprise :), but I still had a bit of a headache figuring it all out, so you might be in the same position.

Let's start with the DNS resolvers (the DNS name servers the router itself uses when it needs to change a name into an IP address or vice versa):
  • You can configure the DNS resolvers with the ip name-server global configuration command or the domain name-server configuration command within the ip dns view default.
  • Both lists are merged and combined with the IP addresses acquired from the DHCP reply messages to get the final list of the DNS resolvers. You can inspect the final list with the show ip dns view default command.
  • The DNS name servers from the DHCP replies never appear in the router configuration.
  • If the same IP address is specified with the ip name-server and domain name-server command, only the global (ip name-server) command will appear in the router configuration.

The rules for DNS forwarders (the DNS servers the router uses to answer incoming DNS queries) are a bit different:

  • The DNS view the current DNS query should use is determined based on view-list assigned to incoming interface (with the ip dns view-group interface configuration command) or the global view-list (specified with the ip dns server view-group global configuration command).
  • The incoming query is (sequentially) sent to the IP addresses configured as dns forwarders in the selected view. No other DNS servers are used.
  • If the selected view has no dns forwarders, but contains domain name-servers, they are used as forwarders.
  • If the selected view has no dns forwarders or domain name-servers, the query is forwarded as IP broadcast ... unless the selected view is the default view, in which case the first DNS server computed according to the the previous bullet list is used (only one DNS server is used in this case, even if you have configured multiple DNS servers with the ip name-server configuration command).
see 6 comments

Warm upgrade

After you've configured the Warm Reload, you can also perform warm IOS upgrade/downgrade (assuming that you already run at least the IOS release 12.3(11)T or 12.4). The Warm Upgrade functionality loads the new IOS image into the main memory, decompresses it and starts it, significantly reducing the downtime (in my case, a 2800 router reloaded in 62 seconds as compared to 415 seconds it took to load the image from a locally-attached server).

Apart from the downtime reduction, the warm upgrade (requested with the reload warm file url command) has a number of other benefits:
  • The new image does not have to be stored in flash
  • You don't have to change the boot image with the boot system command
  • If the new image crashes, the router will revert to the original IOS image stored in flash
You can see when the actual reload takes place in the following printout - the reload starts after the new image has been loaded and decompressed (when the router prints the SYS-5-RELOAD message).
a2#reload warm file tftp://10.0.0.10/c2800nm-advipservicesk9-mz.124-11.T.bin
Proceed with reload? [confirm]y
Loading c2800nm-advipservicesk9-mz.124-11.T.bin from 10.0.0.10 (via FastEthernet0/0):
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[OK - 39782360 bytes]

Decompressing the image : ###################################################################################################################################################################################################################################################################################################################################################################################################################################################### [OK]

00:07:12: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload Command.
00:07:12: %HSRP-6-STATECHANGE: FastEthernet0/0 Grp 1 state Standby -> Init
see 1 comments

Reload EEM Tcl policy with help of Tcl shell

Testing Embedded Event Manager (EEM) Tcl policies is a convoluted process:
  • Source file is usually edited on a general-purpose workstation.
  • The file has to be downloaded to router's local storage (EEM does not register non-local policies).
  • The new version of the EEM policy has to be registered with EEM with event manager policy configuration command
  • After all these steps, the new policy can be tested.
While you can use EEM applet to automate this process, slightly more flexible approach (you can specify the policy name to be replaced) can be implemented with Tcl script:
set policy [lindex $argv 0]
set source "tftp://10.0.0.10/tcl/" # replace with your host and directory
set destination "nvram:" # replace with local storage device
if {[string equal $policy ""]} {
return -code error "expected policy name"
}
puts "replacing policy: $policy"
ios_config "file prompt quiet"
ios_config "no event manager policy $policy" ""
exec "copy $source$policy $destination$policy"
ios_config "event manager policy $policy"
ios_config "no file prompt quiet"
To use the script, follow these steps:
  • Save the script in a .tcl file (for example, changePolicy.tcl)
  • Change the script parameters (remote host and local storage)
  • Save the .tcl file to your router's local storage (you can also run it from a remote server)
  • Configure a command alias, for example alias exec eem tclsh flash:changePolicy.tcl testPolicy.tcl
Now you can replace the target EEM Tcl policy with a simple eem command.

Alternatively, if you define alias exec eem tclsh flash:changePolicy.tcl, you can specify policy name as an argument to the eem command, for example eem testPolicy.tcl.
see 1 comments

Unbundle DNS settings from DHCP client

In one of my previous posts I've been writing about the problems I had when the DHCP client on Cisco IOS was messing up the DNS name-servers I've configured manually with the ip name-server configuration command. As is quite usual in Cisco IOS, there's one more know to turn to fix this - the Configurable DHCP Client feature introduced in IOS release 12.3(8)T.

To stop the router's DHCP client from overwriting the static name-server settings, use the no ip dhcp client request dns-nameserver interface configuration command (you can also exclude a few other DHCP options).
Add comment

Change the Telnet font color from a Cisco router

I've recently got an excellent question: how can I change the color of the terminal emulator font from the CLI by sending the telnet program an escape sequence?

For those of you that haven't worked with escape sequences before - you can control a lot of parameters in you terminal emulation program by sending it a special sequence starting with <ESC>[ (<ESC> begin character code 27). These sequences work even in the simplest telnet clients on Windows and Linux thanks to built-in operating system support or ANSI.SYS driver (on Windows); you can get an in-depth description and the list of all supported escape sequences from Wikipedia.

With the list of escape sequences in hand, the only remaining problem is how to send them from the router. Tcl shell was the immediate answer, but then I remembered the various banners provided by the Cisco IOS. As I wanted to change the font color as soon as possible, the login banner (or the message-of-the-day banner) seemed the right one to use.

To insert the ESC character in a banner from the configuration prompt, you have to use the Ctrl-V/Esc sequence. For example, to change the font color to bright yellow, do the following:
router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
router(config)#banner motd #
Enter TEXT message. End with the character '#'.
<Ctrl-V><ESC>[33;1mThis is a yellow router#
router(config)#
When you look at the running configuration, the router transforms the control characters into ^x sequences, so your banner will look like this:
router#show run | begin banner
banner motd ^C
^[[33;1mThis is a yellow router^C
...
Notes:
  • Saving router configurations on (T)FTP server and reloading them from a server does not pose a problem, as the control characters are preserved throughout the process, but you cannot use cut-and-paste with banners, as the terminal emulator does not transform ^x sequence into a control character.
  • With this technique, you can color-code your routers. For example, internal routers could use green font, publicly accessible ones red font and the firewall routers yellow font, so all the operators would know immediately what to expect on each device.
  • This article is part of You've asked for it series.
see 5 comments

DNS resolver in Cisco IOS is auto-configured with parameters from a DHCP reply

If you're using DHCP to get IP interface addresses on your router (using the ip address dhcp interface configuration command), the router will also inherit the DNS resolver settings included in the DHCP reply. Makes sense, but the implementation is "a bit" unexpected: if you configure the DNS name servers manually with the ip name-server address-list command, the ones matching the values in the DHCP reply packet are not included in the running configuration and thus not saved to NVRAM. Even worse, the statically-configured name-servers overwritten by a DHCP reply are lost if the DHCP-configured interface goes down.

To avoid total confusion, you thus have these options:
  • Do not use DHCP to acquire IP interface addresses
  • Make sure the DHCP server does not send DNS-related parameters (a bit hard if you're using DHCP with your ISP)
  • Rely exclusively on DHCP to provide your router with the DNS name server addresses
Here is also an example of what can happen if you mix static configuration with DHCP. We'll start by configuring the name servers and verifying they are configured:
ro#conf t
Enter configuration commands, one per line. End with CNTL/Z.
ro(config)#ip name-server 192.168.2.1 192.168.2.2
ro(config)#^Z
ro#show run | include name-server
ip name-server 192.168.2.1 192.168.2.2
Next, we'll configure DHCP client on an interface and watch the DHCP debugging to see what's actually going on (only parts of debugging printout are included):
ro(config)#interface FastEthernet 0/0
ro(config-if)#ip address dhcp
...
DHCP: Received a BOOTREP pkt
DHCP: Scan: Message type: DHCP Ack
DHCP: Scan: Server ID Option: 192.168.2.1 = C0A80201
DHCP: Scan: Lease Time: 86400
DHCP: Scan: Renewal time: 43200
DHCP: Scan: Rebind time: 75600
DHCP: Scan: Host Name: ro.address.net
DHCP: Scan: Subnet Address Option: 255.255.255.240
DHCP: Scan: Router Option: 192.168.2.1
DHCP: Scan: Domain Name: address.net
DHCP: Scan: DNS Name Server Option: 192.168.2.2
...
DHCP: Applying DHCP options:
Setting default_gateway to 192.168.2.1
Adding default route 192.168.2.1
Adding DNS server address 192.168.2.2
Setting default domain to address.net

%DHCP-6-ADDRESS_ASSIGN: Interface FastEthernet0/0 assigned DHCP address 192.168.2.5, mask 255.255.255.240, hostname ro
The name server received in the DHCP reply (192.168.2.2) is now missing from the running configuration:
ro#show run | include name-server
ip name-server 192.168.2.1
see 10 comments

Periodic execution of IOS show commands

If you want to execute IOS show commands periodically (for example, to monitor router status or take snapshots of routing tables), you can combine new output redirection features introduced in IOS release 12.2T in an Embedded Event Manager (EEM) applet. For example, to store the brief interface status into a file on an FTP server, use the following EEM applet:
event manager applet SaveInterfaceStatus
event timer watchdog name SaveIfStat time 60
action 1.0 cli command "show ip interface brief | redirect ftp://username@password:host/path"
action 2.0 syslog msg "Interface status saved"
Notes:
  • The timer watchdog EEM event defines a recurring event triggered every X seconds.
  • Output of a show command can be redirected only to a TFTP or FTP server, redirection to a web (HTTP) server does not work yet.
  • The syslog action is configured for debugging purposes only and can be removed in production environment.
  • More complex functionality (for example, sending show command output in an email) can be implemented with help of Tcl EEM policies
see 2 comments

Protecting the primary DNS server on your router

In a comment to my post describing how to make a router into a primary DNS server, one of the readers noted that you could easily overload a router doing that ... and he's obviously right.

Apart from having too many valid DNS requests for the zone the router is responsible for, the observed behavior could be spam-related. Just a few days ago when I've discussed the router-based DNS server with my security engineers, they've pointed out that a lot of spammers perform regular DNS attacks trying to poison the DNS cache of unpatched open caching DNS servers.

Obviously, a router is no match in raw CPU power to a high-end server, so even when running the authoritative server on the router, it might not be a bad idea to use a DNS server of your ISP as the secondary DNS and list only the ISP's DNS server in the NS records for your zone. This would deflect most of the traffic (as nobody would know your router is acting as a DNS server), but I would still apply an inbound access-list allowing only DNS queries from the secondary name server on the Internet-facing interface.

Alternatively, you could protect the router with Control Plane Policing and drop excessive DNS request packets, but that would affect the queries you should respond to as well.
see 6 comments

Try before you buy: Configure multi-area OSPF

In a recent post, I've been writing about our CCNP e-learning offerings. If you're aiming toward the CCNP certification and don't have time to attend a classroom session (or hate sitting through four weeks of training), these products might be the perfect fit for you.

If you'd like to evaluate our e-learning offerings, you can get free access to a sample module and related lab covering area configuration in OSPF; just go to our Blended Solutions Portfolio page and click on the Try our E-course Demo link at the top of the page.
Add comment

USB usability in ISR routers

In his USB Usability Q&A article, Scott Morris states that ISR routers (1800, 2800, 3800) use USB port for security keys. While that's definitely true, the usability of the USB ports go way beyond that. Once you insert a USB token into the port (and IOS release 12.4 recognizes tokens up to at least 1GB), you get a generic storage device (usbflash0: and/or usbflash1:) that can be used to:
see 1 comments

Unicode IPS vulnerability: more details

Cisco has released security response acknowledging that the IPS software does not properly handle a rarely used Unicode encoding. Reading the security notice you might be left wondering what's going on. Here's the whole story.

Within an URI (web address), the ASCII characters can be encoded in one of three ways:
  • Unless they are reserved, they can be included in the URI directly (for example, you can always use the letter a in an URI).
  • You can always escape a character using its hexadecimal value. Letter a can thus be written as %61.
  • Unicode character set includes full-width form of ASCII characters, where letter a can be encoded as a two-byte value 0xFF61 (or %ff%61 in an URI)
The IPS software (standalone or integrated in Cisco IOS) does not recognize the sequence %ff%61 as letter a. It's thus possible to evade some IPS triggers by replacing ASCII characters with their full-width Unicode encoding.

The vulnerability by itself does not open new attack routes, unless you use the IPS as the only means of protection of a vulnerable system (which is a bad practice anyway).
Add comment

DNS views are broken in release 12.4(11)T

The Split DNS functionality introduced in IOS release 12.4(9)T has survived a single maintenance cycle before being broken. While you can still configure the DNS views in 12.4(11)T2 (and they still work), the view names are missing from the router-generated configuration (show running, for example), making the configuration syntactically incorrect. The router will thus reboot without DNS views after you've saved the running configuration to NVRAM.
see 2 comments

Command Authorization Fails with EEM applet or Tcl policy

One of my readers asked an interesting question: „why do the commands executed within a EEM Tcl policy fail with Command authorization fails message?“ The short answer is simple: If you use AAA command authorization (which you can only do if you're using a TACACS+ server), you have to specify the username under which the EEM will execute its CLI commands with the event manager session cli username user configuration command.

read more see 8 comments

Test drive Carrier's Carrier MPLS VPN service

Carrier's Carrier MPLS VPN service is one of the more confusing aspects of MPLS VPN technology; there are simply so many different bits and pieces that have to fit together just right to make it work (although we did a pretty good job describing it in the Cisco Press book MPLS and VPN architectures, Volume II). If you would like to set it up in a test environment, here's what you can do: if you have partner-level Cisco Connection Online access, you can do it free of charge: If you're not working for a Cisco partner, you can buy the whole set of advanced MPLS remote labs from NIL Data Communications.
Add comment

FTP server is gone from Cisco IOS

It was a weird coincidence - yesterday I had to upgrade a 2800-series router and had no TFTP or FTP server available on my workstation, so I've tried to use the FTP server on the router. The ftp-server command was not available, which did not surprise me at all, as I knew the FTP server was only available in selected IOS feature sets. However, as I finally found an USB key, transferred the Advanced IP Services 12.4(11)T image to the router's flash and reloaded the router, the real surprise came: the FTP server was still not there.

Today I've got the bad news: I already knew there were "a few" bugs in the IOS FTP server that you could exploit. Instead of fixing them, Cisco simply decided to remove the FTP server altogether.

Why is this so bad? Look at the list of protocols that you can use to transfer files to and from the router that I've put together in my IP Corner article Using a Web Server to Manage Your Router Configurations. As you cannot transfer a file into the router's flash with the embedded HTTP server, the only protocol that you could use to get a new IOS image to the router from a Windows workstation with no additional software installed was FTP, and now that option is gone.
see 2 comments

Background continuous ping from a router

In a previous post, I've described how you could generate a (almost) continuous ping from a router using the extended ping command. While that approach is extremely simple to implement, it ties up a line (and if you're working from the console, it's highly impractical). You could get the same results (as Tom has already pointed out) using IP SLA feature of Cisco IOS. Configure the ping request with commands similar to these:
ip sla 100
icmp-echo 172.16.1.2
timeout 500
frequency 3
To start the ping, use the ip sla schedule 100 life forever start-time now configuration command, to stop it, no ip sla schedule 100 command. When using IP SLA ping, you can trigger Embedded Event Manager applets to report ping failure (similar to the technique described in this post) or read the ping results from SLA MIB with SNMP.

Note: In IOS release 12.3(14)T, the ip sla command replaced the rtr command. To use this technique in IOS release 12.3 or lower, use the rtr configuration command.
Add comment

What is a cached CEF adjacency?

Whenever a router running CEF switching has LAN interfaces (or any other multi-access interfaces), you'll find cached adjacencies for active directly attached IP neighbors in its CEF table. These adjacencies ensure the smooth traffic flow toward the LAN-attached next-hops (preventing the initial packet drop symptom once the next-hop becomes active).

read more see 3 comments

The self zone in zone-based firewall configuration

One of my readers made an interesting observation when faced with configuring zone-based firewall on Cisco IOS: „My main issue is a confusion between when to use self and when to use in/outside.“

The rules are simple:
  • Whenever you filter traffic transiting the router, you control it with a zone-pair specifying an inside and an ouside zone.
  • The self zone controls traffic sent to the router itself or originated by the router.
  • Unless you specify a zone-pair combining self zone with another zone, all traffic from that zone sent to the router itself is allowed (the router is not protected)
  • To control traffic that the router can send into a zone use a zone-pair from self to another zone. Use inspect in the service-policy to allow the return traffic.
  • To filter the traffic that the router can accept, use a zone-pair from another zone to self. Only the packets accepted by this zone-pair's service-policy will be accepted by the router.
More information about the self zone (as well as other aspects of zone-based firewall configuration) can be found in my digital book Deploying Zone-Based Firewalls published by Cisco Press.
see 8 comments

Use Tcl script to change the interface status

During network troubleshooting or proof-of-concept testing, I often change the state of a loopback interface (to insert or remove IP prefix from the routing protocols) or flap it to test the impact on network stability. The traditional approach to this procedure should be known to everyone:
  • Enter configuration mode;
  • Select the appropriate interface with the interface loopback x command;
  • Try to remember whether you need to disable or enable it;
  • Issue the shutdown or no shutdown command;
  • Exit the configuration mode and continue your debugging/testing process.

After a particularly boring testing session I decided to write a Tcl script to automate the job. To use it, download it and store it into ifchange.tcl. Download the Tcl file to your router (Flash or NVRAM) and define an alias: alias exec ifchange tclsh flash:ifchange.tcl. Now you can use the new ifchange command to change interface status.

A sample router printout is included below:
a1#ifchange
Syntax: tclsh ifchange.tcl interface [on|off|change|flap]
a1#ifchange loop0 off
Interface loop0 changed state to off
a1#
%LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to down
a1#ifchange loop0 on
Interface loop0 changed state to on
a1#
%LINK-3-UPDOWN: Interface Loopback0, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
a1#ifchange loop0 flap
changing state of loop0
shut down interface loop0
... waiting ...
%LINK-5-CHANGED: Interface Loopback0, changed state to administratively down
%LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to down
changing state of loop0
enable interface loop0
a1#
%LINK-3-UPDOWN: Interface Loopback0, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
see 2 comments

IOS Tclsh resources

Before trying to write Tcl procedures to be executed by Cisco IOS tclsh command, read the following articles:

And last but not least, if you want to store Tcl procedures on your router and don't want to write into the router's Flash memory (I hate that the router prompts me whether I want to erase the flash every time I store something into it), you can store them in NVRAM.

Add comment

Enhance the IOS user interface

Have you ever wanted to fine-tune the IOS show commands to provide you with the exact information you need instead of having to dig through long screens full of data you are not interested in to find what you need?

In this month's IP Corner article, Enhance the IOS User Interface, I'm describing how to use simple filters provided by the Cisco IOS to pick only the information you need from the printouts, as well as how to generate tailored printouts (even combining outputs from multiple show commands) with Tcl shell introduced in IOS release 12.3(2)T.
Add comment
Sidebar