Build the Next-Generation Data Center
6 week online course starting in spring 2017

Configuration rollback fails completely with IP SLA

The Configuration Rollback feature (first available in IOS release 12.3(7)T) fails completely when handling configurations containing active IP SLA probes.For example, if you want to rollback from ...

ip sla 100
icmp-echo source-interface Serial0/0/0.100
timeout 200
frequency 10
ip sla schedule 100 life forever start-time now
... to ...
ip sla 100
icmp-echo source-interface Serial0/0/0.100
timeout 200
frequency 5
ip sla schedule 100 life forever start-time now
... the Configuration Replace tries to execute ...
ip sla 100
frequency 5
... which fails as you cannot change parameters of a running SLA probe. Since the Configuration Replace feature contains no special provisions for IP SLA commands (which is the actual bug), it tries the same (failing) configuration sequence a number of times and then reports an error.

Did I report this bug? Of course not :)

Per-destination or per-packet CEF load sharing?

Cisco Express Forwarding (CEF) can perform per-packet or per-destination (actually source/destination IP address pair) load-sharing with no performance degradation (without CEF, per-packet load-sharing requires process switching). Even though there is no performance impact on the router, per-packet load sharing will almost always result in out-of-order packets. The packet reordering might degrade TCP throughput in high-speed environments (in low-speed/few-flows scenarios, per-packet load-sharing actually improves the per-flow throughput) or severely impact applications that cannot survive out-of-order packet delivery, such as Fast Sequenced Transport for SNA over IP or voice/video streams.

To configure per-packet load-sharing, use the ip load-sharing per-packet interface configuration command (default is per-destination). This command has to be configured on all outgoing interfaces over which the traffic is load-shared.

The switch between the load-sharing modes is not immediate; sometimes you have to wait a few seconds for the ip load-sharing command to take effect, worst case a manual clearing of the CEF table (clear ip cef address) is required.

Configuration Change Logging ignores the configuration downloads

The Configuration Change Notification and Logging feature is supposed to log changes to the router's configuration. While it does a great job of logging commands entered in the router configuration mode, it completely ignores configuration changes due to configuration download (for example, with configure network or copy tftp running-config command).Here is an example:

fw#configure terminal
fw(config-archive)#log config
fw(config-archive-log-cfg)#logging enable
fw#configure network
Host or network configuration file [host]?
This command has been replaced by the command:
'copy system:/running-config'
Address or name of remote host []?
Source filename [fw-confg]?
Configure using tftp:// [confirm]
Loading fw-confg from (via FastEthernet0/0): !
[OK - 858 bytes]
%SYS-5-CONFIG_I: Configured from tftp:// by console
fw#show archive log config all
idx sess user@line Logged command
1 1 console@console logging enable

Which switching path does an IOS feature use

I've got an excellent question recently: Which switching path is used in Zone-based firewalls when a packet is dropped? As usual, IOS documentation was not very helpful (which is understandable as the answer might depend on hardware platform, interface encapsulation, other features configured on the router etc.). However, there is a great tool to use - the show interface stats command.

Fine-tuning CEF load-sharing

In environments with a low number of IP hosts you have to fine-tune the CEF load-sharing algorithm to ensure that the traffic is spread between all parallel paths. A typical scenario is a primary-backup data center setup with pairs of replicating servers, as shown in the figure below.

In these cases, you have to try different values of seed parameter of the CEF universal algorithm.
For example, if you have two equal-cost paths between networks and ...
a1#show ip cef detail, version 33, epoch 0, per-destination sharing
0 packets, 0 bytes
via, Serial0/0/0.200, 0 dependencies
traffic share 1
next hop, Serial0/0/0.200
valid adjacency
via, Serial0/0/0.100, 0 dependencies
traffic share 1
next hop, Serial0/0/0.100
valid adjacency
... you might want the traffic between and to flow over a different link than the traffic between and The command that will help you is the show ip cef exact-route source destination. In our example, both traffic flows would go over the same serial link:
a1#show ip cef exact-route -> : Serial0/0/0.100 (next hop
a1#show ip cef exact-route -> : Serial0/0/0.100 (next hop
However, by changing the seed parameter of the ip cef load-sharing algorithm universal command, you can influence the CEF hashing function, eventually reaching a state where the traffic flows are spread between both WAN links:
a1(config)#ip cef load-sharing algorithm universal 1
a1#show ip cef exact-route -> : Serial0/0/0.100 (next hop
a1#show ip cef exact-route -> : Serial0/0/0.200 (next hop

Local username authentication

As I get a lot of hits from Google refering to local login, here's the whole story: Cisco IOS supports local username/password based authentication (almost) forever (it's been there even before the AAA architecture). To change from simple password-based authentication to username+password based on, use login local configuration command on console and/or VTY lines. The local usernames and passwords are defined with the username configuration command.The Cisco IOS thus supports the following local (non-AAA) authentication settings:

  • no login disables any authentication; anyone able to access the line (console or VTY through telnet or SSH) is logged in automatically (do not use outside of lab environment).
  • login enables simple password-based authentication. The password is specified per-line (console or VTY) with the password command (do not specify different passwords on different VTY lines or you'll create total confusion).
  • login local enables local username+password authentication.

The login tacacs configuration command specifies the old TACACS protocol and is almost unusable these days.

This article is part of You've asked for it series.

Cisco IOS violates RFC 2616 (HTTP/1.1)

Update 2012-08-27: Stefan de Kooter reported the bug had been fixed in IOS release 15.1(4)M.

I simply had to check with the RFC; by setting the Host: field of HTTP request to an IP address (instead of a host name), Cisco IOS violates section 14.23 of RFC 2616, which says:

The Host request-header field specifies the Internet host and port number of the resource being requested, as obtained from the original URI given by the user or referring resource ... The Host field value MUST represent the naming authority of the origin server or gateway given by the original URL.

IOS HTTP client sets Host: field to IP address

Update 2012-08-27: Stefan de Kooter reported the bug had been fixed in IOS release 15.1(4)M.

If you run multiple web sites on a single physical server, it's highly likely that you rely on the ability of HTTP/1.1 clients to specify the Host: field in the HTTP request to indicate which web site they're trying to access.

Cisco IOS always inserts the web server's IP address (not the hostname) in the Host: field of the HTTP request, regardless of whether you enter IP address or hostname in the URL part of an IOS command that supports HTTP (for example, copy or more command) ... and regardless of whether the hostname is locally configured with the ip host command or resolved by an external DNS server specified in the ip name-server command.

End result: Cisco IOS-based routers (tested up to release 12.4(11)T) can access only the default web site on a web server hosting multiple web sites.

Log terminal access to your router

In a previous post, I've shown how you can log the changes in interactive user's privilege level. With the Cisco IOS Login Enhancements (introduced in IOS release 12.3(4)T, integrated in 12.4), you can also log all login successes and failures, even when using local user database (a similar functionality was previously achievable only when using central TACACS+ or RADIUS server).

The configuration commands to enable terminal access logging are login on-success log and login on-failure log. You can also specify that you want send SNMP traps in these circumstances (with the trap option) or that you only want to log every Nth attempt with the every n option.After you've configured terminal access logging, the router will start to generate syslog messages similar to the ones below (localport: 23 indicates the user was using telnet to access the router, localport: 80 that she was using HTTP):

%SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: test] [Source:] [localport: 23] at 19:10:27 UTC Sat Dec 2 2006
1d04h: %SEC_LOGIN-4-LOGIN_FAILED: Login failed [user: a] [Source:] [localport: 80] [Reason: Login Authentication Failed - BadPassword] at 19:35:53 UTC Sat Dec 2 2006
If the user accesses the router through the console port, both the source and localport are set to all zeroes:
%SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user: ] [Source:] [localport: 0] at 19:10:48 UTC Sat Dec 2 2006

Router Configuration Management … Too Good to be True?

In the Router Configuration Management … Too Good to be True?, the latest IP Corner article, I'm describing two of the router configuration management features introduced in Cisco IOS release 12.4: Configuration Change Notification and Contextual Configuration Diff utility. While the first one behaves as expected, the second one produced unexpected results under the stress test.