Blog Posts in August 2008
In early releases supporting zone-based policy firewall configuration (IOS 12.4(6)T), match protocol command cannot be used to classify traffic to or from the self zone. Only IP access lists can be used for traffic classification purposes.Misha Volodko reported that the match protocol icmp command works for him when used with the self zone. Another small step toward perfect implementation :) ... and don't forget that you can always use class class-default to catch all the unclassified traffic (and log it before it's dropped, for example).
- Jeremy Stretch explains Path MTU Discovery process with vivid diagrams. He also covers 801.2X guest VLANs and describes how EIGRP goodbyes can detect unidirectional links.
- InformIT has published an overview article covering BGP-based attacks.
- CCIEpursuit documents the "wonderful" experience that most senior IT engineers are exposed to as well as a few useful commands to reset an EIGRP process.
- Arden Packeer is still an OSPF fan, this time covering the OSPF filter lists.
- And, last but not least, Cisco's marketing has upset quite a few people with the new CCIE logos, including Greg Ferro @ Etherealmind.
If I create a SNMPv3 user which has a password (snmp-server user userthree groupthree v3 auth md5 user3passwd), this user does not appear in the running- or startup-config. Cisco even documents this if you know what to look for.Like certificates, the SNMPv3 users are stored in private-config and thus never appear in the router configuration. If you want to have a backup of the user data, create a text file on one of your NMS servers, add SNMPv3 usernames and passwords in the text file and use the copy somewhere running-config to configure SNMPv3 users on the routers.
I strongly suspect (although I did not test this) that these users are also missing from configuration exported to TFTP servers. What would be the recommended way to make usable config backups of routers with such users?
Peter Weymann sent me a really intriguing question:
A few days ago I started reading the Ciscopress book End-to-End Network Security: Defense-in-Depth and stumbled over the scheduler command. This one could be used to allocate time that the cpu spends on fast switching packets or process switching packets, if I understand it correctly. They also mention interrupting CPU processes but honestly I don't really understand how it works.
Cisco routers support (at least) three forms of layer-3 switching (formerly known as routing). CEF switching and fast switching are performed entirely within the interrupt context (I/O adapter interrupts a process the CPU is currently executing and all the work is done before the process resumes). Process switching is performed in two steps: packet is briefly analysed within the interrupt context and requeued into the IP Input process where it's eventually switched. Almost all I/O adapters used these days use a concept of RX/TX rings to communicate with the CPU, meaning that the CPU potentially has to handle more than one packet for each interrupt.
Fast switching is gone starting with IOS release 12.4(20)T.
Under very high load, the packet arrival rate could be so high that the router would constantly service packets within the interrupt context without ever returning back to the IOS processes.
You can check the CPU load incurred by the interrupt context and IOS processes with the show process cpu command. The second number in the five seconds part of the first line tells you the amount of interrupt context activity in the last five seconds.
To prevent the starvation of IOS processes (which could result in keepalive and routing protocol problems, eventually leading to loss of routing protocol neighbors), the scheduler allocate command limits the amount of time that can be spent in the interrupt context and allocates some guaranteed time to the IOS processes. Very probably the routers have a mechanism to mask the requests from the I/O adapters during that period so that the CPU is not interrupted (BTW, this slightly increases the jitter).
A similar command is the scheduler interval command. IOS has high- and low priority processes. Whenever the CPU has to decide what process to run (usually following an interrupt or when a process decides it's done with its work), it will run a high-priority process if one is ready. This could lead to starvation of low-priority processes and the scheduler interval command specifies the maximum amount of time the higher-priority processes can consume before a low-priority process is given a chance to run.
Unless you have serious (and I mean __serious__) problems in your network, don't play with these commands. They are a last-resort things you can do if you're under very heavy load and still need access to the exec to reconfigure the router. In most cases, you should not have to worry ... and anyhow, if the CPU load is close to 100%, you have other problems anyway.
Apart from the Inside Cisco IOS Software Architecture book that you absolutely must have if you're interested in (a bit outdated) view of the internals of Cisco IOS, you can get more information in these documents:
Did you know that RIP, the venerable routing protocol that is present in Cisco routers for the last 20 years, uses an internal database, not the IP routing table, to process RIP updates? This database contains no fancy information (like EIGRP topology table) that would allow RIP to converge faster, but there are still minor differences between the RIP database and the IP routing table.
Did you want to use private IP addresses in a public network? The short recommendation is: “don't”. If you use them in the network core, your customers might have problems with network troubleshooting; if you assign them to your customers and use NAT or PAT, you've just created serious security issues. You can find more details in the Avoiding private IP security risks in public networks article I wrote for SearchTelecom.com.
I'm wondering whether a router performing penultimate hop popping (PHP) imposes an IGP label or not.The value of implicit null is 3; does it mean the router imposes this label (and adds four bytes to the packet)?The penultimate router does not impose the IGP label (that's why this behavior is called penultimate hop popping). However, the egress router has to signal to its upstream neighbor (the penultimate router) that it should NOT impose a label, so it uses "implicit null" label (= 3) in TDP/LDP updates to signal that the top label should be popped, not rewriten.
If you store the reverse mapping for the routers’ loopback interfaces in DNS or configure the name-to-address mappings with the ip host commands, you can use the ip ospf name-lookup global configuration command to display the OSPF router IDs as router names.
Justin sent me an interesting problem description:
Emails that come from EEM don't have a body, it just puts anything after the body keyword in the subject.
I quickly wrote a simple EEM applet ...
event manager applet MailTest
action 1 mail server "10.0.0.10" to "[email protected]" from "[email protected]" →
subject "eeee" body "ffff"
... and started it with event manager run MailTest. The simple SMTP server I'm using for debugging EEM printed out the following message content:
Incoming mail ... from 10.0.0.1 received
From: <[email protected]>
To: <[email protected]>
Date: Fri, 01 Mar 2002 00:01:34 +0000
Message-ID: <[email protected]>
From: [email protected]
To: [email protected]
As you can see, there is no empty line between the Subject header and the message body. A quick look in RFC 822 confirmed that there should be an empty line separating headers from the message body:
message = fields *( CRLF *text )
Conclusion: EEM is not RFC822-compliant e-mail client.
Hint to EEM developers: if it works with sendmail, it's not necessarily correct.
At one point someone posted an article about a command you could run on the Catalyst switch that would give you back the distance of the cable between the switch and end device, but now I can't find it.I remembered reading the same article and after I've figured out the underlying technology is called TDR (Time Domain Reflectometer), uncle Google immediately provided a reader tip from Csaba Farkas.
Working on an implementation of a split DNS design, I encountered an interesting bug in Cisco IOS: the ip dns view-group command works only on interfaces, but not on subinterfaces. As it’s a pure IP feature, there obviously no reason why it shouldn’t work on anything that has an IP address; obviously someone forgot to insert the correct entry in the parser tables.
The CCIE preparation programs also cover an enormous amount of scenarios and variations, giving you lots of material to practice (BTW, when I was teaching CCIE preparation bootcamps 15 years ago, the pass rate of my students was over 90% as I simply forced them to configure all the possible stupidities Cisco IOS could do at that time). The tests don't have to get any easier; the participants (if the calculations are correct) are simply better prepared. Whether the increased number of CCIEs results in the perceived devaluing of the program is another question (remember: the supply/demand rules), but I am absolutely sure that people passing CCIE lab exam these days know approximately as much as those passing it two or three years ago.
Of course you could argue whether someone who did tens (or sometimes hundreds) of scenarios in his lab and then passed the CCIE test is an expert or a braindump cheater (let's wait for the first blog post that claims that), but I doubt anyone is able to remember so many recipes and apply the correct one without a profound understanding of the underlying issues.
I tend to forget whether I'm in configuration mode or not and often type the do command in exec mode or the show command in configuration modes. With the alias functionality you can make the show command a native command in the configuration modes; just configure alias configure show do show.
The “only” drawback of this approach is that IOS has zillion different configuration modes and you have to define the alias in each one of them (you could do it just in the most common ones … or try to remember to type the do keyword first :).
DMVPN is also covered by Jeremy Stretch (I'm starting to wonder what's the root cause for the sudden fascination with this solution), who also provided a nice introduction to EUI-64 IPv6 addresses, a very practical view on shaping-versus-policing dilemma and simple step-by-step introduction to 802.1X.
As one would expect, Joe Harris and Arden Packeer are also ignoring the summer temptations. Joe provided an interesting link to the CCDE practical exam demo and Arden is continuing with his "OSPF over Frame Relay" saga (a few more installments and he'll be getting close to Jordan's Wheel of Time).
And last but not least: Tim Riegert sent me a link to a page full of TCP/IP and IMS Sequence Diagrams. The diagrams serve as a demonstration of EventStudio System Designer capabilities, but they are still good.
BGP route reflectors have been supported in Cisco IOS well before I started to develop the first BGP course for Cisco more than a decade ago (one of the early ABCT course descriptions is still available online thanks to the Wayback machine). It’s a very simple feature, so I was pleasantly surprised when I started digging into it and discovered a few rarely known details. Read all about the BGP route reflector details in the article I wrote in the CT3 wiki.
Once upon a time, in the land of IP, there was a wide area network (WAN) providing connectivity between clients and servers, and all was well. Then, suddenly, bad things started to happen, and paranoia spread throughout the land. Firewalls grew around hamlets to protect them from the unknown beyond the realm of calm, but then packets were forced to travel thorough the dark forests of the WAN. There was a need to provide them with protection.
I've got an interesting question from Colin a while ago:
I would like to generate a different prompt during the login to the router if the TACACS+ server has failed, indicating to the network operators that they have to log-in with the special (local) username, not with the TACACS+ authenticated username/password.
Fortunately he was running TACACS+ which supplies its own prompts during the authentication phase (the solution would not work with RADIUS). If you change the local authentication prompts, you'll get the prompts from TACACS+ server if it's reachable from the router (the AAA authentication is performed via TACACS+ server) and the local prompts if the TACACS+ server has failed (the AAA authentication is performed via any other mechanism). Here's a sample configuration:
aaa authentication login REMOTE group tacacs+ local
aaa authentication fail-message #
Local authentication failed.
aaa authentication password-prompt "Enter local password:"
aaa authentication username-prompt "Enter local username:"
user a secret b
line vty 0 4
login authentication REMOTE
It's obvious why two routers in the same OSPF domain cannot have the same router ID. But requiring unique router IDs on OSPF processes running in different VRFs is probably too harsh, although it does prevent confusion if two VRFs ever get connected through a customer site. Anyhow, if you have overlapping IP addresses on loopback interfaces in different VRFs, OSPF process might not start.