Some fifteen years ago we were building a router-based network using primarily baseband modems (that's how the DSL boxes with symmetrical speeds were called back then). Everything worked great, we even had DECnet running between a few sites. However, after a few weeks, a mystery phenomena crept up: when the users were copying files between two VAX computers, the link between the sites went down … always when copying the same file.

To make the long story short: every modem uses a predefined sequence that enables remote loopback. These sequences are standardized and have been chosen so that the chance of hitting them with real traffic is close to zero (but obviously not zero). The file that the users were trying to copy contained just the correct sequence to trigger the remote loopback in one of the modems and the link went down (most probably changed state to looped) as the routers started to receive their own packets. Disabling remote loopback on the modem (a jumper in those days) solved the problem.

The moral of the story: whenever you use pattern matching to identify something (be it a specific application that you're trying to identify or a virus in your workstation), there is a non-zero chance of false positives, usually in the most unusual places.

Let's conclude this post with what seems to me a ludicrous “invention”: someone patented the flashing of LEDs when performing loopback tests on a modem. If anyone can survive reading the whole patent application, understand it and recognize its true added value, please let me know … I got completely lost.

The flash memory available in newer router platforms (at the very minimum the ISR routers and 37xx series) is capable of being used as a regular disk drive (for example, to store system logging information), but it might be formatted as a traditional Low-End File System (LEFS) flash card (more likely if the router was not manufactured recently). To change the flash card format to disk-like FAT32 format, use the format flash: privileged-level command (and don't forget to store the IOS image to another location before formatting the flash). After the format process is complete, you can create subdirectories on the flash: memory and use it as a regular disk device.

Continuing from the previous mturoute-related post, this is how the mturoute utility behaves when you start it in traceroute mode (with the -t flag):

• Similar to Windows tracert, it tries to find the successive hops in the path by sending ICMP echo packets with increasing values of TTL field.

Contrary to Cisco IOS and most Unix systems that send UDP packets to high-numbered ports, tracert uses ICMP echo packets.

• For each router found in the path (= source IP address in the ICMP TTL exceeded message), mturoute tries to find path MTU to that hop using the same algorithm as in the ping mode.
• During the bisecting phase, the mturoute does not print all the messages it prints in the ping mode, but just the cryptic signs (+/-/u/.) indicating its progress. Their meaning is documented in the previous post.
• After the path MTU to the router under investigation is measured, mturoute reports the router's IP address and path MTU.

A few days ago, Jeremy Stretch asked me whether there's a command to display packet lists attached to router's interfaces. While he got pretty far with the output filters, he would like to have a nice tabular format as well as the contents of the access lists displayed next to the interfaces. The show ip access-list interface name command comes pretty close, but it displays the information only for a single interface, so it was time to write another Tcl script. To install it on your router:

1. Download it from my web site and copy it to your router's flash or NVRAM.
2. Define an alias, for example alias exec filters tclsh flash:packetFilters.tcl.

The script recognizes two parameters: the all parameter displays all interfaces, including ones with no access lists and the verbose parameter displays the contents of the access list after the interface name.

I always thought that the new format of the Cisco web-based documentation was awful, as it consumes way more bandwidth than the old version and is slower to load over low-speed links as it displays the text only after the complete page is loaded due to heavy use of table-based HTML layout (I will refrain from commenting the use of this layout technique in the third millenium). However, the new content structure has some significant benefits; for example, all the 12.4T feature guides are collected on a single page … fantastic if you try to find a feature that you remember was implemented somewhere in 12.4T track.

Jeff West has asked me to document the printout produced by the mturoute utility. Here's the first part of the documentation.

mturoute works in two modes:

• Without the -t flag, it sends variable-lenght ICMP echo packets to the specified destination address, trying to figure out the largest packet that is successfully propagated to the destination.
• With the -t flag, it uses traceroute-like algorithm to find the hop-by-hop IP addresses (the source IP addresses of the ICMP TTL exceeded replies) and uses the same packet-size-calculating algorithm to measure the path MTU to each hop.

Today we'll focus on the non-trace mode. It tries to measure the path MTU with a bisection method varying the packet sizes between minimum MTU (92) and maximum MTU (specified with the -m parameter, default is 10000 bytes). The payload size of the first packet (without the -m flag) is thus 5046 bytes ((10000 + 92)/2).

On each iteration, the algorithm prints a “cryptic” sign indicating whether the ping with the current payload size succeeded. The following indications are given:

• '+': ICMP echo reply arrived
• '-': The ping failed (for various reasons, including exceeding the path MTU)
• 'u': ICMP destination unreachable response arrived, indicating blackhole or access-list.

ICMP unreachable is considered a successful response; at least we're measuring the path MTU up to the failure point

When developing yet another Tcl script, I've stumbed across an interesting show command: the show ip access-list interface name introduced in IOS release 12.4(6)T displays the contents of the inbound and outbound IP access-list applied to the specified interface. The really nice part is that the ACL statistics (number of matches displayed next to the ACL lines) are kept and displayed per-interface.

I almost started writing an EEM applet that would detect and log the changes in router’s system time caused by NTP synchronizations, but then I’ve decided to check the IOS documentation first and found the ntp logging command.

The venerable rules used to establish OSPF router ID on Cisco IOS are all over the Internet:

• Take the highest IP address of all loopback interfaces configured on the router when the OSPF process is started.
• If there is no loopback interface, take the highest IP address of an operating interface.

In the old days, when Cisco believed that the router ID had to match an interface address, this also implied that the router ID would have changed if the interface IP address changed (and we told the students that you have to use loopback interfaces to make your network stable, as the OSPF process would restart if the interface giving the router ID went down).

Yesterday's post has generated quite a few comments (obviously a tool like this comes handy :); some of you were unable to run the .exe file I've provided, others wondered about the unexpected results. While testing the first issue, I've figured out that:

• Any C program compiled with the free Visual C++ compiler from Microsoft requires runtime library that has to be installed separately. Update: not completely true, if you use change the runtime library to the non-DLL version (Project properties/C++/Code generation tab), the exe size increases, but the external dependencies are removed.
• The Visual C++ 2008 that I've used has no publicly available runtime library that you could install.

So I had to scrap my VC++ 2008 installation, download VC++ 2005, reinstall the Microsoft Platforms SDK and (after a few hours) recompiled the program:

• The new build can be downloaded from the same location.
• It includes a README file that documents the changes made to the source.
• To run it, you have to install the VC++ 2005 runtime library from Microsoft

. Update (2007-10-03): I've rebuilt the image with static runtime library, so the VC++ runtime DLL is no longer needed. Thanks to Vladimir Kocjancic for figuring this out for me.

After these changes, the utility should be able to execute on Vista as well.

Apart from the rebuild, I've fixed the ICMP destination network unreachable handling, which is considered identical to successful ping in the MTU measurement code (I still need to fix its handling in the trace part of the code).

There are also a few caveats when using this program on a Windows platform enabled for Path MTU discovery (default for the last few years):

• Whenever the Windows TCP stack receives an ICMP specifying the maximum MTU, it caches the reported MTU size (makes sense).
• The cached MTU sizes eventually expire (but I was not able to find any documentation on the expiration time).
• I was also not able to find any documented way of purging the path MTU cache. The command that works for me is the route -f which flushes the IP routing table.
• Obviously, after executing route -f, the DHCP-installed default route is gone, so you have to execute ipconfig /renew.

Note: Any hints on the internal workings of path MTU cache on Windows platforms are highly appreciated