The flash memory available in newer router platforms (at the very minimum the ISR routers and 37xx series) is capable of being used as a regular disk drive (for example, to store system logging information), but it might be formatted as a traditional Low-End File System (LEFS) flash card (more likely if the router was not manufactured recently). To change the flash card format to disk-like FAT32 format, use the format flash: privileged-level command (and don't forget to store the IOS image to another location before formatting the flash). After the format process is complete, you can create subdirectories on the flash: memory and use it as a regular disk device.

Continuing from the previous mturoute-related post, this is how the mturoute utility behaves when you start it in traceroute mode (with the -t flag):

• Similar to Windows tracert, it tries to find the successive hops in the path by sending ICMP echo packets with increasing values of TTL field.

Contrary to Cisco IOS and most Unix systems that send UDP packets to high-numbered ports, tracert uses ICMP echo packets.

• For each router found in the path (= source IP address in the ICMP TTL exceeded message), mturoute tries to find path MTU to that hop using the same algorithm as in the ping mode.
• During the bisecting phase, the mturoute does not print all the messages it prints in the ping mode, but just the cryptic signs (+/-/u/.) indicating its progress. Their meaning is documented in the previous post.
• After the path MTU to the router under investigation is measured, mturoute reports the router's IP address and path MTU.

A few days ago, Jeremy Stretch asked me whether there's a command to display packet lists attached to router's interfaces. While he got pretty far with the output filters, he would like to have a nice tabular format as well as the contents of the access lists displayed next to the interfaces. The show ip access-list interface name command comes pretty close, but it displays the information only for a single interface, so it was time to write another Tcl script. To install it on your router:

1. Download it from my web site and copy it to your router's flash or NVRAM.
2. Define an alias, for example alias exec filters tclsh flash:packetFilters.tcl.

The script recognizes two parameters: the all parameter displays all interfaces, including ones with no access lists and the verbose parameter displays the contents of the access list after the interface name.

I always thought that the new format of the Cisco web-based documentation was awful, as it consumes way more bandwidth than the old version and is slower to load over low-speed links as it displays the text only after the complete page is loaded due to heavy use of table-based HTML layout (I will refrain from commenting the use of this layout technique in the third millenium). However, the new content structure has some significant benefits; for example, all the 12.4T feature guides are collected on a single page … fantastic if you try to find a feature that you remember was implemented somewhere in 12.4T track.

Jeff West has asked me to document the printout produced by the mturoute utility. Here's the first part of the documentation.

mturoute works in two modes:

• Without the -t flag, it sends variable-lenght ICMP echo packets to the specified destination address, trying to figure out the largest packet that is successfully propagated to the destination.
• With the -t flag, it uses traceroute-like algorithm to find the hop-by-hop IP addresses (the source IP addresses of the ICMP TTL exceeded replies) and uses the same packet-size-calculating algorithm to measure the path MTU to each hop.

Today we'll focus on the non-trace mode. It tries to measure the path MTU with a bisection method varying the packet sizes between minimum MTU (92) and maximum MTU (specified with the -m parameter, default is 10000 bytes). The payload size of the first packet (without the -m flag) is thus 5046 bytes ((10000 + 92)/2).

On each iteration, the algorithm prints a “cryptic” sign indicating whether the ping with the current payload size succeeded. The following indications are given:

• '+': ICMP echo reply arrived
• '-': The ping failed (for various reasons, including exceeding the path MTU)
• 'u': ICMP destination unreachable response arrived, indicating blackhole or access-list.

ICMP unreachable is considered a successful response; at least we're measuring the path MTU up to the failure point

• '.': timeout. The ping will be retried up to three times (or the number specified with the -r option).

In the ping mode, mturoute prints the current ICMP payload size at each step, resulting in a printout similar to the one below. If you'd have specified the -d option, the printout would include detailed status codes from the IcmpSendEcho function.

$ mturoute 10.0.3.3
* ICMP Fragmentation is not permitted. *
* Maximum payload is 10000 bytes. *
- ICMP payload of 5046 bytes failed..
- ICMP payload of 2569 bytes failed..
+ ICMP payload of 1330 bytes succeeded.
- ICMP payload of 1949 bytes failed..
- ICMP payload of 1639 bytes failed..
- ICMP payload of 1484 bytes failed..
+ ICMP payload of 1407 bytes succeeded.
- ICMP payload of 1445 bytes failed..
+ ICMP payload of 1426 bytes succeeded.
+ ICMP payload of 1435 bytes succeeded.
+ ICMP payload of 1440 bytes succeeded.
+ ICMP payload of 1442 bytes succeeded.
+ ICMP payload of 1443 bytes succeeded.
+ ICMP payload of 1444 bytes succeeded.
+ ICMP payload of 1444 bytes succeeded.
Path MTU: 1472 bytes.

Note: To use the debug-enabled version of mturoute, or the version that does not need VC++ runtime, download the new ZIP archive from my web site.

When developing yet another Tcl script, I've stumbed across an interesting show command: the show ip access-list interface name introduced in IOS release 12.4(6)T displays the contents of the inbound and outbound IP access-list applied to the specified interface. The really nice part is that the ACL statistics (number of matches displayed next to the ACL lines) are kept and displayed per-interface.

I almost started writing an EEM applet that would detect and log the changes in router’s system time caused by NTP synchronizations, but then I’ve decided to check the IOS documentation first and found the ntp logging command.

The venerable rules used to establish OSPF router ID on Cisco IOS are all over the Internet:

• Take the highest IP address of all loopback interfaces configured on the router when the OSPF process is started.
• If there is no loopback interface, take the highest IP address of an operating interface.

In the old days, when Cisco believed that the router ID had to match an interface address, this also implied that the router ID would have changed if the interface IP address changed (and we told the students that you have to use loopback interfaces to make your network stable, as the OSPF process would restart if the interface giving the router ID went down).

Yesterday's post has generated quite a few comments (obviously a tool like this comes handy :); some of you were unable to run the .exe file I've provided, others wondered about the unexpected results. While testing the first issue, I've figured out that:

• Any C program compiled with the free Visual C++ compiler from Microsoft requires runtime library that has to be installed separately. Update: not completely true, if you use change the runtime library to the non-DLL version (Project properties/C++/Code generation tab), the exe size increases, but the external dependencies are removed.
• The Visual C++ 2008 that I've used has no publicly available runtime library that you could install.

So I had to scrap my VC++ 2008 installation, download VC++ 2005, reinstall the Microsoft Platforms SDK and (after a few hours) recompiled the program:

• The new build can be downloaded from the same location.
• It includes a README file that documents the changes made to the source.
• To run it, you have to install the VC++ 2005 runtime library from Microsoft

. Update (2007-10-03): I've rebuilt the image with static runtime library, so the VC++ runtime DLL is no longer needed. Thanks to Vladimir Kocjancic for figuring this out for me.

After these changes, the utility should be able to execute on Vista as well.

Apart from the rebuild, I've fixed the ICMP destination network unreachable handling, which is considered identical to successful ping in the MTU measurement code (I still need to fix its handling in the trace part of the code).

There are also a few caveats when using this program on a Windows platform enabled for Path MTU discovery (default for the last few years):

• Whenever the Windows TCP stack receives an ICMP specifying the maximum MTU, it caches the reported MTU size (makes sense).
• The cached MTU sizes eventually expire (but I was not able to find any documentation on the expiration time).
• I was also not able to find any documented way of purging the path MTU cache. The command that works for me is the route -f which flushes the IP routing table.
• Obviously, after executing route -f, the DHCP-installed default route is gone, so you have to execute ipconfig /renew.

Note: Any hints on the internal workings of path MTU cache on Windows platforms are highly appreciated

I wanted to get in-depth details on how various MTU parameters interact in GRE/IPSec/MPLS environment. Before going into router configuration details, I wanted to have a tool that would reliably measure actual path MTU between the endpoints. After a while, Google gave me a usable link: supposedly the tracepath program on Linux does what I needed. As I'm a purely Windows user (for me, PCs are just a tool), I needed a Windows equivalent … and found mturoute, the utility that does exactly what I was looking for.

The MPLS VPN implementation on Cisco IOS has always allowed you to create VRF static routes that pointed to interfaces belonging to other VRFs. The feature can be used to implement interesting overlapping VPN (or common services VPN) designs, some of which are explained in the MPLS and VPN Architectures books.

However, quite often the ability to create inter-VRF static routes is considered a major security problem, as an operator configuration error could establish undesired inter-VPN connectivity. In these cases, use the no ip route static inter-vrf configuration command to prevent such routes from being installed in the VRF routing table.

You might also want to read a good explanation of MPLS VPN route leaking from Cisco systems

Shaun needed an extra TFTP server in CCNP labs and asked whether you could use a router to act as one. The read-only (download only) TFTP functionality has been available in Cisco IOS for a long time, but the common wisdom was that you could only use the TFTP server function to serve current IOS image.

Fortunately, as of IOS 11.0, the function is more generic; you can serve any file residing on the router (you still cannot upload files), but you have to declare each file to be served with the tftp-server path global configuration command. You could even specify an alias to have the file available under a different name and attach an access list to each configured file to restrict its availability.

Note: This article is part of You've asked for it series.

Just found a few interesting articles in the Cisco blog:

• How to apply QoS policies based on time-of-day
• Some more Cisco IOS CLI and configuration tips & tricks

If you use EEM applets with action syslog commands, the messages generated by those commands will be displayed in the same format as the debugging messages, regardless of their severity. This might be a problem if you decide to use different timestamps for debugging and regular syslog messages (or disable debugging timestamps completely).

In case you've missed yesterday's post … the weather was just way too good to stay in the office :) However, even if I would decide to work on my routers, I could take them with me (well, the laptop would be a bit heavy and the sun was too bright) thanks to Christophe Fillot (Dynamips) and Greg Anuzelli (Dynagen).

In case you haven't heard about Dynamips/Dynagen yet: Dynamips emulates a variety of IOS platforms (from 2600 to 7200) on Intel platform and Dynagen provides friendlier user interface (more than friendly enough for me, probably too cryptic for GUI addicts). I've seen Dynamips a year or two ago, checked what it can do and decided to stay with the real routers in a remote lab environment. In the meantime, the software has improved drastically, allowing you to test all sorts of IOS features and topologies, as long as you don't expect QoS to work or real-time features to act in real-time (simulation is, after all, a bit slower than the real life).

To start using this tool, download it from dynagen.org, read the tutorial and you're in business. I will also start providing more interesting scenarios in the dynagen configuration file format.

Let me conclude with a few tips:

• If you don't need 7200-specific features, select 37xx or 26xx platform, it consumes less virtual memory per router.
• Setting idlepc is mandatory if you want to have decent response. Read the tutorial, the idlepc section is great.
• Unzip the IOS files. With uncompressed files, the routers are ready to be configured in under a minute on my laptop; if the IOS image is compressed, it takes several minutes.
• If you have larger topologies, use GhostIOS and Sparsemem features.
• Reduce the size of NVRAM and Flash to minimum that would work. These are stored as persistent files on your disk; you can have 256MB Flash if you want, but then you'll have 256MB less of your hard drive (per router).

With all the above-mentioned features enabled, I was easily running eight 3700-series routers on my laptop (IBM T60).