Category: WAN

I had to figure out various PPP parameters (and associated Cisco IOS behavior) and didn't have real dial-up equipment in my lab setup. I could have gone with PPPoE, but it turned out it's way simpler to emulate dialup connections (at least the PPP negotiations work as expected) on fixed serial lines. This is the minimum setup you need on the “caller” side …

interface Serial1/0
 ip address negotiated
 encapsulation ppp
 ppp authentication pap optional
 ppp pap sent-username client password 0 client

… and this is the “server”-side configuration:

interface Serial1/0
 ip address 10.0.0.33 255.255.255.252
 encapsulation ppp
 peer default ip address 10.0.0.34
 ppp authentication pap callin
!
username client password client

To trigger PPP negotiations, shut down and re-enable the serial interface on either side.

Note: As I'm using PAP authentication, I could use the more secure username secret configuration command, which would not work with CHAP.

Thinking about the implications of bit stuffing I wrote about in the SDLC post, I realized that long sequences of ones would be transmitted slower than long sequences of zeroes due to an extra bit being inserted after every fifth consecutive one. The theory would predict a 20% decrease in transmission speed.

Of course I wanted to test this phenomenon immediately. I connected two routers with a low-speed (64 kbps) link, and started a series of pings. Not surprisingly, the results confirmed the theory:

My recent post about problems with old modems has generated a lot of comments with some very useful ideas, but nobody addressed the question “why was a long string of ones not a problem?”, so let's start there. Almost all WAN synchronous protocols in use today are descendants of venerable SDLC invented by IBM more than 30 years ago. SDLC was later extended to support connectionless and balanced modes, resulting in HDLC. PPP is just an extension of HDLC, adding support for negotiations and standard layer-3 protocol demultiplexing. In SDLC, IBM also solved the frame delimiting and associated escape character problem inherent in previous protocols like BSC (DLE was used in BSC) by introducing bit stuffing: a zero would be inserted after five consecutive ones (and silently removed by the receiver) to differentiate the regular data stream from framing (six consecutive ones) and abort (more than six consecutive ones) sequences. Thus, the HDLC (or PPP) data stream can never contain more than six consecutive ones and the long sequences of ones never cause synchronization loss.

IBM obviously also had problems with bad modems and solved it with the NRZI encoding that was part of SDLC standard (and a major pain in the good old days when the appliques on the old Cisco routers did not support it and we've been trying hard to penetrate IBM accounts). You can still configure NRZI encoding on most routers' serial links (it might depend on the actual hardware platform) with the nrzi-encoding interface configuration command (you had to do it with jumpers in the AGS+). Incidentally, changing interface encoding to NRZI was really helpful when you had to break things in the preparation for the troubleshooting part of the original CCIE lab).

Enough theory, let's summarize the proposed solutions:

• The nrzi-encoding (if available) is the best one, as it reliably solves the problem, is transparent and does not incur additional overhead.
• Compression or encryption are OK, but they result in significant CPU overhead (unless you have hardware encryption/compression modules) and might (at least in theory) still produce a long sequence of zeroes, although with a very low probability. IPSec also introduces overhead due to additional IPSec headers.
• LFI (effectively multilink PPP over a single link) is also a good solution, as the PPP framing and MLPPP headers break the long sequences of zeroes (you might have to fine-tune the fragment size with ppp multilink fragment size configuration command), but it introduces overhead on the WAN link.
• IP fragmentation would work, but would be quite bandwidth-consuming. If the fragmentation would be performed by the router, the overhead would be 20 bytes per fragment (IP header), if the sending host performs the fragmentation, the overhead is 40 bytes per fragment for TCP sessions. For example, if we reduce the IP MTU size to 256 bytes, the TCP session overhead is over 18% (and we were scoffing at the ATM designers that made us live with 10% overhead).

There were also a few suggestions that would not work very well:

• The invert data command would only help if the modem has problems with long strings of zeroes, not with long strings of the same value.
• The tunnel key command just sets a 4-byte field in the GRE header but does not affect the encapsulated data at all.

William Chu sent me a working configuration he uses to measure jitter with the IP SLA tool and react to excessive jitter on the primary link. First you have to create the jitter probe with the IP SLA commands:

ip sla monitor 3000
 type jitter →
   dest-ipaddr 199.11.18.168 dest-port 12333 →
   source-ipaddr 199.11.18.169 codec g729a →
   codec-numpackets 100
 tos 184
 frequency 10

Note: The continuation character (→) indicates that the configuration command spans multiple lines

Next you have to define the IP SLA reaction to excessive jitter. William configured his router to react when the jitter exceeds 300 milliseconds and returns back to normal when the jitter falls below 290 milliseconds (some hysteresis is always a good thing).

ip sla monitor reaction-configuration 3000 →
  react MOS threshold-value 300 290 →
  threshold-type consecutive →
  action-type trapOnly

As the last step in the SLA configuration, you have to start the probe:

ip sla monitor schedule 3000 →
  life forever start-time now

After the SLA probe and out-of-bounds reaction have been configured, the router will generate syslog messages whenever the jitter gets above the threshold as well as when it falls below the second threshold. You can then use the EEM applets to act on the syslog messages:

event manager applet MOS-Below
 event syslog occurs 1 period 120 →
   pattern "Threshold below for MOS"
 ... actions ...
!
event manager applet MOS-Above
 event syslog occurs 1 period 120 →
   pattern "Threshold exceeded for MOS"
 ... actions ...

In the pre-DSL days, you had two options to get a short-haul high-speed link (at least in Europe): take E1 (or fractional E1) from a telecom (which was more expensive than a highway robbery, as the cost was recurring) or use baseband modems with proprietary encoding techniques on physical copper wires (assuming you could get them). As it turned out, some of these encoding techniques were not as good as the others (but the equipment was relatively cheap, so the budget limits usually forced the decision). We had our own share of modem-related problems, but they were never as bad as what I've heard from one of my students: his modems would lose synchronization when transmitting a long string of zeroes over a regular synchronous serial interface; ping ip 1.2.3.4 size 1000 data 0000 would be enough to bring down the link.

And now two questions for you:

• What could you do on the router to fix this problem?
• Why was the synchronization retained when transmitting a long string of ones?

Some fifteen years ago we were building a router-based network using primarily baseband modems (that's how the DSL boxes with symmetrical speeds were called back then). Everything worked great, we even had DECnet running between a few sites. However, after a few weeks, a mystery phenomena crept up: when the users were copying files between two VAX computers, the link between the sites went down … always when copying the same file.

To make the long story short: every modem uses a predefined sequence that enables remote loopback. These sequences are standardized and have been chosen so that the chance of hitting them with real traffic is close to zero (but obviously not zero). The file that the users were trying to copy contained just the correct sequence to trigger the remote loopback in one of the modems and the link went down (most probably changed state to looped) as the routers started to receive their own packets. Disabling remote loopback on the modem (a jumper in those days) solved the problem.

The moral of the story: whenever you use pattern matching to identify something (be it a specific application that you're trying to identify or a virus in your workstation), there is a non-zero chance of false positives, usually in the most unusual places.

Let's conclude this post with what seems to me a ludicrous “invention”: someone patented the flashing of LEDs when performing loopback tests on a modem. If anyone can survive reading the whole patent application, understand it and recognize its true added value, please let me know … I got completely lost.

In the “good old days” we've been teaching our students that although a router can act as a Frame Relay switch, it supports only the rudimentary functionality of switching the packets, but not the policing/marking features available in Frame Relay switches. That hasn't been true for a while - in IOS release 12.1T, Cisco has introduced the congestion management features. You can specify the congestion management per-interface (with the frame-relay congestion-management interface configuration command) and set the DE drop/ECN mark percentages for all PVCs on the interface or you can set the parameters within a map-class.

I don't know how useful this feature is to you; I was fond of finding it because it solves some interesting problems I had a (long) while ago. If you need more in-depth description or actual configurations, post a comment or send me a message.

I've always wanted a short summary display of DLCIs configured on my Frame Relay boxes (or whatever your favorite WAN technology is), but the only printout I would get from the router would be the lengthy show frame pvc printout. Fortunately, a judicious use of output filters can get you a summary printout from almost anything Cisco IOS produces.

Cisco IOS supported Frame Relay switching (emulation of a Frame Relay switch) for a very long time. First they've implemented local switching, then remote switching over a GRE tunnel. With the introduction of generic Layer 2 transport across a layer-3 backbone (L2TPv3 or AToM), Frame Relay switching got integrated into the new infrastructure, but never implemented completely ... that is, until release 12.0(27)S and 12.4(11)T which finally supports local switching in the new architecture. We've also got a few extra goodies: now you can do DTE-to-DTE switching (interconnecting two Frame Relay switches with a router) or same-port switching (switching two DLCIs terminating on the same router port).