War story: almost zero is not good enough

Some fifteen years ago we were building a router-based network using primarily baseband modems (that's how the DSL boxes with symmetrical speeds were called back then). Everything worked great, we even had DECnet running between a few sites. However, after a few weeks, a mystery phenomena crept up: when the users were copying files between two VAX computers, the link between the sites went down … always when copying the same file.

To make the long story short: every modem uses a predefined sequence that enables remote loopback. These sequences are standardized and have been chosen so that the chance of hitting them with real traffic is close to zero (but obviously not zero). The file that the users were trying to copy contained just the correct sequence to trigger the remote loopback in one of the modems and the link went down (most probably changed state to looped) as the routers started to receive their own packets. Disabling remote loopback on the modem (a jumper in those days) solved the problem.

The moral of the story: whenever you use pattern matching to identify something (be it a specific application that you're trying to identify or a virus in your workstation), there is a non-zero chance of false positives, usually in the most unusual places.

Let's conclude this post with what seems to me a ludicrous “invention”: someone patented the flashing of LEDs when performing loopback tests on a modem. If anyone can survive reading the whole patent application, understand it and recognize its true added value, please let me know … I got completely lost.

4 comments:

  1. But it has *always* been that way, Ivan - since waaaay back to modems and the "+++" escape sequence

    http://en.wikipedia.org/wiki/
    Command_and_Data_modes_%28modem%29

    And actually, the moral of the story is: please, no CONTROL over the DATA channel :)

    We could argue that modern devices run control and data over the same media (haven't yet seen a router with ethernet interfaces for forwarding packets, and ethernet interfaces for routing protocol ;)), but those are different protocols sharing the same media - routers don't try to find LSA updates in the middle of the DATA stream for an FTP transfer (or not that I know ;))

    ReplyDelete
  2. But many smart L2 switches look into ethernet frames for IGMP snooping...

    ReplyDelete
  3. @Anonymous(1): You're right, the +++ was hidden somewhere deep in my subconsciousness. But at least that was a well known "feature" introduced because someone somewhere couldn't get his modem signal handling right (if you were using the DTR correctly, you could disconnect the call simply by dropping it, no need for an escape sequence). The synchronous modems were always believed to be better than that ;)

    @Anonymous(2) (gee, it's hard to tell you guys apart, you all look the same :) The IGMP snooping is well-controlled activity. The switch knows what Ethernet frames are, knows which frames are IGMP frames and knows where to look and how to interpret the data ... whereas the stupid modem just latched onto whatever looked like the right bitstream having no clue about frames or anything above them.

    ReplyDelete
  4. Ivan, anything to make you happy - now you will know "Anonymous 1", aka the "+++" guy as "BlueDemon" - not because I'm blue or a Demon, but just because I happen to be drinking a can of Full Throttle Blue Demon - drink that I highly recommend - as it contains caffeine, niacin, vitamins B6 and B12 and tastes way better than Red Bull ;)

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.