Jason Edelman wrote a great blog post after watching Ethan Banks struggle with yet another multi-vendor IPsec deployment. Some of his ideas make perfect sense (wiki-like web site documenting working configurations between vendor X and Y for every possible X and Y), others less so (tunnel broker – particularly in view of recent Tor challenges), but let’s step back a bit and ask ourselves “Why is IPsec so complex?”

Whatever happened to running code? IETF standard were supposed to be based on running code. That might have been the case 30 years ago; I’m positive some of the standards we love to hate wouldn’t be around if running code would be the first step in standard development process.

Where’s the operational experience? Looking from the sidelines it seems that vendors lead a lot of standard developments (because their engineers are the only ones with time to work on IETF documents, argue in mailing lists, and have the budget to travel all across the globe to attend IETF meetings).

Engineers with operational experience are too busy running their networks, and get involved when it’s way too late (see also: IPv6). Making matters worse, their complaints usually get shot down with flames from ivory towers (including: you don’t get it, you’re stuck with old mentality, your use case isn’t relevant, your idea violates the shiny principles…).

Exploring the solution space. Instead of focusing on “a solution is done when there’s nothing to cut away”, we often get a designed-by-a-committee “solution is done when we cannot add another option to it” (example: MPLS-over-UDP because MPLS-over-GRE is so last millennium).

Partial implementations. IETF documents are full of SHOULDs and MAYs, begging the vendors to implement only parts of them … and then there are people deliberately messing up the standards.

Making matters worse, multiple (incompatible) drafts document similar functionality. Not surprisingly, a multi-vendor deployment becomes an exercise in futility when a desperate engineer tries to find the maximum subset of shared functionality. Dial tone signaling in SIP comes to mind, as does IPsec. This is what Bruce Schneier had to say about IPsec complexity:

Our main criticism of IPsec is its complexity. IPsec contains too many options and too much flexibility; there are often several ways of doing the same or similar things. This is a typical committee effect. Committees are notorious for adding features, options, and additional flexibility to satisfy various factions within the committee. As we all know, this additional complexity and bloat is seriously detrimental to a normal (functional) standard. However, it has a devastating effect on a security standard.

Clueless designs. We don’t have time for reading the standards, trying to understand the technology, and building designs and architectures on sound foundations.

It’s way faster to read a whitepaper or two, throw a few icons together in PowerPoint or Visio, and pass the buck to the deployment team.

The Nerd Knobs. A technology eventually meets reality and has to be implemented, at which point all of the above strikes at the poor implementation/operations team.

If their company bought enough equipment to have significant leverage with the vendor, they can force the vendor to implement a knob (or two … or a dozen) to get their design working (hint: check the hundreds of VoIP “features” added to Cisco IOS). Welcome to the wonderful world of nerd knobs.

Not surprisingly, the nerd knobs usually work well only in a particular scenario they were designed to solve and get used in all sorts of weird ways. They might also introduce non-standard behavior (because they had to be put in place to make a standard work in real life), making multi-vendor deployments even more fun.

Can we do any better?

It’s hard to stop a tanker by swimming against it, but we can try. When was the last time you said NO to a shiny new technology and tried to solve a problem using the tools that have been proven to work?