Blog Posts in March 2011

Open Networking Foundation – Fabric Craziness Reaches New Heights

Some of the biggest buyers of the networking gear have decided to squeeze some extra discount out of the networking vendors and threatened them with open-source alternative, hoping to repeat the Linux/Apache/MySQL/PHP saga that made it possible to build server farms out of low-cost commodity gear with almost zero licensing costs. They formed the Open Networking Foundation, found a convenient technology (OpenFlow) and launched another major entrant in the Buzzword Bingo – Software-Defined Networking (SDN).

Networking vendors, either trying to protect their margins by stalling the progress of this initiative, or stampeding into another Wild West Gold Rush (hoping to unseat their bigger competitors with low-cost standard-based alternatives) have joined the foundation in hordes; the list of initial members (see the press release for details) reads like Who’s Who in Networking.

read more see 10 comments

NAT-PT is dead! Long live NAT-64!

I’m getting questions like this one all the time: “Where are we with NAT-PT? It was implemented in IOS quite a few years ago but it has never made it into ASA code.

Bad news first: NAT-PT is dead. Repeat after me: NAT-PT is dead. Got it? OK.

More bad news: NAT-PT in Cisco IOS was seriously broken after they pulled fast switching code out of IOS. Whatever is left in Cisco IOS might be good enough for a proof-of-concept or early deployment trials, but not for a production-grade solution.

read more see 8 comments

MPLS/VPN-over-GRE-over-IPSec: Does It Really Work?

Short answer: yes, it does.

During the geeky chat we had just after we’d finished recording the Data Center Fabric Packet Pushers podcast, Kurt (@networkjanitor) Bales asked me whether the MPLS/VPN-over-DMVPN scenarios I’m describing in Enterprise MPLS/VPN Deployment webinar really work (they do seem a bit complex).

I always test the router configurations I use in my webinars and I usually share them with the attendees. Enterprise MPLS/VPN Deployment webinar includes a complete sets of router configurations covering 10 scenarios, including five different MPLS/VPN-over-DMVPN designs, so you can easily test them in your lab and verify that they do work. But what about a live deployment?

read more see 15 comments

TRILL/Fabric Path – STP integration

Every Data Center fabric technology has to integrate seamlessly with legacy equipment running the venerable Spanning Tree Protocol (STP) or one of its facelifted incarnations (for example, RSTP or MST). The alternative, called rip-and-replace when talking about other vendors’ boxes or synchronized upgrade when promoting your wares (no, I haven’t heard it yet, but I’m sure it’s coming), is simply indigestible to most data center architects.

TRILL and Cisco’s proprietary Fabric Path take a very definitive stance: the new fabric is the backbone of the network routing TRILL-encapsulated layer-2 frames across bridged segments (TRILL) or contiguous backbone (Fabric Path). Both architectures segment the original STP domain into small chunks at the edges of the network as shown in the following figure:

read more see 2 comments

IPv6-Enabling Your Legacy Applications with F5 BIG-IP LTM

In every enterprise-focused IPv6 presentation, including my Enterprise IPv6 – the first steps webinar, I’m telling the attendees that they can easily make their legacy applications reachable over IPv6 with a little help from F5 load balancers. After all, Facebook is doing exactly that, so it should work (in theory)… but as we all know, in practice, the theory and practice are wildly different.

read more see 2 comments

Don’t Try to Fake Multi-chassis Link Aggregation (MLAG)

Martin sent me an interesting challenge: he needs to connect an HP switch in a blade enclosure to a pair of Catalyst 3500G switches. His Catalysts are not stackable and he needs the full bandwidth between the switches, so he decided to fake the multi-chassis link aggregation functionality by configuring static LAG on the HP switch and disabling STP on it (the Catalysts have no idea they’re talking to the same switch):

read more see 10 comments

IPv6 security issues: Fixing implementation problems

Let’s assume we’re all past the IPv6 myths phase and know that IPv6 does not offer more (or less) inherent security than IPv4. Will the IPv6 networks be as secure as IPv4 ones? Not necessarily, because we’re lacking feature parity and implementation experience. As I explained in the “IPv6 security issues: Fixing implementation problems” I wrote for SearchTelecom:

Until equipment vendors fill in the gaps and offer true feature parity between IPv4 and IPv6 security features, we can expect the IPv6 networks to be less secure that today’s IPv4 networks -- not because IPv6 is insecure, but because today’s IPv6 implementations still lag behind their IPv4 counterparts.

Read more @ SearchTelecom (or consider the excellent IPv6 Security book by Eric Vyncke).

see 1 comments

You can't ignore IPv6 any longer (in seven steps)

We know the world will eventually run out of IPv4 addresses, but while at least some service providers got the message and already deployed IPv6, it seems like most enterprise IT departments still practice the denial strategy. It’s worrisome to read articles from Jeff Doyle describing the ignorance of his enterprise clients, so I’ll try (yet again) to explain why you should start IPv6 planning NOW.

read more see 14 comments

Does Bridge Assurance Make UDLD Obsolete?

I got an interesting question from Andrew:

Would you say that bridge assurance makes UDLD unnecessary? It doesn't seem clear from any resource I've found so far (either on Cisco's docs or on Google)."

It’s important to remember that UDLD works on physical links whereas bridge assurance works on top of STP (which also implies it works above link aggregation/port channel mechanisms). UDLD can detect individual link failure (even when the link is part a LAG); bridge assurance can detect unaggregated link failures, total LAG failure, misconfigured remote port or a malfunctioning switch.

read more add comment

Get the right troubleshooting tools for the job

A while ago Matthew Norwood wrote an excellent article describing the troubleshooting process they used to figure out why a particular web application worked way too slowly. Greg Ferro was quick to point out that it doesn’t make sense to assume the network is the problem and work through the whole chain slowly eliminating every potential networking device as the source of the problem when you might be facing an application design issue. However, there’s an even more important consideration: your network troubleshooting toolbox lacks the right troubleshooting tools for this job.

read more see 9 comments

Ensuring multi-tenant security in cloud services

One of the interesting problems I was facing in the recent weeks was multi-tenant security. Combine it with fuzzy all-encompassing vapor-based terminology and you have a perfect mix that can fit anything you want to sell. In the Ensuring multi-tenant security in cloud services I wrote for I tried to structure the cloudy visions a bit: let’s figure out which type of service we’re talking about, then we can discuss what security mechanisms make sense.

As you might expect, I find IaaS the most challenging as you’re bound to hit a number of roadblocks, from VLAN limitations to architectural limitations of virtual security appliances.

Read more @ SearchTelecom ...

see 2 comments

Framed-IPv6-Prefix used as delegated DHCPv6 prefix

Chris Pollock from io Networks was kind enough to share yet another method of implementing DHCPv6 prefix delegation on PPP interfaces in his comment to my DHCPv6-RADIUS integration: the Cisco way blog post: if you tell the router not to use the Framed-IPv6-Prefix passed from RADIUS in the list of prefixes advertised in RA messages with the no ipv6 nd prefix framed-ipv6-prefix interface configuration command, the router uses the prefix sent from the RADIUS server as delegated prefix.

This setup works reliably in IOS release 15.0M. 12.2SRE3 (running on a 7206) includes the framed-IPv6-prefix in RA advertisements and DHCPv6 IA_PD reply, totally confusing the CPE.

read more see 10 comments

Delegated IPv6 prefixes – RADIUS configuration

In the Building Large IPv6 Service Provider Networks webinar I described how Cisco IOS uses two RADIUS requests to authenticate an IPv6 user (request#1) and get the delegated prefix (request#2). The second request is sent with a modified username (-dhcpv6 is appended to the original username) and an empty password (the fact that is conveniently glossed over in all Cisco documentation I found).

FreeRADIUS server is smart enough to bark at an empty password, to force the RADIUS server to accept a username with no password you have to use Auth-Type := Accept:

Site-A-dhcpv6   Auth-Type := Accept
cisco-avpair = "ipv6:prefix#1=fec0:1:2400:1100::/56"
read more see 6 comments

Internet morons are so amazing!

A few days ago I used Google to search for an article I’d written. My article was among the top results, but there was also another web site with very similar text. I’m used to blockheads publishing content stolen from my RSS feed (which is one of the primary reasons you won’t see a full feed of my blog any time soon), but this guy seemed to be copying the whole articles ... only they sounded somewhat crazy. For example,

Yesterday I described how the IPv6 architects split the functionality of IPCP into three different protocols (IPCPv6, RA and DHCPv6). While the split undoubtedly makes sense from the academic perspective, the service providers offering PPP-based services (including DSL and retrograde uses of PPP-over-FTTH) went berserk.

... became ...

Yesterday we dеѕсrіbеd hοw thе IPv6 architects rip thе functionality οf IPCP іntο 3 odd protocols (IPCPv6, RA аnd DHCPv6). Whіlе thе rip positively mаkеѕ clarity frοm thе educational perspective, thе use providers charity PPP-based services (counting DSL аnd opposing uses οf PPP-over-FTTH) wеnt berserk.
read more see 17 comments

Don’t lie about proprietary protocols

A few months ago Brocade launched its own version of Data Center Fabric (VCS) and the VDX series of switches claiming that:

The Ethernet Fabric is an advanced multi-path network utilizing an emerging standard called Transparent Interconnection of Lots of Links (TRILL).

Those familiar with TRILL were immediately suspicious as some of the Brocade’s materials mentioned TRILL in the same sentence as FSPF, but we could not go beyond speculations. The Brocade’s Network OS Administrator’s Guide (Supporting Network OS v2.0, December 2010) shows a clear picture.

read more see 9 comments

The Data Center Fabric architectures

Have you noticed how quickly fabric got as meaningless as switching and cloud? Everyone is selling you data center fabric and no two vendors have something remotely similar in mind. You know it’s always more fun to look beyond white papers and marketectures and figure out what’s really going on behind the scenes (warning: you might be as disappointed as Dorothy was). I was able to identify three major architectures (at least two of them claiming to be omnipotent fabrics).

Business as usual

Each networking device (let’s confuse everyone and call them switches) works independently and remains a separate management and configuration entity. This approach has been used for decades in building the global Internet and thus has proven scalability. It also has well-known drawbacks (large number of managed devices) and usually requires thorough design to scale well.

read more see 19 comments

DHCPv6-RADIUS integration: the Cisco way

Yesterday I described how the IPv6 architects split the functionality of IPCP into three different protocols (IPCPv6, RA and DHCPv6). While the split undoubtedly makes sense from the academic perspective, the service providers offering PPP-based services (including DSL and retrograde uses of PPP-over-FTTH) went berserk. They were already using RADIUS to authenticate PPP users ... and were not thrilled by the idea that they should deploy DHCPv6 servers just to make the protocol stack look nicer.

read more see 4 comments


Last week I got an interesting tweet: “Hey @ioshints can you tell me what is the radius parameter to send ipv6 dns servers at pppoe negotiation?” It turned out that the writer wanted to propagate IPv6 DNS server address with IPv6CP, which doesn’t work. Contrary to IPCP, IPv6CP provides just the bare acknowledgement that the two nodes are willing to use IPv6. All other parameters have to be negotiated with DHCPv6 or ICMPv6 (RA/SLAAC).

The following table compares the capabilities of IPCP with those offered by a combination of DHCPv6, SLAAC and RA (IPv6CP is totally useless as a host parameter negotiation tool):

read more see 2 comments