Blog Posts in November 2008
I received an interesting comment on one of my knowledge/certification-related posts:
I used to think that certifications were a useful indicator of knowledge or at least initiative, but I’m changing my mind. [...] I feel like I’ve gotten a lot out of studying for certifications, especially CCIE, but I’m starting to wonder if that’s the exception.
I guess a lot of prospective internetworking engineers are thinking along the same lines, so here’s my personal perspective on this issue.
The Network World recently published a story describing the results of an independent security product testing lab, where they’ve discovered (surprise, surprise) that adding security features to Cisco routers “presents a tremendous bottleneck” and “can turn a 60G router into a 5G one or even a 100M bit/sec device”.
The test results haven’t been published yet; I’ve got all the quotes from the NW story, so they might be the result of an ambitious middleware.
We don’t need “independent experts” for that. Anyone who has ever configured VPNs in a high-speed environment can tell you how to kill the performance. The basics are always the same: make sure the dedicated silicon can’t handle the job, so the packets have to be passed to the CPU. Here are a few ideas:
You might remember that the last polls I did using Blogger all resulted in every option having exactly the same number of votes. At that time, I've blamed Google ... and I have to apologize. It was obviously someone who has nothing better to do in his life. The log files I've collected indicate he's coming from Poland and I would appreciate if my Polish readers could help me persuade this troubled individual that he should spend his time doing something else (details in the rest of the post).
The post describing my ideas about interactive online sessions resulted in a few comments and several off-line suggestions. Unfortunately most of the suggestions you’ve made in the comments are too generic. Remember, I was talking about 30-60 minute sessions and some suggestions would easily fill a week’s worth of training at the level of detail I’m aiming at. Running high-level introductory sessions is not my idea of fun; you could get as many of them as you want at Networkers.
Several suggestions are still “in the pipeline”: I have to envision how to structure them to make them manageable. In the meantime, the rest of the post lists the topics we can definitely cover. Please vote on them, the most popular one will be featured in December session.
I was delighted when I got access to Cisco's ACE XML Gateway/Web Application Firewall (WAF) box. This box is the perfect intersection of three fields I'm really interested in: networking, security and web programming, so I'll work with it quite a lot in the future and post interesting tips and tricks about its usage.
As this blog is currently focused exclusive on Cisco IOS, I'm wondering how to cover these new products. I won't create another blog; it simply doesn't make sense to build another blog from the ground up, but there are a few other options. Please help me select the best one by voting in the poll.
The November Technical Services News from Cisco included the Annotating Troubleshooting Sessions document from the Cisco’s support wiki. The document describes two well hidden features of Cisco IOS:
- The send log exec-level command writes a line in the syslog, allowing you to delineate logging or debugging outputs.
- The exclamation mark used as the first character in any IOS command line (not just in the configuration) serves as a comment. If you’re logging the TTY session, you can use these comments to document the session.
When I chose the word “unfortunately” in my post describing how Cisco IOS performs DNS lookup when you enter a host name in an access list, I’ve triggered several responses that disagreed with my choice of words. Here’s why I still think IOS ACL could be improved with dynamic DNS lookup:
When I was configuring the access list that should prevent spammers from misusing my workstations, I obviously had to figure out the IP address of the ISP’s SMTP server (access lists and object groups accept IP addresses). I almost started nslookup on my Linux workstation, but then decided to try entering a hostname in an IOS ACL … and it works. Unfortunately, IOS performs a DNS lookup when you enter the hostname (assuming you have configured the ip name-server) and stores the resulting IP address in the ACL definition:
rtr(config)#ip access-list extended InsideList
rtr(config-ext-nacl)#permit tcp any host smtp.example.com eq smtp
Translating "smtp.example.com"...domain server (192.168.0.1) [OK]
rtr(config-ext-nacl)#do show access-list InsideList
Extended IP access list InsideList
10 permit tcp any host 192.168.2.3 eq smtp
You can enter hostnames in ACLs or network object groups. In both cases, the name is immediately translated into an IP address.
We’ve spotted some of our best engineers when they were in the final years of their undergraduate studies. To continue the trend, NIL offers a student-engagement program that attracts highly promising candidates each year. They offer them CCNA training (after which the students have to pass the exam), a few weeks of hands-on instructor-led introductory bootcamps and the first CCNP course. These training courses should give students a solid foundation and a framework that they can expand on their own—which is the point where it's time to stress-test them with advanced bootcamps.
My MPLS QoS: Implementing the best model for guaranteed service article published by SearchTelecom gives you a high-level overview of the pipe and hose QoS models in the MPLS VPN environment. I’m also describing basic DiffServ QoS mechanisms available in an MPLS backbone.
If you’re new to IP QoS, you should start with the IP QoS: Two generations of class-of-service tools article.
In mid-December, I’ll do my first IOS Hints Online Session. These sessions will be short (30-60 minutes), very interactive (I hope, but that’s your choice) and focused on an interesting design/deployment aspect. The description of the design/deployment challenge addressed by the session will be available well in advance at the time when you’ll be able to register.
Each session will start with a few diagrams explaining the proposed solution to the session’s topic and continue with hands-on explanation on actual devices. Each session will be limited to ~15 participants who will be able to actively participate, ask questions, propose alternative solutions or even discuss their actual issues (assuming they are somewhat related to the primary topic of the session).
I have a “few” ideas what could be covered in these sessions, but having a real-life challenge coming from the readers of my blog would be much better. If you have a good idea that could fit into this concept, please send me a short description before Friday, November 21st. I’ll collect the best ones, publish short descriptions in a blog post and you’ll prioritize which ones you’d like to see first.
I always thought that there was no need to restrict outbound sessions across a firewall in low-security environments. My last encounter with malware has taught me otherwise; sometimes we need to protect the rest of the Internet from our clumsiness. OK, so I decided to install an inbound access-list on the inside interface of my SOHO router that will block all SMTP traffic not sent to a well-known SMTP server (and let the ISP’s SMTP server deal with relay issues).
The control and management planes in a network device run numerous mission-critical processes, including routing protocols and network management services (SNMP, telnet or SSH access to the router, web access to the router), and is thus the most vulnerable part of any network device.
A determined attacker can quickly overload the CPU of any router (or switch) with a targeted denial-of-service attack, either by sending IP packets that are propagated from the switching fabric (or interrupt code on software-only platforms) to the control plane processes or by targeting individual services running on the router. The situation is becoming worse with widespread use of high-speed hardware switching platforms that are connected to an underpowered CPU over a PCI bus; getting enough traffic to a network device to saturate the ASIC-to-CPU connection, or the CPU, is becoming trivial.
Reading the stories of Windows workstations becoming members of a spam botnet becomes way less enjoyable when you’re faced with the same problem (one of my kids managed to install a Trojan). It took me a day to clean the infected computer (it would have been easier to just format it, but the repeated installation of the Windows XP + Office software is so boring), but I’ve learned a few interesting networking lessons in the process that I’ll document in the next days.
I was very anxious to get my copy of Cisco Secure Firewall Services Module (FWSM) from Cisco Press, as I’m a purely router-focused person, and I wanted to understand the capabilities of the Firewall Services Module (PIX/ASA-like blade for the Catalyst 6500 switching system with virtual firewall capability). I have a good background in IOS-based firewalls and network address translation (NAT), so the book was a perfect fit for me. However, if you’re looking for “best practices for securing networks with FWSM,” you’ve been misled by the subtitle.
One of my kids managed to get infected with a particularly sneaky Facebook Trojan: a link from a friend (probably also infected) pointed to a web page with a video that required installation of a newer version of the Flash player … which was actually the first part of the Trojan. It quickly downloaded a few more components and made itself cozy deep within Windows XP.
Before you start telling me that kids would click anything … we had “a few” not so very pleasant discussion after previous infections and they know not to open anything or click on something that looks strange. Unfortunately the update-happy industry has conditioned them to constant prompts to upgrade one or another component and the request to upgrade the Flash player was obviously too legitimate-looking.
As always, Jeremy Stretch posted several interesting articles: how to hijack HSRP, introduction to split horizon in distance vector routing protocols and (long needed) default redistribution metrics.
Petr Lapukhov started playing with HTTP URL regular expressions within NBAR and documented his findings. The most interesting is the last Q/A pair: can I use NBAR as a content filtering engine?
And last but definitely not least, if you’re worried what will happen to WPA2 now that WPA has been cracked, Robert Graham explains the fundamental differences between WPA and WPA2. Also, make sure you read the detailed explanation of the WPA flaw to understand its implications.
BFD is one of those simple ingenious ideas that make you wonder “Why did it take them so long to figure this out?” It’s a UDP-based protocol that replaces dozens of link-level failure-detection mechanisms and routing protocol tweaks with a simple, focused solution: detect hop-by-hop layer-3 failures.
In November 2008 IP corner article I described BFD principles, its configuration on Cisco IOS and give you practical examples how you can use BFD to improve next-hop failure detection. You’ll find the link to the article somewhere in this list.
For more details on how BFD interacts with the routing protocols watch the How Networks Really Work webinar.
I’ve received almost a dozen responses to the second OSPF challenge, most of them correct. The key to the solution is the way OSPF checks neighbor’s IP address on point-to-point links (we already know that the subnet mask is ignored):
- If the interface is unnumbered, the router ignores the source IP address in the OSPF hello packets.
- If there’s an IP address configured on the interface, the router checks that the neighbor’s IP address (the source IP address in the OSPF hello packets) belongs to the same subnet. If the source IP address is not in the same subnet, the OSPF hello packet is ignored.
This post brought back some ancient memories … and I’m always amazed how far we’ve got in the last 30 years. For me, it all started with an IBM 360, having 48K (forty eight kilobytes) of core memory in which it ran an operating system and three user partitions. Fortran IV was the only programming language and card reader the only input device.
Moving to a VAX 11/780 was a major improvement; it was a multitasking environment with real terminals. VAX was an interesting beast: the first step in the boot process was to start an embedded PDP-11 processor that read an 8” floppy disc and uploaded the microcode to the main CPU. The only drawback was that 30 users had to share 2M (two megabytes) of main memory and so I couldn’t crash the machine whenever I wanted.
A few years later, I managed to get access to a really cute research PDP-11 running RSX-11M. Finally I could start writing device drivers and kernel code without risking the wrath of dozens of users years older than myself. And then the personal computers appeared and I probably made one the best choices I could – the BBC Micro from Acorn. It was never popular, but it had an amazingly well-designed operating system that you could extend in any way you wish (and even symbolic assembly language built into its BBC BASIC).