Category: performance

Watch Out: ISR Performance License

Bill Dagy sent me an annoying ISR gotcha. In his own words:

Since you have a large audience I thought I would throw this out here. Maybe it will help someone avoid spending 80 man hours troubleshooting network slowdowns.

Here’s the root cause of that behavior:

Cisco is now shipping routers that have some specified maximum throughput, but you have to buy a “boost license” to run them unthrottled. Maybe everyone already knew this but it sure took us by surprise.

Don’t believe it? Here’s a snapshot from Cisco 4000 Family Integrated Services Router Data Sheet:

read more see 10 comments

Must Watch: How NOT to Measure Latency

A while ago someone pointed me to an interesting talk explaining why 99th percentile represents a pretty good approximation of user-experienced latency on a typical web page (way longer version: Understanding Latency and Application Responsiveness, also How I Learned to Stop Worrying and Love Misery)

If you prefer reading instead of watching videos, there’s also everything you know about latency is wrong.

add comment

Worth Reading: SD-WAN Scalability Challenges

In January 2020 Doug Heckaman documented his experience with VeloCloud SD-WAN. He tried to be positive, but for whatever reason this particular bit caught my interest:

Edge Gateways have a limited number of tunnels they can support […]

WTF? Wasn’t x86-based software packet forwarding supposed to bring infinite resources and nirvana? How badly written must your solution be to have a limited number of IPsec tunnels on a decent x86 CPU?

read more see 2 comments

Fast Linux Packet Forwarding with Thomas Graf on Software Gone Wild

We did several podcasts describing how one could get stellar packet forwarding performance on x86 servers reimplementing the whole forwarding stack outside of kernel (Snabb Switch) or bypassing the Linux kernel and moving the packet processing into userspace (PF_Ring).

Now let’s see if it’s possible to improve the Linux kernel forwarding performance. Thomas Graf, one of the authors of Cilium claims it can be done and explained the intricate details in Episode 64 of Software Gone Wild.

read more see 6 comments

Sometimes It’s Not the Network

Marek Majkowski published an awesome real-life story on CloudFlare blog: users experienced occasional short-term sluggish performance and while everything pointed to a network problem, it turned out to be a garbage collection problem in Linux kernel.

Takeaway: It might not be the network's fault.

Also: How many people would be able to troubleshoot that problem and fix it? Technology is becoming way too complex, and I don’t think software-defined-whatever is the answer.

see 10 comments

Is Flow-Based Forwarding Just Marketing Fluff?

When writing the Packet- and Flow-Based Forwarding blog post, I tried to find a good definition of flow-based forwarding (and I was not the only one being confused), and the one from Junos SRX documentation is as good as anything else I found, so let’s use it.

TL&DR: Flow-based forwarding is a valid technical concept. However, when mentioned together with OpenFlow, it’s mostly marketing fluff.

read more see 16 comments

Can Virtual Routers Compete with Physical Hardware?

One of the participants of the Carrier Ethernet LinkedIn group asked a great question:

When we install a virtual-router of any vendor over an ordinary sever (having general-purpose microprocessor), can it really compete with a physical-router having ASICs, Network Processors…?

Short answer: No … and here’s my longer answer (cross-posted to my blog because not all of my readers participate in that group).

read more see 10 comments