Building network automation solutions

9 module online course

Start now!

Blog Posts in April 2023

Interesting: BGP Zombie Outbreak on Juniper Routers

BGP zombies are routes in the BGP table that refuse to disappear even though they should have been long gone. Recent measurements estimate between 0.5% and 1.5% of all routes in the global BGP table are zombies, which sounds crazy – after all, BGP is supposed to be pretty reliable.

Daryll Swer identified one potential source – Juniper routers do not revoke suppressed aggregated prefixes – and documented it in Navigating a BGP zombie outbreak on Juniper routers.

see 1 comments

Advantages and Drawbacks of EVPN-based Multihoming

Lukas Krattiger wrapped up his EVPN-versus-MLAG presentation (part of EVPN Deep Dive webinar) with an overview of the advantages and drawbacks of EVPN-based multihoming solutions:

  • N-way multihoming
  • Flexible connectivity (no need for a peer link)
  • Fabric-wide scope (MAC multipathing required on ingress node)
You need Free Subscription to watch the video. To watch the whole webinar, buy Standard or Expert Subscription.
add comment

Why Is OSPF (and BGP) More Complex than STP?

I got this question from one of my readers:

Why are OSPF and BGP are more complex than STP from a designer or administrator point of view? I tried everything to come to a conclusion but I couldn’t find a concluded answer, ChatGPT gave a circular loop answer.

There are numerous reasons why a protocol, a technology or a solution might be more complex than another seemingly similar one (or as Russ White would have said, “if you haven’t found the tradeoffs, you haven’t looked hard enough”):

read more see 3 comments

Is Dynamic MAC Learning Better Than EVPN?

One of my readers worried about the control-plane-induced MAC learning lag in EVPN-based networks:

In all discussions about the advantages/disadvantages of VXLAN/EVPN, I can’t find any regarding the lag in learning new macs when you use the control plane for mac learning.

EVPN is definitely slower than data plane-based dynamic MAC learning (regardless of whether it’s done in hardware or software), but so is MLAG.

read more see 5 comments

IPv6 Security in Layer-2 Firewalls

You can configure many firewalls to act as a router (layer-3 firewall) or as a switch bridge (layer-2 firewall). The oft-ignored detail: how does a layer-2 firewall handle ARP (or any layer-2 protocol)?

Unless you want to use static ARP tables it’s pretty obvious that a layer-2 firewall MUST propagate ARP. It would be ideal if the firewall would also enforce layer-2 security (ARP/DHCP inspection and IPv6 RA guard), but it looks like at least PAN-OS version 11.0 disagrees with that sentiment.

Straight from Layer 2 and Layer 3 Packets over a Virtual Wire:

read more see 2 comments

Use Existing Device Configurations in netlab

Anne Baretta decided to use netlab to test a proposed DMVPN topology. As netlab doesn’t support DMVPN (and probably never will), he decided to use netlab capabilities to start the lab topology and perform initial configuration, adding DMVPN configuration commands as custom configurations. Here’s how he described the process:

In this case I used netlab as a quick way to get a topology up and running, and then add the DMVPN configuration by hand.

read more add comment

New: Network Infrastructure as Code Resources

While I was developing Network Automation Concepts webinar and the network automation online course, I wrote numerous blog posts on the Network Infrastructure as Code (NIaC) concepts, challenges, implementation details, tools, and sample solutions.

In March 2023 I collected these blog posts into a dedicated NIaC resources page that also includes links to webinars, sample network automation solutions, and relevant GitHub repositories.

add comment

IPv6 Addressing on Point-to-Point Links

One of my readers sent me this question:

In your observations on IPv6 assignments, what are common point-to-point IPv6 interfaces on routers? I know it always depends, but I’m hearing /64, /112, /126 and these opinions are causing some passionate debate.

(Checks the calendar) It’s 2023, IPv6 RFC has been published almost 25 years ago, and there are still people debating this stuff and confusing those who want to deploy IPv6? No wonder we’re not getting it deployed in enterprise networks ;)

read more see 7 comments

Using VLAN and VRF Links in netlab Topologies

I already mentioned the introduction of VRF- and VLAN access links in netlab release 1.5.1. Let’s see how they can simplify your lab topologies.

I always tried to make lab topologies as concise as I could, sometimes cheating using JSON-in-YAML syntax. For example, the topology describing three routers running OSPF could be as simple as this:

module: [ ospf ]
nodes: [ r1, r2, r3 ]
links: [ r1-r2, r2-r3, r3-r1 ]

Let’s unravel that:

read more add comment

ChatGPT Explaining the Need for iSCSI CRC

People keep telling me how well large language models like ChatGPT work for them, so now and then, I give it another try, most often resulting in another disappointment1. It might be that I suck at writing prompts2, or it could be that I have a knack for looking in the wrong places3.

This time4 I tried to “figure out5” why we need iSCSI checksums if we have iSCSI running over Ethernet which already has checksums. Enjoy the (ChatGPT) circular arguments and hallucinations with plenty of platitudes and no clear answer.

read more see 2 comments

Worth Reading: Was MPLS TE Worth the Effort?

Bruce Davie continues documenting the tradeoffs we had to make in networking, this time with Was MPLS Traffic Engineering Worthwhile? I found this bit particularly familiar:

It wasn’t hard to make a theoretical argument that MPLS-TE could improve network performance and average link utilization, by moving traffic from congested links to uncongested ones. The hard part was proving that it would actually do a better job in practice than the more traditional methods such as using link weights and multipath routing to achieve the same ends.

see 1 comments

Video: netlab Topology File

After introducing netlab in the Network Automation Tools webinar, I spent a few minutes describing the structure of the netlab lab topology file.

As always, use the video only as a starting point. For more details, read the netlab documentation (overview, reference guide).

You need Free Subscription to watch the video and Standard Subscription to watch the rest of the webinar.
add comment

Building a Small Network with ChatGPT

I must be a good prompt engineer – every time I ask ChatGPT something really simple it spews out nonsense. This time I asked it to build a small network with four routers:

I have a network with four Cisco routers (A,B,C,D). They are connected as follow: A-B, B-C, A-D, D-C. Each router has a loopback interface. Create router configurations that will result in A being able to reach loopback interfaces of all other routers.

Here’s what I got back1:

Here’s an example configuration for the four routers that should allow Router A to reach the loopback interfaces of all other routers:

read more see 1 comments

Should I Care About RPKI and Internet Routing Security?

One of my subscribers sent me this question:

I’m being asked to enter a working group on RPKI and route origination. I’m doing research, listening to Jeff Tantsura, who seems optimistic about taking steps to improve BGP security vs Geoff Huston, who isn’t as optimistic. Should I recommend to the group that the application security is the better investment?

You need both. RPKI is slowly becoming the baseline of global routing hygiene (like washing hands, only virtual, and done once every blue moon when you get new IP address space or when the certificates expire). More and more Internet Service Providers (including many tier-1 providers) filter RPKI invalids thus preventing the worst cases of unintentional route leaks.

read more add comment

DHCP Relaying with Redundant DHCP Servers

Previous posts in this series (DHCP relaying principles, inter-VRFs relaying, relaying in VXLAN segments and relaying from EVPN VRF) used a single DHCP server. It’s time to add another layer of complexity: redundant DHCP servers.

Lab Topology

We’ll use a lab topology similar to the VXLAN DHCP relaying lab, add a second DHCP server, and a third switch connecting the two DHCP servers to the rest of the network.

read more add comment

Turning WiFi into a Thick Yellow Cable

The “beauty” (from an attacker perspective) of the original shared-media Ethernet was the ability to see all traffic sent to other hosts. While it’s trivial to steal someone else’s IPv4 address, the ability to see their traffic allowed you to hijack their TCP sessions without the victim being any wiser (apart from the obvious session timeout). Really smart attackers could go a step further, insert themselves into the forwarding path, and inject extra payload into unencrypted sessions.

A recently-discovered WiFi vulnerability brought us back to that wonderful world.

read more see 1 comments

Kicking the Tires of GitHub Copilot

A friend sent me a video demo of his AI-driven network device configuration proof-of-concept. Before commenting on that idea, I wanted to see how well AI works as an assistant. Once Kristian Larsson mentioned he was using GitHub Copilot, it was obvious what to do next: try it out while working on the next netlab release.


  • It works.
  • Some Copilot suggestions are uncannily accurate; others are fishing expeditions.
  • It’s bland.
read more add comment

New: High Availability Clusters in Networking

Years ago I loved ranting about the stupidities of building stretched VLANs to run high-availability network services clusters with two nodes (be it firewalls, load balancers, or data center switches with centralized control plane) across multiple sites.

I collected pointers to those blog posts and other HA cluster resources on the new High Availability Service Clusters page.

add comment

netlab Release 1.5.1: VLAN and VRF Links

netlab release 1.5.1 makes it easier to create topologies with lots of VRF- or VLAN access links, or topologies with numerous similar links. It also includes support for D2 diagram scripting language in case you prefer its diagrams over those generated by Graphviz.

Even if you don’t find those features interesting (more about them later), you might want to upgrade to fix a nasty container-related behavior I discovered in recently-upgraded Ubuntu servers.

read more add comment

DHCP Relaying in EVPN VRFs

After figuring out how DHCP relaying works and testing it with VRFs and in VXLAN segments, it seems like a no-brainer to make it work with EVPN.

TL&DR: It works, at least when using Arista vEOS as the relay and Cisco CSR 1000v as the DHCP server.

Lab Topology

We’ll keep using the exact same “physical” topology we used in the VXLAN DHCP relaying lab, add EVPN and BGP to the control-plane cocktail, and put the VXLAN segment into a VRF. We’ll use CSR 1000v as the DHCP server because Cisco IOSv doesn’t support some of the DHCP option-82 sub-options we need.

read more add comment