IGMP and PIM in Multicast VXLAN Transport Networks

Got a really interesting question from A. Reader: “When and how does VXLAN use IGMP and PIM in transport (underlay) networks?

Obviously you need IGMP and PIM in multicast environments only (vCNS 5.x, Nexus 1000V in multicast mode).

IGMP is used by the ESXi hosts to tell the first-hop routers (in case you run VXLAN across multiple subnets) that they want to participate in particular multicast group, so the subnet in which they reside gets added to the distribution tree.

PIM is used between routers to figure out how the IP multicast flooding tree should look like.


Slide from VXLAN webinar

I covered IGMP usage in multicast-based VXLAN networks in my VXLAN Technical Deep Dive webinar.

Do We Have To Use IGMP Snooping?

Some VXLAN design documents talk about IGMP snooping – here’s why that feature might be relevant in your data center network.

IP multicast datagrams are sent as MAC frames with multicast destination MAC addresses. There frames are flooded by dumb L2 switch, resulting in wasted network bandwidth and server CPU cycles.

IGMP snooping gives some L3 smarts to L2 switches - they don't flood IP multicast frames out all ports, but only on ports from which they've received corresponding IGMP joins. You might decide not to care in a small data center network with tens of servers; IGMP snooping will definitely help in large (hundreds of servers) deployments.

Finally, if you want to use IGMP snooping in L2-only environment (all VXLAN hosts in the same IP subnet), you need a node that pretends it's a router and sends out IGMP queries, or the L2 switches have nothing to snoop.

More information

You’ll find in-depth discussion of VXLAN in my VXLAN Technical Deep Dive webinar. VMware NSX Architecture webinar briefly discusses the flooding changes introduced in VMware NSX for vSphere.

1 comment:

  1. Cisco has a solution regarding the layer 2 only IGMP Snooping: IGMP Querier on Nexus 5000.
    http://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/CLIConfigurationGuide/IGMPSnooping.html#wp1071684

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.