Building Network Automation Solutions
6 week online course starting in September 2017

The Future of Multicast and QoS

A. Friend sent me a long list of questions after listening to excellent Future of Networking podcast with Martin Casado because (as he said) he prefers “having a technical discussion with arguments and not just throwing statements out there.

He started with “Martin's view seems to be that network is all plumbing and all the intelligence should be in the applications.

Which I totally agree with. Complexity belongs to the network edge (ideally into the end-hosts) and every time we managed to move it from the network to the end-hosts, we got cheaper and faster networks (example: moving from X.28 to Telnet, and from FC to NFS/iSCSI).

They make statements that multicast and QoS are going away.

Also somewhat agree with. Not unconditionally, but yes (and their discussion was more nuanced than that anyway, so go and listen to it). Let’s go into details.

While I can agree with some statements, multicast is there for a reason.

There are a few well-known use cases for multicast (live 1-to-N video streaming, stock exchange feeds), and a few others where the developers wanted to push the problem of service discovery to someone else (yet again proving RFC 1925 section 2.6). Apart from that, multicast has been a zombie for decades.

They mention the application keeping track of who requested the stream instead. OK, so we moved the state to the application. That's fine and dandy when app is hosted on one device. Now our service is popular and needs to be hosted on multiple devices, who keeps state? How do we sync the state? Hasn't the amount of state increased compared to having it in the network? Not even taking concern of the bandwidth are wasting.

First, state is cheap when it’s implemented in low-speed software. Low-speed RAM is cheap, high-speed packet forwarding engines using TCAM (because a walk down a data structure would be too slow) is expensive. Moving state to low-speed components makes perfect sense.

Also, how many times have you seen endpoints requiring the exact same content, which could then be effectively replicated on-the-fly within the network (apart from the two use cases I mentioned above)?

On a somewhat tangential topic, creating unnecessary state and/or syncing state across multiple devices is best avoided if you want to scale your solution. You can start with something as simple as not keeping session state in local files on your web server (so you don’t have to use session stickiness on your load balancer), or go as far as Facebook did.

Regarding QoS, it's just a tool to differentiate traffic based on business needs. I can agree that throwing more bandwidth at things is often the best solution. However, that is not always an option and as long as we consider a certain type of traffic to be more important than other traffic we must have some form of QoS or queuing.

Generic widespread end-to-end QoS has been dead even before it was born ;) One would hope that we’ve learned that lesson from ATM. Read what Geoff Huston wrote on the topic of Internet-wide QoS, and listen to the Packet Pushers podcast with Douglas Comer.

In the data centers, it’s easier to throw more bandwidth at the problem, and solve the potential remaining 5% in the application stack. MP-TCP is one of the solutions addressing the hashing problems of elephant flows, as is MPIO. Other tricks like FlowBender modify hashing fields (IPv6 flow label or TTL) until the TCP session hits a non-congested link (they already have a patch for Linux kernel).

We’ll talk about these issues in the next Leaf-and-Spine Fabric Designs webinar sometimes in autumn.

As for QoS on WAN, the real question is “can you control the congestion point?”, and in most cases, the answer is NO (because the congestion happens in DSLAM or within the SP access/core network), which leaves the end-to-end congestion tracking offered by some SD-WAN vendors as the only alternative. Yes, it’s QoS (shaping + subsequent queuing) but it’s done at the very edge of the congestion domain, not everywhere in the network.

16 comments:

  1. We use multicast for IPTV streaming. The content provider send each channel to a different multicast address and we use multicast routing. If we won't use multicast our uplinks will choke.

    ReplyDelete
    Replies
    1. Oh and also for radars. So there is a few more scenarios for multicast that is needed

      Delete
    2. Well, I'd guess that IPTV falls within the 1:N video streaming category.

      However, have you considered what happens once people stop watching TV and turn to Netflix? All those video streams will become unicast streams.

      Delete
    3. Not applicable when used with live streaming like sports or news when people want to watch live.
      I still think there is a need for multicast and network engineers need to master it, it's a great technology where a lot of time not implemented correctly because network engineers are afraid of it.

      Delete
  2. Pushing controller to end-host also can be done for routing. BGP for example, host will have the control to advertise the UP network required and 'take over' the complexity from the network.

    However, it doesn't mean that the plumbing in the middle does not require BGP let alone routing protocol.

    In this case, end-host is participating but not actually taking over.

    I cannot find real-life example (probably I haven't live that long) on how multicast control can be pushed to end-host. Session control/record is required on each hop.

    Idea is good, you both are futurist.

    ReplyDelete
  3. I am involved in Public Safety business Tetra / Apco where the multicasts are used in my opinion in right way. We have group communication 1 to many and it seems that multicasts are very efficient for this kind of network. From the application perspective we do not to have to care about how to build optimal audio delivery tree. The network take care of this task.

    ReplyDelete
  4. The mantra that the multicasts are dead and not useful is repeated in this "Future of the Networking" series many times. I guess that all guests (Comer, Huston) think about Internet. The Enterprise networking is somewhat different story and there are many niches where we do things differently to be efficient. They often mention CDN as the ultimate solution which kills the multicasts. Again, what works fine for the Internet (from the cost perspective) is not always good for non Internet networks

    ReplyDelete
    Replies
    1. Even with video streams, it's far easier and more reliable to setup an internal CDN which sends unicast streams to caching/re-broadcasting appliances closer to the end-users than it is to keep a multicast network working reliably. Setting up multicast routing isn't difficult, but troubleshooting a multicast network issue is a nightmare. Can you setup a debug on a production router in the middle of a production day? Can you get a packet-capture? How often can you upgrade router firmware to get in patches for buggy multicast routing?

      At a small scale (100 Mbps links, Cisco ISR routers) I'm sure multicast works fine. At a large scale (10Gbps links, Cisco 6500/7600 routers), where multicast should be the most beneficial, multicast is a troubleshooting nightmare.

      Delete
    2. Sounds like FUD to me. We do both CDN and IP multicast for different applications. We have been doing IP multicast video distribution for many years. It's not rocket science, it hasn't changed much in a decade, and it's not really that hard to troubleshoot. We use it where it makes sense to use it, and much higher speeds than 10Gbps. BIER is even new work going on to create a stateless multicast delivery method, so it's certainly not dead.

      As for QoS, even in long-haul networks there can be bottlenecks requiring QoS. There is also a movement towards more agile optical restoration to replace L3 capacity in long-haul/metro networks. During the restoration events you need QoS in order to keep higher priority traffic flowing while best effort traffic is dropped.

      Delete
    3. Good point Phil. Even at 10gig there's room to utilize QoS. Until the day when everyone will have all the bandwidth they'd ever need, QoS will be needed to prioritize traffic.

      Delete
  5. Thanks for the nice discussion. Makes sense to see different view points on that. Just to throw in some more facts on how far CDNs are away from, let's say, delivering the super bowl: https://www.youtube.com/watch?v=w479dF6C9YA

    ReplyDelete
  6. Getting software images on computers with multicast within an educational environment with hundreds of buildings placed all over the town. Not everyone has one big campus...

    ReplyDelete
    Replies
    1. I made that same argument a long while ago, and someone quickly pointed me to an article describing how a very large installation (don't remember who it was) got way better results replacing multicast-based software distribution with Bittorrent.

      Delete
    2. Ivan, not sure if it is the same group you are referring to, but one of our faculties is using Bittorrent this way.
      https://uqcs.org.au/2014/03/23/seminar-pervade-and-hypervade.html

      Delete
  7. Multicast won't die, at least for a while, but for sure its role in the networking world will diminish.

    On the DC side, for applications that are high volume and require efficient delivery, multicast will still play a key role. Perfect example is high volume market data distribution. On the WAN side, the use cases are mainly for video streaming (e.g. IPTV), broadcast, or applications that require fair delivery to recipients.

    More and more applications are now relying on unicast delivery and doing multicast in a much localized fashion. Between the application layer and the network layer, it's getting more common to have a transport layer which is responsible for delivering data to the end points. This layer running in the servers can provide more flexibility and customization of data delivery. And that could also simply be a message bus in a pub-sub fashion.

    Multicast has always been difficult to conquer. If you are involved, for example, in deploying multicast to MPLS WAN, then you should remember the fun debate related to NG-MVPN, mLDP VS RSVP-TE, etc. Why bothers if you don't have to? While multicast won't die, its future is dim. And this is not necessary a bad thing.

    ReplyDelete
    Replies
    1. Multicast is one of those protocols, just like good network engineers ;) you don't realize what it does until it's gone. How about all dynamic routing protocols for communication..224.0.0.x anyone? Even VXLAN utilizes multicast for 1 to many BUMs for simple configurations. Alas multicast seems to be the child no one understands, but everyone needs.

      Delete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.