Category: QoS

Data Center Ethernet (or DCB or CEE, depending on who you are) is a hot story these days and it’s no wonder that misconceptions galore. However, when I hear several CCIEs I highly respect talk about “Priority Flow Control can be used to stop all the other traffic when storage needs more bandwidth”, I get worried. Exactly the opposite is true: you use PFC to stop the overzealous storage traffic (primarily FCoE, but also iSCSI) to make sure you don’t drop it.

Enhanced Transmission Selection (ETS) is the second part of the Data Center Bridging puzzle (I’ve already described Priority Flow Control). It specifies two different technologies:

• Queuing mechanisms in bridges
• Data Center Bridging eXchange protocol: a Control/Negotiation protocol that allows bridges and hosts to negotiate QoS parameters in a bridged network.

Although some bridges from some vendors supported numerous QoS mechanisms in the past, 802.1Qaz is the first attempt to standardize a richer set of QoS behaviors than the strict priority queuing defined in 802.1p.

Yesterday I wrote that you don’t need DCB technologies to implement FCoE in your network. The FC-BB-5 standard is quite explicit (it also says that 802.1Qbb is the other option):

Lossless Ethernet may be implemented through the use of some Ethernet extensions. A possible Ethernet extension to implement Lossless Ethernet is the PAUSE mechanism defined in IEEE 802.3-2008.

The PAUSE mechanism (802.3x) gives you lossless behavior, but results in undesired side effects when you run LAN and SAN traffic across a converged Ethernet infrastructure.

The emerging Ethernet bridging technologies have been hyped to an extent where the lines between them completely blurred, resulting in statements like “we need DCB and TRILL for FCoE”. Actually, none of that is true, but let’s focus on DCB and TRILL first.

As expected, my P2P traffic is bad for the network post generated lots of comments; from earning me another wonderful title (shill for Internet monopolies) that I’ll proudly add to my previous awards to numerous technical comments and even a link to a very creative use of BitTorrent to solve software distribution problems (thanks again, @packetlife).

Most of the commentators missed the main point of my post and somehow assumed that since I don’t wholeheartedly embrace P2P traffic I want to ban it from the Internet. Far from it, what I was trying to get across was a very simple message:

I’m positive you all know that. I also hope that you’re making sure it’s not hogging your enterprise network. Service Providers are not so fortunate – some Internet users claim using unlimited amounts of P2P traffic is their birthright. I don’t really care what kind of content these users transfer, they are consuming enormous amounts of network resources due to a combination of P2P client behavior (which is clearly “optimized” to grab as much as possible) and the default TCP/QoS interaction.

TL&DR: QPPB works in MPLS VPNs… with a few limitations (at least in Cisco IOS implementation).

And now for the long story: A while ago I’ve noticed that my LinkedIn friend Joe Cozzupoli changed his status to something like “trying to get QPPB to work in MPLS VPN environment”. I immediately got in touch with him and he was kind enough to send me working configurations; not just for the basic setup, but also for Inter-AS Option A, B and C labs.

Knowing that QPPB relies on CEF, I doubted it would work as well on VRF interfaces as it does in pure IP environments, so I decided to do a few tests of my own. Here are the limitations I found:

Jared Valentine found an interesting bug in the EEM’s SNMP event detector: if you’re triggering your EEM applet when the increment of an SNMP variable exceeds the threshold, you cannot re-arm the applet; the exit-type increment does not work. He fixed the problem with a somewhat more convoluted approach:

• The first EEM applet reads the SNMP variable, waits a second, does a second read and stores the difference in a counter.
• The second EEM applet is triggered based on the counter values.

I’m collecting tips like this one in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop; we can also organize a dedicated event for your networking team.

Here’s the source code for the first applet (he had to execute CLI show commands to work around the CB-QoS MIB limitations).

More than a year ago, I wrote about the very slow update rate of the variables in the CB-QoS MIB. In August WB found a workaround (do a show policy-map interface before reading the counters) and now Jared has tested it and confirmed that it works. He’s configured a simple EEM applet that executes the show command once per second:

event manager applet UpdateMibTables 
event timer watchdog time 1 
action 1.0 cli command "enable" 
action 2.0 cli command "show policy-map int dialer0" 

With this fix, he can use the SNMP variables in other EEM applets to detect VoIP calls within 1-2 seconds.

Don’t forget: numerous EEM topics are described in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop; we can also organize a dedicated event for your networking team.

After doing the initial tests of the HQF framework, I wanted to check how “hierarchical” it is. I’ve created a policy-map (as before) allocating various bandwidth percentages to individual TCP/UDP ports. One of the classes had a child service policy that allocated 70% of the bandwidth to TCP and 30% of the bandwidth to UDP (going to the same port#), with fair queuing being used in the TCP subclass.

Short summary: HQF worked brilliantly.

Continuing my tests of the Hierarchical Queuing Framework, I’ve tested whether the fair queuing works similarly to the previous IOS implementations, where the high-precedence sessions got proportionally more bandwidth.

Summary: Fair queuing within HQF ignores IP precedence.

Continuing from my first excursion into the brave new world of HQF, I wanted to check how well the intra-class fair queuing works. I’ve started with the same testbed and router configurations as before and configured the following policy-map on the WAN interface:

policy-map WAN
 class P5001
    bandwidth percent 20
    fair-queue
 class P5003
    bandwidth percent 30
class class-default
    fair-queue

The test used this background load:

Several readers told me that the Hierarchical Queuing Framework introduced in IOS releases 12.4(20)T and 15.0 (why do I always have the urge to write 12.5?) works much better than CB-WFQ. After spending several hours trying to break HQF, I have to concur with them: Cisco’s engineers did a splendid job. However, the HQF behavior might be slightly counterintuitive to those that became too familiar with CB-WFQ.

Most of the respondents to my last week’s challenge got it almost right. The minor (common) error was the assumption that police rate percent 50 would result in a TCP session getting 50% of the bandwidth. Eyal got that right: the TCP throughput is always significantly lower than that due to frequent drops caused by low burst sizes assumed by the police command and resulting TCP restarts (the most I was able to push through was around 90 kbps; half of the bandwidth would be 128 kbps).

I have to admit I was somewhat surprised by the lab test results I’ve published in my previous CB-WFQ post. It looks like we’ve been fed misleading information about (classic) CB-WFQ behavior for years.

Don’t tell me that things are completely different with HQF implemented in IOS releases 12.4(late)T and 15.0. I know that … but 95+% of the installed base do not use those releases.

Let’s see whether you can figure out what my next lab test results showed. I’ve been running three parallel TTCP sessions on ports 5001, 5002 and 5003 across a 256 kbit point-to-point link. Here’s the relevant part of my router configuration: