The Impact of tx-ring-limit
Setting the size of the hardware output queue in Cisco IOS with the (then undocumented) tx-ring-limit (formerly known as tx-limit) has been a big deal when I was developing the first version of the QoS course that eventually became the initial release of the Implementing Cisco Quality of Service training.
However, while it's intuitively clear that the longer hardware queue affects the QoS, years passed before I finally took the time to measure the actual impact.
However, of course, I have a few questions.
The diagrams are confusing. Is the circular yellow 'Transmit ring' the same device as the square blue 'Hardware Queue'?
The transmit ring use a FIFO queue. Is your article saying there is a software FIFO queue controlled by the hold-queue out command? That there is a software FIFO queue (used when no fancy queueing), and the tx-ring FIFO queue?
The Cisco Command Lookup tool has the following warning for 'hold-queue' command : "This [small hold queue] is generally better than having duplicate copies of the same packet within the network (which can happen with large hold queues)." Why would large hold queue cause duplicate packets?
Why would lengthening the transmit ring increase jitter or delay?
No, the hardware queue is not adjustable, and is (from my understanding) part of the PHY hardware. The tx-ring is part of the router memory (as allocated by IOS)
Why would lengthening the transmit ring increase jitter or delay?
I always thought that the tx-ring is processed at a certain frequency ie: n packets-per-second. As the tx-ring gets larger, a packet takes longer to be processed.
If you use FIFO queuing, the FIFO queue (configured with the hold-queue command) is the same as the tx-ring.
I would assume that the tx-ring is filled when the HW interface decides to interrupt the CPU. However, to avoid speculations, I'll study a few chipset specs before saying anything more on this topic.
The warning for the 'hold-queue' command is probably a leftover from 9.1 SW (or something similar) when the time a packet could spend in transit (waiting in various queues) was comparable to the retransmission timeout (which could result in two identical packets being in transit).
As for the impact of transmit ring on jitter/delay, I'll write a more in-depth explanation (just give me some time, I'm juggling too many things at the moment)
one comment here regarding periodic pools of TX descriptors: in fact from what You've stated here for low-utilized interfaces we will have some extra delay related with polling interval.
Are You sure about this ? From what I know the indication of pooling the next TX descriptor is based on other counters in the system, which are based on feeding the queue with incoming packets (from system perspective we know extactly what is the status/number of packets to be processed with sending sub-process.
I was also involved in some network-based OS development for systems based on Freescale uP and we've solved this issue in the same way, as I've described.
Do You have any other sources/links to verify this ?
Thanks a lot for any support,
regards
msewera
I cannot be completely sure what exactly individual releases of Cisco IOS do on various platforms. A reader comment prompted me to study the hardware specs of various controllers and the results are as described in the Wiki article.
By default, the AMD PCnet controller will poll the first transmit descriptor every 2 msecs until it finds a packet ready in which case it will transmit all the queued packets and revert to polling mode after it encounters an empty description (see Page 66 of the Am79C976 technical documentation.
thanks for the reply!
Regarding this polling mode, as it is stated in the documentation, it works only if enabled ("If the TXDPOLL bit in CSR4 is set, then the transmit polling function is disabled.") and also it works only if "there is no network channel activity and there is no pre- or post-receive or pre- or post-transmit activity being performed by the Am79C976 controller".
As I wrote before, in most routing/switching implementations transmission of the packet is done by enabling transmission circuits /by writing to control register/ in case of we have a packet in the system to be transmitted or disabling tranmission circuits in case of system is empty (no packets to be handled). In this way we are bypassing the polling interval issue.
We had this approach in our OS kernel implementation, so I was just curious how the Cisco guys implemented this for low-end platforms;-)
thanks for any support,
best regards
michal
Andrew