Ethernet inter-frame gap: Another living fossil?

Recently I’ve stumbled across a year-old post by James Ventre describing the reasons output rate on an Ethernet-type interface (as reported by the router) never reaches the actual interface speed. One of them: inter-frame/packet gap (IPG).

I was stunned ... I remember very well the early days of thick/thin coax Ethernet when the IPG was needed for proper carrier sense/collision avoidance detection (probability of a collision decreases drastically as you introduce IPG), but on a high-speed point-to-point full duplex link? You must be kidding.

Unfortunately, that’s not the case. IEEE 802.3 standard is very specific. Section 4.2.3.2.1, full duplex operation:

After the last bit of a transmitted frame, (that is, when transmitting changes from true to false), the MAC continues to defer for a proper interPacket-Gap (see 4.2.3.2.2)

And further on (4.2.3.2.2):

This is intended to provide interpacket recovery time for other CSMA/CD sublayers and for the physical medium.

What recovery time? What is there to recover if you continuously transmit on a full-duplex channel?

Next line of thought: maybe IPG is really needed to make sure your brand-new $9.99 Fast Ethernet switch can still work with the ancient 10 Mbps NE1000 NIC. But how does enforcing the same behavior on 10GE make sense? According to the Section 4.4.2 of the IEEE 802.3 standard, the IPG is 96 bits regardless of MAC data rate. Can anyone enlighten me? Or is IPG just another one of those you-don’t-want-to care things?

7 comments:

  1. Dmitri Kalintsev08 October, 2010 09:12

    As far as I understand, the IFG is there to provide headroom to compensate for clocking errors.

    ReplyDelete
  2. The interfame gap isn't there for benefit of the sending station, nor is it there to reduce collisions.

    Quoting Rich Seifert (who wrote that part of the specification):

    "the purpose of the IFG is to allow receivers time to "recover" after reception of a frame (e.g., finish DMA operations, post status, adjust buffer pointers, etc.). This time is needed in both half- and full-duplex mode equally"

    IFG doesn't help with collisions because IFG != random backoff timer. IFG is /always/ 96 bit times, so two shared-media stations who both have a frame queued will /always/ collide on the first try, after both having waiting the /same/ 96 bit times. Collisions are not reduced. It's CSMA/CD <- Collision Detection, not Collision Avoidance.

    Note that the "gap to avoid collision" strategy does exist in 801.11-based networks with CSMA/CA (avoidance), where the IFG before a new sender comes on the air consists of both a fixed time AND a QoS-enabling/collision-avoiding roll of the dice.

    I mention the 802.11 scheme because it's clever: high priority packets roll a die with (say) 4 sides, resulting in a delay of 0-3 units. Low priority packets roll a die with (say) 16 sides, resulting in a delay of 0-15 units. The probability is that your voice frame will hit the air before your ftp frame, even though they are being broadcast onto a shared media (air) by different and uncoordinated radios.

    IPG has been eliminated in 802.11n with frame aggregation. A sending station is now allowed to stream a number frames back-to-back-to-back. This strategy is a huge improvement to both latency and throughput in a shared media environment. On a full-duplex link the effect would be less dramatic.

    FA can be a problem for voice, however. If the voice frame loses the dice roll, then he has to wait for a whole stack of FTP frames before he gets to roll for airtime again.

    ReplyDelete
  3. Thank you for the explanation. Looks like we've traded bandwidth for simpler/cheaper silicon. Makes perfect sense.

    On the collision debate: in a coaxial cable, one bit @ 10 Mbps ~= 20 meters. 96 bit times is thus almost 2 km. The IFG thus (probably) ensures that a workstation that has just finished transmitting a frame will not compete with others for the next frame slot as there's almost no chance of collision if someone else starts transmitting as soon as the carrier is gone from the cable.

    ReplyDelete
  4. Ivan, thank you for the effort you put into your blog. I'm happy to be able to give a little something back.

    Actually, any time that two stations have a frame queued, they're /guaranteed/ to collide.

    One station might start to transmit "early" (perspective matters here) because:
    - he's cheating
    - his clock is slightly broken
    - his position on the cable makes the transmission "just appear" early relative to somebody else
    - ?

    No matter the reason, if another station has a frame queued (there's a race for wire time), that 2nd station /will/ transmit. Even after somebody else has started, the 2nd guy will force a collision. There's no "collision avoidance" here.

    The rationale is to force stations into the (extended) random backoff interval, rather than allow a station with a slightly too-fast clock to dominate the link.

    ReplyDelete
  5. Interesting new perspective ... Now I'm starting to understand why (coax/shared) Ethernet rocked with two stations (FTP could get 1.2 MBps throughput) and slowed down significantly with multiple traffic flows ... 15 years too late, but nonetheless quite intriguing.

    Thank you!

    ReplyDelete
  6. If another station has a framed queue, won't it first 'sense' whether a packet is already being transmitted by another station before starting its own transmission (CSMA)?

    ReplyDelete
  7. In wiki, for 1 Gigabit connection minimum interface gap is 8 byte = 64 bit times. Do you know of any vendor implement IFG less than 96 bit times?

    How does the two system tell each other on what bit times ( IFG )to use?

    Thanks

    ReplyDelete

You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354, is the chief technology advisor for NIL Data Communications. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.