Firewalls kill TCP performance when faced with out-of-order packets

In my discussion of per-packet versus per-destination load sharing, I've relied on the "accepted wisdom" that out-of-order TCP packets reduce session performance (as a side note, out-of-order UDP packets are a true performance killer; just try running NFS with out-of-order packets).

Today I've discovered another huge show-stopper: stateful firewalls (read: almost everything in use today) might just drop out-of-order packets, resulting in TCP timeouts and retransmissions (and repeated timeouts will totally wreck the session throughput). Here's how Cisco devices handle this problem:


  1. A similar problem is IP fragmentation (although that should not happen too often if MTU discovery is working properly), which is handled with the Virtual Fragmentation Reassembly feature introduced in IOS release 12.3(8)T.

  2. But problem really is not caused by load-sharing. It is because many things aren't prepared for out ouf order packets and it can just happen without load sharing, because that is how Internet works. You can fix your configuration by using per-destination, but it won't fix remote locations.

  3. As I wrote in a comment to another post, I really cannot see packets getting reordered unless you use per-packet load sharing or weird QoS setup that treats packets from the same flow differently.

    If there's something I'm missing, I would be glad to hear about it.


You don't have to log in to post a comment, but please do provide your real name/URL. Anonymous comments might get deleted.

Ivan Pepelnjak, CCIE#1354 Emeritus, is an independent network architect. He has been designing and implementing large-scale data communications networks as well as teaching and writing books about advanced technologies since 1990. See his full profile, contact him or follow @ioshints on Twitter.