VanLUG Email Archive
> Linux has to copy the data because it has to allocate a buffer large enough to
> hold the fragments. The original buffers are discontinuous, yet Linux wants the
> packet to be linear at all levels.
Indeed.
> Does anyone here have any scientific analysis of sk_buff's versus mbuff's or is
> this yet another religious debate? Linear buffers are easy to deal with which
> leads to efficient code in some places, at the cost of having to do extra
> copies so that a fragmented packet can be linear. It's a trade-off.
The loss from doing chaining of packet buffers is pretty low; you're
only talking about a few memory accesses here. On the other hand,
the loss of doing a copy is extremely high: easily tens of thousands
of memory accesses, often half or more of which are not in cache.
And note that it's not just in fragment reassembly that you can
get nailed on this. You might have to do a full recopy if an upper
layer mis-estimated the space to reserve for prepending headers.
Another area that chained buffers may help with is when you have
alignment restrictions, such as on an Alpha processor. Some Ethernet
cards insist on bringing in a full packet via DMA on an aligned
boundary which, since the Ethernet header is 14 bytes long (Bob
Metcalfe will go to hell for this. :-)) With chained buffers you
can avoid a copy of the entire packet when realigning the IP header
on an Alpha (necessary with some Ethernet cards that insist on
bringing in an entire packet on an aligned boundary) by moving some
pointers around and copying just the header.
Chained buffers can also make life easier (and may even be necessary,
with some hardware) when you're dealing with media that support
very large packets and protocols other than IP. HPPI doesn't even
have a packet size limit.
It appears that Linux developers themselves are convinced that
buffer chaining is a good idea, since they appear to have plans to
add it to Linux.
> Also, the only time you will get a lot of fragmented packets is if you are
> using some brain-dead UDP application that sends large datagrams.
What are you saying? That "smart" applications restrict themselves
to packets less than 1.5K in size, in order to be optimal for
Ethernet? The performance loss on fast networks that support large
packets is going to be enormous. You wouldn't apply the adjective
`brain-dead' to an appliaction that can't push fast Ethernet beyond
60% capacity, or Gigabit Ethernet beyond 10% capacity?
> The defragmentation only takes place if the packets are destined for the local
> machine.
Indeed. But those fragmented packets are going to be destined for
some machine, and reassembled there. Are you saying that one should
use Linux for routers, but not for end systems?
cjs
Curt Sampson
Info at http://www.portal.ca/
Internet Portal Services, Inc. Through infinite mist, software reverberates
Vancouver, BC (604) 257-9400 In code possess'd of invisible folly.