[gPXE] any negative implications of setting TCP window size to (65536 - 4) ?

[H. Peter Anvin]

On 02/19/2010 09:45 AM, Glenn Brown wrote:
>> The lack of out-of-order reception [...]  really needs to be addressed.
> 
> To put some numbers on the performance impact of out-of-order packets: a 
> 1997 paper reported "Overall, 2.0% of all of [...] data packets and 0.6% 
> of the acks arrived out of order".  That's from
> http://cseweb.ucsd.edu/classes/wi01/cse222/papers/paxson-e2e-packets-sigcomm97.pdf
> section 3.  I used this paper only because it was the first source I 
> found via Google.
> 
> Those statistics are enough to estimate the effect of dropping instead 
> of reordering: in-order data packets will come in bursts of 50 packets 
> on average and have a ~round-trip-time delay, say 200ms, before the next 
> burst.  With 1500 byte MTU on an 'infinite bandwidth' network, that 
> limits performance to ~3Mbit/s.
> 
> This thread was started because gPXE's limited window size was 
> throttling performance.  For comparison, gPXE's 4K window and the same 
> assumed 200ms round trip time limits performance to ~4K/200ms or 
> 0.163Mbit/s.  So, it seems the window-size problem is a much more severe 
> performance limitation than is the lack of out-of-order support.
> 
> The window size must be addressed before out-of-order support will help.
> 

No doubt, from what I can tell (and I asked mcb30 about this), the only
thing that might be needed is to add some more memory to the heap (to
support the sender side -- in theory we could avoid that by doing an
assymmetric window, but it hardly seems worth it.)

> That said, out-of-order is not hard to address.  It simply requires a 
> sorted list of 'early' packets per connection, which is checked after 
> each in-order packet is processed.  The only complexities that spring to 
> mind are "What's the right way to deal with buffer starvation?" and "Can 
> one avoid copies when buffering early receives?"  I'd be happy to share 
> thoughts on these if anyone is planning to implement it.  Unfortunately, 
> I don't have time to implement+debug+support it myself.

Well, yes.  The best way to do this has been debated several times over
the last two years.

	-hpa