[gPXE] [Fwd: Re: any negative implications of setting TCP window size to (65536 - 4) ?]
Peter Scheie
peter at scheie.homedns.org
Tue Feb 2 20:33:41 EST 2010
Marty Connor wrote:
> Joshua Oreman wrote on 2/1/10 6:38 PM:
>> On Mon, Feb 1, 2010 at 6:08 PM, Peter
>> Scheie<peter at scheie.homedns.org> wrote:
>>> The default setting for the TCP window size, as set by #define
>>> TCP_MAX_WINDOW_SIZE in src/include/gpxe/tcp.h, is 4096. With this
>>> setting, it
>>> was taking 8-10 minutes for gpxe to download my 22MB initramfs over the
>>> internet. At the suggestion of people on #etherboot, I changed it to
>>> 65536 - 4,
>>> which was listed in tcp.h but commented out in favor of the 4k
>>> setting. At this
>>> new setting, downloading the 22MB file takes only about 30 seconds!
>>>
>>> I understand that gpxe has no ability to handle out-of-order packets,
>>> so a
>>> possible negative effect is that if there is disruption in the packet
>>> flow, the
>>> whole window will need to be resent. But in my testing so far, even
>>> if that is
>>> happening, downloading is still much faster. Are there any other
>>> potential
>>> pitfalls from using the larger window size? Why is the default set
>>> so low?
>>
>> I think the problem relates to gPXE's memory constraints. For various
>> reasons relating to boot-time memory constraints, we use a fixed-size
>> heap; currently its size is set at 128kB, and a 64kB window size would
>> take fully half of that space away. Some network drivers need lots of
>> memory too (sky2 for example).
>
> This seems like kind of a tricky constant to set. The core doesn't know
> what kind of buffer space for rx the driver has allocated, but maybe the
> driver could tell it, and it could try to set the window size to hold
> (at least) the number of packets the network card can accept.
>
> I think with 4096 (4KB) as a window size, and the size of segments we
> are receiving, we could only hold 2 * 1536byte packets before the window
> is full.
>
>> On the other hand, we're probably safe bumping it to 256kB. Thoughts
>> from other developers?
>> -- Josh
>
> I think 256kB must be a typo because that's twice the heap size. Did
> you mean 16KB or 24KB?
>
> I was going to run some WAN tests to see what the difference in download
> speed was with various sizes, like 4KB, 8KB, 16KB, and 32KB.
> Peter, have you tried any other settings? It would be nice to get more
> data points.
>
I did try one other setting, 48K (or was it 40K?), and then the time to download
my initramfs increased from ~30 seconds to ~2 minutes, although my recollection
is a bit fuzzy, and I may have had some other networking strangeness going on at
the time so that latter figure is not very reliable. For my upcoming demo, I
want the most impressive time, so I went back to (65536 - 4) for the time being.
I will try a variety of memory settings and post my results to the list.
Peter
More information about the gPXE
mailing list