(14:07:39) alanshieh: I could not find a KEEP_IT_REAL option in Etherboot 5.4, so I am going to build a call stub into 16:32 protected mode.
(14:10:32) alanshieh: Also, I don't know if adding an interface to export segment limits like we talked about is strictly needed; I think it's possible to use a flat (4GB) segment limit, and use the holes in the e820 output from ksyslog() to infer the possible memory that the PXE stack might need to use, and map in those holes from the undi-user program.
(14:10:32) alanshieh: I think that might work well for generic PXE stacks that use PIO to talk to the card.
(14:12:19) mdc: back
(14:12:29) alanshieh: We can find the NIC's PIO and MMIO resources from Linux lspci interfaces, so we don't need to teach the PXE stack to provide that part of the memory map either.
(14:14:23) Michael: You'll probably need to run in IOPL 3 anyway, which means no worrying about PIO resources
(14:14:26) Michael: Hi Marty
(14:14:42) mdc: Hi Michael, Hi All :)
(14:15:27) alanshieh: Yeah, the lame NE2K-PCI "PIO for everything design" will probably work out great for the initial part of the development process :-)
(14:15:55) alanshieh: I don't have to worry about read/write to 0xc0000000 - 0xffffffff

...

(16:26:43) alanshieh: Should I still have a signature between 0-1MB ? 0-9fc00 is marked as "usable" in my e820 map
(16:26:48) Michael: what's the chain of control through the boot process now?
(16:27:12) alanshieh: grub was giving me grief, so it's out of the loop completely.
(16:27:31) alanshieh: It's now
(16:27:31) alanshieh: Etherboot (via iso image) => ELF w/ Linux 2.6 => boot linux from HD image
(16:28:28) Michael: If Etherboot is loading an ELF image, it won't set up a PXE stack
(16:28:38) Michael: because the ELF image wouldn't expect one
(16:28:59) alanshieh: Ah
(16:29:39) alanshieh: Would doing
(16:29:39) alanshieh: Etherboot => pxelinux => Linux 2.6
(16:29:39) alanshieh: set up the stack?
(16:30:01) Michael: It would, though pxelinux will probably tear it down.  There might be an option to direct pxelinux to keep the PXE stack
(16:30:10) Michael: aha
(16:30:12) Michael: yes there is
(16:30:25) Michael: add "keeppxe" to the kernel command line
(16:30:33) Michael: in the pxelinux config file
(16:33:27) alanshieh: Ok, giving this a shot... this is a really long test cycle -- at least a minute to boot. I haven't scrubbed my rc.d.
(16:36:26) alanshieh: Awesome, that worked.
(16:39:33) mdc: hpa strikes again :)
(16:39:48) alanshieh: Hunting for pixies...found !PXE at 0009f000...ok, found at 0x9f000
(16:39:48) alanshieh: API 9f00:680 St 9e00:1000 UD 9e00:1000 UC 9f00:ab0 BD 0:0 BC 0:0
(16:39:48) alanshieh: Seems reasonable to me.
(16:40:14) Michael: Yes, that looks correct for an Etherboot stack
(16:40:26) Michael: no base code, stack and data in same segment
(16:40:37) Michael: is the e820 map now not showing this memory?
(16:40:40) Michael: (if not, we have a problem)
(16:40:46) alanshieh: 0-1MB is still available
(16:40:58) alanshieh: I mean "usable"
(16:41:05) Michael: that cannot be the case; 640kB-1MB is always reserved
(16:41:15) Michael: what is the usable range shown in the kernel log?
(16:41:24) alanshieh:  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
(16:41:24) alanshieh:  BIOS-e820: 0000000000100000 - 0000000007eef040 (usable)
(16:41:39) alanshieh: whoops
(16:41:46) Michael: whoops?
(16:42:02) alanshieh: misread the number of 0s in the first range
(16:42:23) Michael: but it still says 9fc00 even when the PXE stack is showing up?
(16:42:33) Michael: something is badly wrong if that is the case
(16:42:34) alanshieh: yes
(16:42:44) Michael: which Etherboot version?
(16:43:22) alanshieh: 5.4.2
(16:43:31) Michael: hmmm
(16:44:03) Michael: looks as though either (a) the e820 map mangling is broken, or (b) pxelinux is re-intercepting INT 15 for some nefarious purpose of its own
(16:44:30) Michael: while our PXE stack is loaded, the memory we occupy should be invisible to all other programs
(16:44:39) Michael: that's the only way we guarantee that they won't tread on us
(16:44:54) Michael: we hook INT 15 and intercept e820, e801 and 88
(16:46:58) alanshieh: Is there a way for me to bypass pxelinux?
(16:47:45) Michael: not that will still cause Etherboot to load a PXE stack
(16:49:01) Michael: the code that does the INT 15 handling is in arch/i386/firmware/pcbios/e820mangler.S
(16:49:09) Michael: controlled by arch/i386/firmware/pcbios/hidemem.c
(16:49:28) Michael: you could put some debugging printf()s in hidemem.c just to check that the mangler is actually being installed (and not uninstalled)
(16:50:12) Michael: one possibility is that the "keeppxe" option causes pxelinux to shut down the stack and leave it in memory, rather than leaving the stack active
(16:50:20) Michael: we unhook INT 15 when the stack is shut down
(16:50:26) Michael: so that would produce the symptom you're seeing
(16:50:54) Michael: and would be fairly logical, since leaving a PXE stack running when you start an OS is generally a bad idea (unless you're trying to do exactly what you're doing now)
(16:50:55) Michael: :)
(16:51:04) alanshieh: heh
(16:51:40) alanshieh: Well, I'll need to get my bochs environment working for this. My Etherboot images do serial printout properly in bochs, but not in qemu
(16:56:17) alanshieh: Here's what pxelinux  does: 
(16:56:17) alanshieh: http://rafb.net/paste/results/bNrUVe33.html
(16:57:15) alanshieh: BTW, what's my e820 map supposed to look like?
(17:00:04) Michael: The first region should run from 0 to the start of the PXE stack; 9e000 in your case

After an adventure through 16-bit real mode code. I fixed the 640KB e820 hole, but there were other problems with the E820 map:

Date: Mon, 12 Jun 2006 04:20:08 -0400
From: "Alan Shieh" <alancshieh@gmail.com>
To: Etherboot <Etherboot-discuss@lists.sourceforge.net>
Subject: E820 map
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Delivered-To: alancshieh@gmail.com

As discussed on IRC, my PXE booted Linux was not getting a E820 map
that reflected the resident PXE stack.

I tracked down the problem for my Etherboot 5.2 snapshot. _hide_memory
only had 2 entries, one for the .text segment, and one for the heap.
However, the UNDI Code and Data segments are allocated from base
memory, which is not covered by either of these regions.

So I fixed hide_etherboot() and _hide_memory to reflect the UNDI
segments, and now they are correctly omitted from the e820 map.

However, I'm still wary of my e820 map:

 BIOS-e820: 0000000000000000 - 000000000009e000 (usable)
 BIOS-e820: 0000000000100000 - 0000000007eeeee0 (usable)

I expect another hole for the Etherboot .text and heap segments,
located at [01eeef70,01f00000). On entry to pxelinux (and while Linux
is active), the hide_memory regions array includes

[01eeef70,01f00000)
[01eeef60,01eeef70)
// as well as UNDI Data & Code

Based on my understanding of the E820 adjustment code, I should get a
memory region of [1f00000,7eeeee0).

I've instrumented my Linux kernel to dump the E820 map as it is
processed, and it looks like the E820 map I am getting has not been
modified from the BIOS version.

Any ideas on how to proceed?

Date: Wed, 14 Jun 2006 15:12:16 +0100 (BST)
From: Michael Brown <mbrown@fensystems.co.uk>
X-X-Sender: mcb30@dolphin.home
To: Alan Shieh <alancshieh@gmail.com>
Cc: Etherboot <Etherboot-discuss@lists.sourceforge.net>
Subject: Re: [Etherboot-discuss] E820 map
In-Reply-To: <1a7e305d0606120120l39b78b49n610b4ab7c9c03e05@mail.gmail.com>
Message-ID: <Pine.LNX.4.62.0606141509050.27611@dolphin.home>
References: <1a7e305d0606120120l39b78b49n610b4ab7c9c03e05@mail.gmail.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Mon, 12 Jun 2006, Alan Shieh wrote:
> As discussed on IRC, my PXE booted Linux was not getting a E820 map
> that reflected the resident PXE stack.
>
> I tracked down the problem for my Etherboot 5.2 snapshot. _hide_memory
> only had 2 entries, one for the .text segment, and one for the heap.
> However, the UNDI Code and Data segments are allocated from base
> memory, which is not covered by either of these regions.
>
> So I fixed hide_etherboot() and _hide_memory to reflect the UNDI
> segments, and now they are correctly omitted from the e820 map.
>
> However, I'm still wary of my e820 map:
>
> BIOS-e820: 0000000000000000 - 000000000009e000 (usable)
> BIOS-e820: 0000000000100000 - 0000000007eeeee0 (usable)
>
> I expect another hole for the Etherboot .text and heap segments,
> located at [01eeef70,01f00000). On entry to pxelinux (and while Linux
> is active), the hide_memory regions array includes
>
> [01eeef70,01f00000)
> [01eeef60,01eeef70)
> // as well as UNDI Data & Code

How much memory does this machine actually have?  The 0x07eeeee0 suggests 
a figure around the 128MB mark.  Etherboot should be relocating itself to 
somewhere near the top of this, yet 0x01eeef70 is around the 32MB mark. 
Something here doesn't add up.

Michael

Date: Wed, 14 Jun 2006 16:14:11 +0100 (BST)
From: Michael Brown <mbrown@fensystems.co.uk>
X-X-Sender: mcb30@dolphin.home
To: Alan Shieh <alancshieh@gmail.com>
Cc: Etherboot <Etherboot-discuss@lists.sourceforge.net>
Subject: Re: [Etherboot-discuss] E820 map
In-Reply-To: <1a7e305d0606140727r2cdc2576u527c082b5b39eef6@mail.gmail.com>
Message-ID: <Pine.LNX.4.62.0606141611220.27611@dolphin.home>
References: <1a7e305d0606120120l39b78b49n610b4ab7c9c03e05@mail.gmail.com>
 <Pine.LNX.4.62.0606141509050.27611@dolphin.home>
 <1a7e305d0606140727r2cdc2576u527c082b5b39eef6@mail.gmail.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed

On Wed, 14 Jun 2006, Alan Shieh wrote:
>>> [01eeef70,01f00000)
>>> [01eeef60,01eeef70)
>>> // as well as UNDI Data & Code
>>
>> How much memory does this machine actually have?  The 0x07eeeee0 suggests
>> a figure around the 128MB mark.  Etherboot should be relocating itself to
>> somewhere near the top of this, yet 0x01eeef70 is around the 32MB mark.
>> Something here doesn't add up.
>
> Yes, the machine has 128 MB.

Thanks.  Can you add some debug statements to the relocation logic, to see 
how Etherboot ends up making the decision to relocate to the 32MB mark?

For Etherboot 5.4 and below, the e820 map mangler can only truncate 
regions; it can't split them.  This can result in wasting huge amounts 
of memory space unless the region to be hidden is located near the end of 
an existing region.

This is fixed in gpxe-0.5 (which can split regions), but that branch 
doesn't currently have a working PXE API and so is not much use for your 
tests.

Michael