✎ soc:2008:stefanha:journal:week8

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Stefan Hajnoczi: GDB Remote Debugging ======

===== Week 6 =====

**Milestones:**
  * [b44] Tested and clean for mainline review.
  * [gpxelinux.0] Merge Award BIOS return-to-PXE workaround.

==== Mon Jul 14 ====
Git commit: [[http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=commit;h=86d1342ffbb0dcfc6bb02da81af766f7a0e8f306|[init] Shutdown flags for fine-grained exit behavior]]

Time is passing quickly.  Today I worked on fine-grained ''shutdown()''.  There are two scenarios under which ''shutdown()'' is called:
  - Booting an image that does not use gPXE interfaces (like BIOS int 13h or PXE).  Here everything should be cleaned up and shut down.
  - Exiting gPXE to return control to firmware (usually to boot the next device if gPXE failed).  Here we may wish to leave some resources, like the underlying PXE stack, running.

There is currently no way to distinguish these two cases for ''shutdown()''.  Therefore, we are not able to clean up or leave resources as appropriate.

I have added a flag to ''shutdown()'' to indicate either ''SHUTDOWN_BOOT'' or ''SHUTDOWN_EXIT''.  As part of ''undionly.kpxe'', we need to unload the PXE base code if ''shutdown(SHUTDOWN_BOOT)'' is performed.  I still need to figure out how to do the unloading and freeing of base memory... ;-)

==== Tue Jul 15 ====
Git commit:
  * [[http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=commit;h=b370b8791c1b99a143a8b915170fac0b184f35b3|[prefix] kkpxe prefix for keep UNDI, return-to-PXE images]]

Sent the ''e820mangler.S'' patch for full clipping into fragments for review today.

**Created ''.kkpxe'' image type** for PXE images that keep PXE+UNDI loaded and return to PXE (instead of int 18h).  [[http://etherboot.org/share/stefanha/gpxelinux.kkpxe|Here]] is a ''gpxelinux.0'' image built from ''undionly.kkpxe''.

==== Wed Jul 16 ====
**Sent gpxelinux merge patches for review**.  I spent some time this week working on fine-grained ''shutdown()'' so that gPXE can free resources appropriately.  However, I believe that this work is independent of the gpxelinux merge and our priority is to get gpxelinux merged.  Therefore, I have submitted the gpxelinux patches now, without fine-grained ''shutdown()''.

==== Thur Jul 17 ====
Git commit: [[http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=commit;h=0d15b1612bb1258c2cdccc7a106f94a5bb02bd88|[image] Multiple embedded images and no hardcoded DHCP]]

Users who want a static IP address instead of DHCP find they need to edit
the source to remove hardcoded DHCP.  They can then use the ''EMBEDDED_IMAGE''
build option to include a gPXE script that configures their static network
settings inside the gPXE binary.

Since editing the source is inflexible, there is a discussion to change gPXE's
behavior to something more general than hardcoded DHCP.  The new policy would
skip DHCP for embedded images since they likely want to set up network settings
themselves.

While this solves the problem for embedded gPXE scripts which do their own
network configuration, it breaks when ''pxelinux.0'' is embedded inside gPXE.
PXELINUX expects the network interface to be (auto-)configured.

I think the most general way to solve this dilemma is by allowing multiple
embedded images and making gPXE //always// boot from an embedded image.  Even
when you build without explicit ''EMBEDDED_IMAGE'' settings, a default gPXE
script is built in.

Anyway, I have created a ''multiembed'' branch with these features implemented.
I think most of this solution has already been thought about by hpa, mcb30, and
mdc.  I just wanted to code something last night; hope this is a step in the
right direction.

//Update: removed detailed description since the reworked patch changes things
slightly//.  See [[.:week8#sat_19_2008|newer entry]].

==== Fri 18 2008 ====
**Back to the b44 driver**, I have removed uhmalloc and am using malloc's allocator instead.  Once this reworking is complete, I can move up to the DMA mapping API to support DMA pools.  Then I will send these patches for review and merge before sending the actual b44 driver.

**Sent cleaned up gpxelinux merge patch** for mainline review.

==== Sat 19 2008 ====
Git commits:
  * [[http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=commit;h=26b0197cb63725a58f596bf279d4e37e6862b5c2|[image] Make dependencies for EMBEDDED_IMAGE]]
  * [[http://git.etherboot.org/?p=people/stefanha/gpxe.git;a=commit;h=16dc7dddc05ef292dcda3f4c67632ef911eb984c|[image] Clean up embedded image support]]

**Reworked multiembed patch**.  The following text documents the user-visible changes:

=== Embedded images ===
gPXE normally fetches images over the network using TFTP, HTTP, iSCSI, or other network protocols.  It is also possible to //embed// an image file inside gPXE and make it available without fetching over the network.

By embedding gPXE scripts, you can customize gPXE's behavior before its first network access.  For example, static network configuration can be done by embedding a gPXE script.

Images are embedded when building gPXE with ''make'':
<code>
$ make EMBEDDED_IMAGE=../contrib/scripts/static.gpxe
</code>

This builds a gPXE image which executes the ''static.gpxe'' script on startup.  The ''static.gpxe'' sample script works with QEMU's user network stack:
<code>
#!gpxe
ifopen net0
set net0/ip 10.0.2.15
set net0/netmask 255.255.255.0
set net0/gateway 10.0.2.2
set net0/dns 10.0.2.3
kernel http://etherboot.org/gtest/gtest.gpxe
boot
</code>

Multiple images can be embedded:
<code>
$ make EMBEDDED_IMAGE=../contrib/scripts/gpxelinux.gpxe,/usr/lib/syslinux/pxelinux.0
</code>

This builds a gPXE image which executes the ''gpxelinux.gpxe'' script on startup.  The ''gpxelinux.gpxe'' script performs DHCP before loading the embedded ''pxelinux.0'' image and executing it:
<code>
#!gpxe
dhcp net0
imgload img1
boot
</code>

Why was ''img1'' used an not ''pxelinux.0''?  Embedded images are automatically named ''img#'' starting from ''img0''.  They do not retain their original filename.  For example:
<code>
$ make EMBEDDED_IMAGE=first.gpxe,second.gpxe,third.gpxe
                      ^ img0     ^ img1      ^ img2
</code>

Also, notice that we had to use ''imgload'' before doing ''boot''.  This is because embedded images are not automatically loaded for booting.  They are simply available as if they had been fetched using ''imgfetch''.

The full behavior is described as follows.  When gPXE starts, it does the equivalent of ''imgfetch'' for every embedded image and assigns them names starting with ''img0'' for the first image.  Then it loads and boots the first image (''img0'').  If booting ''img0'' fails, it removes all embedded images and gives up.

It's interesting to note that gPXE //always// embeds an image, even if you do not give it one.  By specifying an embedded image, you are overriding the default embedded image.  The default embedded image is a script that tries DHCP booting from each network interface in turn:
<code>
#!gpxe
autoboot
</code>

**Limitations of my patch**:
  * The semantics of replacing ''autoboot()'' with embedded images are not 100% correct.  The autoboot process will bring up each network interface in turn using DHCP and attempt to run the embedded image or fetch a file from the network.  Now, embedded images run only once and without any network interface being up or configured.

Next steps:
  * [uhmalloc] Kill this patch, use ''malloc'' allocator instead for DMA buffers.
  * [DMA] DMA pool API so drivers can reserve DMA buffers on ''open()''.
  * [b44] Cleanup, testing, performance.
  * [shutdown] Remove gPXE allocated memory and free up PXE+UNDI, if necessary.
  * [bzImage] Expand the heap size to the full 64K segment when loading a bzImage kernel with version 2.02 or higher.
  * [GDB] Real-mode remote debugging.
Trace: • week11 • week1 • indolent_playground • week1 • rpcbind_-_build_instructions • journal • week0 • pxeknifedsl • week11 • week5 • project_plan • week3 • project_plan • project_plan