This is an old revision of the document!


Joshua Oreman: 802.11 wireless development

Journal Week 11

Monday, 3 August

We have a mailing list regular who's interested in using gPXE on OS X. Since cross-gcc is very strict about printf specifiers, this led to several patches to fix compilation under both i386 and x86_64. Also, the recent “startpxe” command addition broke EFI builds by unconditionally dragging in real-mode UNDI code; fixed by making PXE_CMD a default only for pcbios builds.

Also, my sky2 driver (used by many Macs) has been merged:

I updated the Building on OS X page to reflect some suggestions and the new driver availability.

Before SoC, I submitted a patch to enable debugging over FireWire, but it was very ad-hoc and somewhat ugly (the user had to enter an address displayed by gPXE into the program that would try to connect over FireWire). Since gPXE is loaded in high memory, both on pcbios and EFI architectures, it's infeasible to scan through memory (the only thing a FireWire client can do, since we use the physical-DMA interface) to find anything. We have no idea how much memory is installed, and even over FireWire, 2GB to 4GB takes a long time to scan through.

To solve this problem in a hopefully generic way, I've implemented a function umalloc_low() to allocate memory that is guaranteed to fall below 640k. On EFI, we can allocate EfiConventionalMemory through a boot services call; on pcbios, though, the only segment that's safe to use is the one we've already taken up with our 16-bit text and data. Thus, on pcbios I implemented umalloc_low() like malloc(), allocating data out of a heap in BSS; the only difference is that the heap is linked into the bss16, i.e. low memory. Also, because the expected usage pattern involves a persistent need to interface with something, there is no ufree_low(); memory allocated is kept until gPXE shuts down. This lets the allocator itself be extremely tiny.

For the FireWire side of the equation, I decided on the concept of a “portal structure” aligned to a 16-byte boundary within low memory. It contains 8 bytes worth of magic, and fields “request” “reply” and “address”, that a debugging host can use to connect to some FireWire-accessible service and gain access to a service-specific communication structure (containing e.g. ring buffers and state fields). It's implemented in a way that avoids races if multiple debugging hosts try to connect at the same time (which is probably overkill, but it's the Right Thing). Currently I've implemented three services over the FireWire debug link, two of which are broadly useful on machines that don't have a serial port:

  • GDB over FireWire (gdbfire), with a host-side utility program firegdb (used to be firebug, but the name's already taken by a popular Firefox extension) that can either connect GDB automatically for you or listen for TCP debugger connections and proxy them over the FireWire link;
  • Console over FireWire (fwconsole), with a host-side utility program fireconsole (compiled from the same source as firebug due to the high level of similarity between the two) that acts as a simple interactive terminal emulator, optionally printing all of gPXE's output to a local file; and
  • File transfer over FireWire (fwload), which I wrote for my own use while developing sky2; I wanted a way to load a new gPXE quickly onto a machine whose only supported booting mechanism was a CD-ROM drive. I burned a version of gPXE with fwload support onto a CD, booted off it, and chained the gPXE I wanted to test over FireWire. I don't expect this will be generally useful.

It should be easy enough to do other things (IP over FireWire, anyone?) if people feel the need for them. :-)

Tuesday, 4 August

Spent most of today figuring out the idiosyncracies of the linker as it relates to gPXE.

Currently we have two macros for managing “I want to pull in this object”: PROVIDE_SYMBOL() and REQUIRE_SYMBOL(). (REQUIRE_OBJECT() just does REQUIRE_SYMBOL() on a special obj_objectname symbol provided automatically by compiler.h for each object.) I discovered recently that REQUIRE_OBJECT() doesn't actually require anything; it will pull in the object if it exists, but if not a linker error will not be produced. While this can be useful behavior, it doesn't fit the semantics of the word “require”.

In effect, REQUIRE_SYMBOL(foo) generates assembler code like the following:

        .equ    __need_foo, foo

That creates an absolute (not directly associated with a piece of memory) symbol called __need_foo whose value is that of the symbol foo. Since foo is not defined in the same file, that creates an entry in the symbol table for the file it shows up in marking foo as undefined. The linker will try to resolve such references at link-time, and searches through all the gPXE object files for a symbol named foo. If it finds one, that object file gets pulled into the link and its functionality will be available to gPXE at runtime.

However, an interesting thing about the above line may have occurred to you. The special __need_foo absolute symbol is never actually used. If foo remains undefined despite the linker's searching, __need_foo will be undefined too… but so what? There's no reason for the linker to stop linking just because a symbol is undefined, if it's not going to impact the code. When it runs, gPXE doesn't even have the symbol table; why would it matter what's in it?

The replacement for REQUIRE_SYMBOL(foo) (the old behavior has been renamed to REQUEST_SYMBOL(foo)) should clarify:

  extern char foo;
  static char * __require_foo __attribute__ (( section ( ".discard" ), used )) = &foo;

This doesn't just define an absolute symbol; it defines a global variable, a symbol with storage space attached, that stores the value of symbol foo. (Symbol values to the linker are like variable addresses to the compiler.) The variable (__require_foo in this case) is placed in a special output section, .discard, which we can tell the linker to throw away in the final linking stage (so that the result doesn't take up precious bytes in the final gPXE). It's marked used so the compiler doesn't throw it away thinking it's never used. This time, when the linker goes to resolve its undefined symbols and can't find any foo, it'll notice there's a relocation on it—an instruction to the linker that says “I don't know what the address of foo is yet, because it's not in this file; when you figure it out, please put it here in the variable __require_foo.” The same sort of thing is used when you call an external function; the linker knows how to interpret the machine code and change the address being jumped to. And when there's a relocation the linker can't satisfy, it has to refuse to link the program, since its execution without part of its code or data set properly would be undefined. Thus, this formulation of REQUIRE_SYMBOL() really requires.

I've also added macros EXPORT_SYMBOL() and IMPORT_SYMBOL(), that can be used for REQUEST_SYMBOL()-like behavior in cases where you actually want to use the symbol being requested. It's necessary for some cooperation from the file providing the symbol (saying EXPORT_SYMBOL(symname)), because there's no way to do it otherwise (it'd be necessary for the same undefined symbol to be both strong and weak, which is impossible). I leave it to the curious to look at the code to see how these work :-)

Finally, I spent several hours working on a desirable functionality called REQUIRE_IF()—pull in one object file only if another is already being compiled in. This could be used for “pull in WEP if 802.11 is compiled in”, “pull in undiheader if undiprefix is compiled in”, etc. Unfortunately, the limitations of linker script syntax and a particularly braindead way of handling undefined symbols (refusing to search libraries for them) combine to make the only possible solution I could find extremely ugly. If I do wind up implementing it, the gory details will wind up on this page, but I'm hoping we can agree to use a simpler method requiring slightly more human intervention. :-)

I'm not going to push these changes to staging until we have a solution to the REQUIRE_IF() fiasco, but here's the commit so far:

Wednesday, August 5

Back to debugging ath5k…

My Internet connection is experiencing an outage as I write this, so commits will not be available until sometime tomorrow. (I'm borrowing the neighbors' Wi-Fi to post this, but I'm using my laptop for it and it doesn't have my git tree.)

I solved the problem I was having with ath5k: I was processing the status bits in a way that is only suitable for interrupt-driven use of the card. Making a fairly obvious fix (removing the “is interrupt pending” check) allows it to work in polling mode. With this, I'm able to scan for networks and associate with a WPA network, including both sending and receiving packets.

Unfortunately, there's a memory corruption bug somewhere, of an extremely difficult-to-track variety. When I DHCP with my neighbor's network, gPXE locks up after receiving the DHCPOFFER. It appears that it's the card's DMA that's doing the corruption, as the data structures I set up to track things (in hopes of reading them out over FireWire after the lockup) get zeroed without triggering gdb watchpoints.

I did find a rather nifty use for FireWire: with -finstrument-functions (which asks gcc to insert calls to special functions at the beginning and end of every function) and a very small amount of code, one can keep track of function calls and wind up with a backtrace on demand, even after a lockup (as long as it doesn't zero out memory like this one is!) I'll post the code for this tomorrow.

Another strange issue: -finstrument-functions causes gcc to report phantom “may be used uninitialized in this function” that don't show up without that option.


QR Code
QR Code soc:2009:oremanj:journal:week11 (generated for current page)