Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revision Both sides next revision
dev:devmanual [2006/07/16 09:02]
stockholm created by copy-pasting from html developer manual and formatting that
dev:devmanual [2006/07/16 09:38]
stockholm
Line 132: Line 132:
  
  
 +
 +===== The architecture of Etherboot =====
 +Leaving aside the peripheral code for the loaders, the rest of Etherboot can be divided into the core, the drivers and miscellaneous (catch-all category). ​
 +<​file>​
 +        +------------------+------------------+
 +        |  Etherboot core  |   ​Miscellaneous ​ |
 +        +------------------+------------------+
 +        |           ​Driver support ​           |
 +        +-------------------------------------+
 +        |           ​Hardware drivers ​         |
 +        +-------------------------------------+
 +        |              Hardware ​              |
 +        +-------------------------------------+
 +</​file>​
 +
 +These parts are mapped on the directory structure in the src directory as follows: ​
 +  * ''​arch'':​ Architecture specific files. Each architecture has an own subdirectory,​ currently we have ''​i386''​ and ''​ia64''​.
 +  * ''​bin'':​ The directory where all binary files are built.
 +  * ''​core'':​ Etherboot core.
 +  * ''​drivers/​disk'':​ Drivers for floppies and hard disks.
 +  * ''​drivers/​net'':​ Drivers for NICs.
 +  * ''​firmware'':​ BIOS-specific files.
 +  * ''​include'':​ Include files.
 +  * ''​util'':​ Utility programs that are needed for building, but not included in the resulting roms.
 +
 +These directories are duplicated under arch where appropriate.
 +
 +
 +
 +
 +==== Etherboot core ====
 +The core of Etherboot handles the protocol for obtaining a network identity and for loading data over the network. This comprises the files ''​main.c''​ (main program), ''​config.c''​ (probe routines), ''​osloader.c''​ (support for various load image formats), ''​nfs.c'',​ ''​proto*.c''​ (support for network I/O) and ''​vsprintf.c''​ (printing routines). (More here.)
 +
 +
 +
 +==== Drivers ====
 +Generally each file represents a family of network adapters. For example ''​tulip.c''​ handles all adapters that use a Tulip compatible network controller, even if they are from different manufacturers. ''​3c90x.c''​ handles a whole family of related adapters from 3Com. 
 +
 +The interface between the core and the drivers is well-defined and explained in the Section called *FIXME* Writing an Etherboot Driver. ​
 +
 +The files ''​timer.h''​ and ''​timer.c''​ provide routines for access to the second hardware timer of the PC. This is used for implementing microsecond timeouts. ​
 +
 +The files ''​pci.h''​ and ''​pci.c''​ provide a PCI initialisation subsystem that is executed for PCI adapters. ​
 +
 +''​skel.c''​ is a skeleton driver that an aspiring driver writer can use as a starting point. ​
 +
 +
 +
 +==== Miscellaneous ====
 +In this category are all the files that don't fit into the first two categories.
 +
 +
 +==== External auxiliary programs ====
 + In Etherboot 5.0 and later there is the facility to load and run external auxiliary programs. As Etherboot runs in high memory. All the memory below that and above ''​0x10000''​ is fair game for loading. What if we did not load the target operating system but instead loaded an external program, one that returned to Etherboot after doing something. This something could be a fancy menu system. The advantage of this approach is that the menu system does not have to be compiled into Etherboot, which means it can be changed without changing ROMs, and can be much larger. ​
 +
 +How does such a menu program indicate to Etherboot which image is to be loaded next? Several new features of Etherboot make this possible. Firstly, a bit in the header of the loaded image is used to indicate that the loaded program intends to return to Etherboot, as opposed to never returning, in the case of an operating system. Next, the external program is passed a pointer to the BOOTP/DHCP structure obtained by Etherboot, which includes the filename field. Finally, the external program can return one longword of status to Etherboot to indicate various conditions. ​
 +
 +So here's how the external menu program would work. Etherboot is told to load an image which is a menu program. The program would present the user with a list of images to choose from. The presentation can be as simple as a numbered menu or as fancy as a point and click interface (if you can arrange to interface to the pointer). When a particular image has been chosen, the menu program alters the filename field in the BOOTP/DHCP structure and returns a longword of status indicating that Etherboot should retry the loading. Since the filename has been altered, Etherboot will end up loading the desired image instead of the menu this time around. Think of it as a program chaining facility.
 +
 +
 +
 +===== The development environment of Etherboot =====
 +Etherboot is written in C, with certain routines written in x86 assembler for access to machine resources or to do things not possible in C. The tools used are the GNU C compiler and GNU x86 assembler. ''​ld''​ is used to link the object files. An auxiliary program called ''​makerom.pl''​ converts this binary into a form suitable for writing into a ROM.
 +
 +A Makefile coordinates all the building procedures. However there are many network adapters that differ only in the PCI vendor and device IDs. This information must be programmed into the ROM header using ''​makerom''​. An auxiliary script called ''​genrules.pl''​ scans all driver source files for the device names and IDs and outputs make rules to a file called Roms that is included by the main Makefile to generate all the ROM images. Another file included by Makefile is ''​Config'',​ which contains user configurable build parameters.
 +
 +Let us take an example of a ROM image that we could build, ''​bin/​mx98715.zrom''​. According to this line in the file tulip.c: ​
 +<​file>​PCI_ROM(0x10d9,​ 0x0531, "​mx98715",​ "​Macronix MX987x5"​),</​file>​
 +the driver code is in ''​tulip.c''​ and the vendor ID is ''​0x10d9''​ and the device ID is ''​0x0531''​. If one were to say ''​make bin/​mx98715.zrom'',​ the following actions happen:
 +
 +  * Any needed utilities such as ''​bin/​nrv2b''​ are built. ​
 +  * ''​drivers/​net/​tulip.c''​ is compiled to ''​bin/​tulip.o''​.
 +  * The startup routine, ''​arch/​i386/​core/​start32.S''​ is assembled to ''​bin/​start32.o''​. ​
 +  * All the core Etherboot files, e.g. ''​core/​main.c'',​ ''​osloader.c'',​ etc, are compiled to corresponding object files in ''​bin/​*.o''​ and combined into an ''​ar''​ archive, ''​bin/​bootlib.a'',​ for convenience. ​
 +  * The startup routine (which must be first in the list of objects), the driver object, ''​bootlib.a'',​ the platform specific objects are linked to produce an ELF binary, linked to run at the relocation address of Etherboot, currently ''​0x20000''​. The ELF binary is converted to an Etherboot binary using GNU utilities. This binary is can be on non-x86 platforms by conversions. (To fill in later.) For the x86 platform, we need to prepend loaders for the chosen load method. ​
 +  * The prepended ROM loaders, ''​rloader'',​ and ''​prloader'',​ are produced by assembling ''​arch/​i386/​prefix/​loader.S''​ with different defines. The meaning of the prefixes are:
 +    * ''​rloader''​ = basic ROM loader, and
 +    * ''​prloader''​ = PCI header basic ROM loader.
 +  * The loaders contain the needed headers for the BIOS to recognise the code as an extension ROM. See the Section called *FIXME* The execution environment of Etherboot on the extension BIOS mechanism. ​
 +  * For example, in the case of ''​bin/​mx987x5.zrom'',​ this is for a PCI adaptor and we want a compressed ROM, so the loader is ''​prloader''​. This is prepended to the Etherboot image to generate a ROM image.
 +  * ''​makerom.pl''​ is run over this ROM image to pad it to the size of a standard EPROM. (8 kB, 16 kB, 32 kB, etc.) The PCI IDs are written into the image into the PnP structure at the right spots. A string identifying the device and Etherboot version is written into the unprogrammed bytes at the end of the ROM image, if space is available, and a pointer to that written into the PnP structure. The vendor string and the device string are normally printed by the PCI PnP BIOS at boot time. Makerom also calculates the checksum (both for the PnP structure, if present and for the whole image) and alters designated spare bytes in the image so that the checksums come out right. The image is then ready to be written into an EPROM. ​
 +  * By prepending alternate loaders to the Etherboot image, we can load the image from floppy, LILO/​SYSLINUX,​ PXE or DOS. In the case of floppy loading, the preloader is called ''​floppyload.bin''​. It is one sector (512 bytes) long and loaded from the floppy, starting at the beginning of the floppy. It then loads the blocks following itself, which it assumes to be an Etherboot image, into memory, then jumps to the start of it. 
 +  * In the case of LILO/​SYSLINUX booting, the preloader is ''​bin/​liloprefix.bin''​. It contains what looks like a floppy sector and a startup segment to LILO/​SYSLINUX and makes LILO/​SYSLINUX "​think"​ that this is a Linux kernel.
 +  * The loader for chaining from PXE is in ''​bin/​pxeprefix.bin'',​ and when prepended to an image, produces a file that can be downloaded and executed by a PXE conforming boot rom as a first stage loader. Etherboot then takes over the remainder of the load process.
 +  * Yet another preloader is not for a device but for an OS environment. This is ''​bin/​comprefix.bin'',​ and when prepended to an image, makes it look like a DOS .com executable, including the peculiarity of starting at ''​COM_SEGMENT:​0x100''​. All it does however is jump to the Etherboot image. ​
 +  * All the preceeding special preloaders have associated ''​Makefile''​ rules and are created by asking for images with the appropriate suffix. For example, if one wanted an image for writing onto a floppy one would say: 
 +<​file>​make bin/​mx98715.zdsk</​file>​
 +
 +
 +
 +===== Frequently asked questions =====
 +
 +==== How portable is Etherboot ? ====
 +Work leading up to 5.2 made Etherboot much more portable. The platforms on which Etherboot runs, other than the x86, are the Itanium and the Hammer. The portability issues can be looked at in two categories: ​
 +
 +Support routines needed for platforms need to be written. Platform independent and dependent code have been separated and it should be straightforward to identify the support that is needed for running Etherboot on a new platform. ​
 +
 +Byte order, operand size and alignment issues in the core routines and PCI drivers have been mostly dealt with, although there are some known places where the code has not been made portable, e.g. longword access of registers in the eepro100 driver. We hope to find these bugs as time goes by. In some cases, e.g. ISA bus NICs, there is no need to fix the problem as the ISA bus only exists on the PC platform (unless somebody attaches a NE2000 to a big-endian microcontroller and plans to run Etherboot on it). 
 +
 +Here are the low-level services that need to provided: writing and reading from the console, determining the size of memory, obtaining the an elapsed time, inserting a boot vector to be called, and a microsecond timer for short delays. Depending on the target environment,​ these services may be provided in different ways.
 +
 +==== How can I get Etherboot to boot a Cardbus (PCMICA) or a USB NIC? ====
 +Cardbus (PCMCIA) and USB NICs are interfaced to the CPU through their respective bus controllers. Before Etherboot can address the registers and memory on these NICs, it must initialise the bus controller appropriately. In Linux this is done by the PCMCIA and USB subsystems. In Etherboot, all this must be done by the code in the ROM. What needs to be written is a subsystem like the PCI subsystem in Etherboot. Volunteers are most welcome.
 +
 +===== Writing an EtherBoot Driver =====
 +
 +==== Preliminaries ====
 +So Etherboot does not have a driver for your network adapter and you want to write one. You should have a good grasp of C, especially with respect to bit operations. You should also understand hardware interfacing concepts, such as the fact that the x86 architecture has a separate I/O space and that peripherals are commanded with ''​out''​ instructions and their status read with ''​in''​ instructions. A microprocessor course such as those taught in engineering or computer science curricula would have given you the fundamentals. (Note to educators and students in computer engineering:​ An Etherboot driver should be feasible as a term project for a final year undergraduate student. I estimate about 40 hours of work is required. I am willing to be a source of technical advice.)
 +
 +Next you need a development machine. This can be your normal Linux machine. You need another test machine, networked to the development machine. This should be a machine you will not feel upset rebooting very often. So the reset button should be in working condition. :-) It should have a floppy drive on it but does not need a hard disk, and in fact a hard disk will slow down rebooting. Alternatively,​ it should have another network adapter which can netboot; see discussion further down. Needless to say, you need a unit of the adapter you are trying to write a driver for. You should gather all the documentation you can find for the hardware, from the manufacturer and other sources. ​
 +
 +==== Background information ====
 +There are several types of network adapter architecture. The simplest to understand is probably programmed I/O. This where the controller reads incoming packets into memory that resides on the adapter and the driver uses ''​in''​ instructions to extract the packet data, word by word, or sometimes byte by byte. Similarly, packets are readied for transmission by writing the data into the adapter'​s memory using ''​out''​ instructions. This architecture is used on the NE2000 and 3C509. The disadvantage of this architecture is the load on the CPU imposed by the I/O. However this is of no import to Etherboot (who cares how loaded the CPU is during booting), but will be to Linux. Next in the sophistication scale are shared memory adapters such as the Western Digital or SMC series, of which the WD8013 is a good example. Here the adapter'​s memory is also accessible in the memory space of the main CPU. Transferring data between the driver and the adapter is done with memory copy instructions. Load on the CPU is light. Adapters in this category are some of the best performers for the ISA bus. Finally there are bus mastering cards such as the Lance series for the ISA bus and practically all good PCI adapters (but not the NE2000 PCI). Here the data is transferred between the main memory and the adapter controller using Direct Memory Access. Setting up the transfers usually involves a sequence of operations with the registers of the controller. ​
 +
 +==== Structure of the code ====
 +==== Rom naming rules ====
 +==== Booting the code from a floppy ====
 +==== Booting the test code with another EtherBoot ROM ====
 +==== Writing the code ====
 +==== Things to watch out for ====
 +==== Tidying up ====
 +==== Keeping your driver in sync with the Linux version ====
 +===== A potted history of Etherboot =====
 +===== Appendix A. Draft Net Boot Image Proposal 0.3, June 15, 1997 =====
 +==== Preamble - the why ====
 +==== The target ====
 +==== Net Boot Process Description ====
 +==== Image Format with Initial Magic Number ====
 +==== Boot prom entry points ====
 +==== Example of a boot image ====
 +==== Terms ====
 +==== References ====
 +===== Appendix B. Compression algorithm, 1 July 1997 =====
 +===== Appendix C. Copying conditions of the compression code, 1 July 1997 =====

QR Code
QR Code dev:devmanual (generated for current page)