A collection of headless, diskless servers, utilising a range of operating system images.
A web-based control system enables a system administrator to pick from a range of operating system images for use by the specific machine, thereby adding it to the “pool” of available servers / workstations under his control.
We need to serve one operating system image to some machines and other types of operating system images to other machines. The operating system images contain software specific to a task and the kernels and configurations are optimised accordingly.
As the machines are identified by their MAC addresses, we decided to use these as the basis for which operating system image to serve each machine.
The problem is therefore: how to get gpxe to load up the correct operating system image based on mac address
There are a number of ways to solve this problem:
Use built-in (network adapter) pxe to get pxelinux.0 details from DHCP server and then get and boot pxelinux.0 from a tftp server. OS images are then downloaded to the machine via tftp.
pxelinux has support for MAC based directories and config files, but tftp is very slow and prone to issues.
Flash the network adapter with gpxe. This solution is covered here: http://etherboot.org/wiki/romburning
gpxe can then download and run a mac address-specific script, which in turn downloads the operating system image.
We decided not to use this for the first run of testing.
Use built-in (network adapter) pxe to boot gpxe, use gpxe to get a script from a webserver which then downloads an operating system image according to the Mac address.
Use built-in (network adapter) pxe to boot gpxe that has an Embedded script which appends the mac address to the http://{next-server}/bootdirectory url.
We elected to use Method 3 for the short term, switching to method 4 and then finally - once everything was working as intended - method 2.
The process of using pxe to boot gpxe is covered here: http://etherboot.org/wiki/pxechaining but we have included a step-by-step guide here for completeness.
We found ISC DHCPD to be adequate for Linux systems and can be obtained here: http://www.isc.org
The one we used is “tftp-hpa” and is available here: http://www.kernel.org/pub/software/network/tftp/
Your tftp server doesnt have to be the same as your webserver, but is usually the case as the workload for either is very small.
Remember to make sure that you create a /tftpboot directory and actually start the tftp server.
We use our own modified version of apache, but any webserver will do.
We use the following file structure:
webserver/bootdirectory - Scripts live here
webserver/bootdirectory/images - Kernel and other images live here
Ensure that permissions are set so that files can be served from these directories and that the files themselves have the appropriate permissions set.
Put the following lines of code in your /etc/dhcpd.conf file:
next-server nn.nn.nn.nn;
if exists user-class and option user-class = “gPXE” {
filename http://yourwebserver/bootdirectory/boot.gpxe;
} else {
filename undionly.kpxe;
}
host node01 {
hardware ethernet xx:xx:xx:xx:xx:xx;
fixed-address 192.168.1.1;
}
host node02 {
hardware ethernet xx:xx:xx:xx:xx:xx;
fixed-address 192.168.1.2;
}
Note 1 - The “host node01…” directives are for assigning fixed ip addresses. They are not essential for this excercise, but included here for completeness. If you are NOT issuing fixed ip addresses, you need to configure your dhcpd.conf file accordingly.
Note 2 - Replace nn.nn.nn.nn with the ip address / name of your bootserver
Note 3 - Replace xx:xx:xx:xx:xx:xx with the mac addresses of the machines (fixed ip addresses only)
Note 4 - If you intend to use advanced gxpe features, you will need to include a range of directives in the beginning of the dhcpd.conf file as defined here: http://etherboot.org/wiki/dhcpd
Note 5 - Elsewhere in this tutorial, {next-server} is used in scripts. This is the same as http://bootserver.
Create a plain text file called “boot.gpxe” and put it in your webserver/bootdirectory/ directory.
boot.gpxe file:
#!gpxe
chain http://${next-server}/bootdirectory/${net0/mac}.gpxe
Create individual mac files and put them in the webserver/bootdirectory folder:
xx:xx:xx:xx:xx:xx.gpxe
#!gpxe
kernel http://${next-server}/bootdirectory/images/bzImage1
initrd http://${next-server}/bootdirectory/images/initrd1.gz
boot
yy:yy:yy:yy:yy:yy.gpxe
#!gpxe
kernel http://${next-server}/bootdirectory/images/bzImage2
initrd http://${next-server}/bootdirectory/images/initrd2.gz
boot
and so on, where xx:xx:xx:xx:xx:xx and yy:yy:yy:yy:yy:yy are MAC addresses of your machines.
Ensure that your DHCPD, TFTP and Web servers are running and try to boot a machine.
The boot process now becomes:
1) Machine PXE looks for DHCP server
2) DHCP allocates ip address to the machine and points it to the tftp server file: undionly.kpxe
3) Machine gets undionly.kpxe and runs it
4) Machine asks DHCP server for ip address but this time the DHCP server serves the machine the http://bootserver/bootdirectory/boot.gpxe address
5) Machine goes to http://bootserver/bootdirectory/boot.gpxe and downloads the script
6) Machine runs the boot.gpxe script
7) boot.gpxe points the machine to the http://bootserver/bootdirectory/{mac address} script
8) Machine runs the http://bootserver/bootdirectory/{macaddress}.gpxe script
9) {macaddress}.gpxe script points to which operating system images to use
10) Machine downloads operating system images specified in {mac address}.gpxe and boots.
The process will be simplified somewhat if a script were embedded and would become:
1) Machine PXE looks for DHCP server
2) DHCP allocates ip address to the machine and points it to the tftp server file undionly.kpxe
3) Machine gets undionly.kpxe file and runs it
4) Machine is pointed directly to the operating system images
5) Machine downloads the operating system images and boots
The “Final” (production) solution (solution 2) involves burning a custom gpxe boot rom to the network adapter of the machine and reduces the boot process to:
1) Machine is pointed directly to: http://bootserver/bootdirectory/{mac address}.gpxe
2) Machine downloads the operating system image files specified in {mac address}.gpxe and boots
This is of course a much simpler and elegant solution. However burning custom roms to network adapters might not be something you want to do during the setting up / testing stage of a system.