Kyle's diskless boinc cluster - SpaceHeaters@home

Note: This is a design in progress

Design Summary

  • Minimize use of painful technologies where and if possible (tftp, nfs)
  • Uses samba for network file access
  • Boyscout level of security - program credentials into gpxe boot rom
  • Controlled via ssh

Get libraries for certain binaries

ldd `which cifs.upcall` > /tmp/libs
ldd `which ssh` >> /tmp/libs
ldd `which sshd` >> /tmp/libs
ldd `which bash` >> /tmp/libs
ldd `which mount.cifs` >> /tmp/libs
ldd `which watch` >> /tmp/libs
ldd `which screen` >> /tmp/libs
# The Name Service Switch related libs aren't found with ldd
echo /lib/ >> /tmp/libs
echo /lib/ >> /tmp/libs
cat /tmp/libs | cut -f 2 -d '>' | cut -f 1 -d '(' | xargs -n 1 -I {} sh -c "readlink -f {} || echo {}" | sort | uniq | xargs -I {} cp {} .

embedding keys onto the rom image

Nothing needs to get changed. Files in the initrd provided via dhcp overwrite identically named files from the embedded initrd archive. It's still a good idea to segregate the embedded files for example /embed/*

kernel vmlinuz-2.6.31-14-generic
initrd smith.ilz
00:50 < Docteh> i attached a gzipped tar to a gpxe image, and its automatically used along with the initrd specified via a script via dhcp, is that a known
00:51 < Docteh> gzipped cpio, my bad
00:52 < Docteh> I like that this is happening, I'm just wondering if I can expect it to work like that in the future

Formats: cpio archive compressed with gzip is the classic default, but I've seen support recently for bzip2 lzma

In make menuconfig under General Settings
[*]   Support initial ramdisks compressed using gzip
[*]   Support initial ramdisks compressed using bzip2
[*]   Support initial ramdisks compressed using LZMA

Note: My assumption of the default gpxe script might be incorrect.

echo \#\!gpxe > default.gpxe
echo autoboot >> default.gpxe

Build the desired image (cwd = gpxe/src)

mkdir embed
put files in embed directory
# Check permissions, ssh private keys wont get used if they're not set to be -rw-r--r--
find embed -xdev |sort| cpio --quiet -H newc -o |TIME gzip > embed.img
make EMBEDDED_IMAGE=./default.gpxe,./embed.img bin/pcnet32.pxe
make EMBEDDED_IMAGE=./default.gpxe,./embed.img bin/10ec8139.rom

Burning an eeprom

Burning an eeprom is two steps, enabling bootrom with rset8139 and programming it I fired up RSET8139 in freedos with the bootrom socket empty and set the bootrom size to 128kb. RSET8139 has options for 8kb 16kb 32kb 64kb and 128kb. This appears to work if multiple cards are plugged in.

Note: running rset8139 with a 1mbit eeprom in the socket while the 8139 was set to not have a boot rom caused rset8139 to hang. can write to an eeprom on a rtl8139b/c board via a /proc interface. I compiled a 2.6.17 kernel and used that with ctflasher

Issues so far

cifs client doesn't appear to watch connections as well as it should

there is not much documentation on ProxyDHCP concept

QR Code
QR Code wiki:user:kylek (generated for current page)