Daniel Verkamp: Automated regression testing

Wrote up a list of some potential test cases on notes page.

Discussed Autotest framework with mcb30; goals for the week:

• Create a list of test cases
• Get Autotest running and add a simple test to verify gPXE builds, then document
• Optionally get one simple functional test working with Autotest (boot Linux in qemu and somehow report success to the host)

Also mcb30 discovered the reason for the apparent increase in size with GCC 4.4.0 (unneeded .eh_frame sections were being counted in 'size' output); with a patch to eliminate these sections, GCC 4.4.0 now has the smallest total code size of all the GCC versions I'm testing (http://drv.nu/gpxe/size/).

Investigating Autotest architecture today.

Some notes:

• The Autotest client is in the “client” directory of the full download; this seems to be usable without any of the other pieces, at least for testing.
• Within client, there are several sample “control files”; for my initial goal (building gPXE), samples/build is the most interesting.
• The control files use “job” objects; the build sample uses job.kernel, which seems to be located in bin/kernel.py.
• bin/kernel.py has several methods involved in building a kernel, like “kernelexpand” (download, extract, patch kernel), “config” (use a specific config or defconfig), and “build”

Some investigation of cloning Autotest SVN repository with git (see http://git.drv.nu/?p=autotest.git).

Spent most of the day reading BuildBot documentation (http://djmitche.github.com/buildbot/docs/0.7.10/) - I think this might be a better fit than Autotest. Autotest is closely tied to testing the Linux kernel (although there is at least one other project, KVM, that is not using it to test Linux kernels, although the KVM build process is not under its version of Autotest's control). BuildBot, on the other hand, is designed to coordinate building and testing of arbitrary projects, and it is in use by several large projects (http://buildbot.net/trac/wiki/SuccessStories).