PXE Booting Utilities With FreeBSD

Last updated 2010-07-29

Available in HTML or PDF. Created with AsciiDoc.

Introduction

Numerous useful utilities like Clonezilla, HDT, and FreeBSD’s "memstick" are available as bootable images. A PXE boot server can make these utilities more convenient and accessible. With PXE, no external media is needed, just boot the target computer from the network.

We will set up all of the components necessary to provide PXE booting to any computer networked with the FreeBSD server. Instead of simply booting a single install CD image, this setup will boot into a menu with multiple utilities.

A Note About Security

A reasonable effort has been made to present configurations that are secure, but no guarantees are made or implied as to the safety or accuracy of this information. Use this information at your own risk, and test and verify before implementing it on your own network.

Baseline Setup

FreeBSD server address of 192.168.1.1.

A plain IP address is used in the menu files instead of a server hostname because Clonezilla and some other initramfs-based utilities have problems with DNS resolution on boot. See Preprocessing Menus With cpp for a way to define hostname constants in the menus.

net/isc-dhcp41-server installed and already working as a DHCP server.

ftp/tftp-hpa installed as a TFTP server. It’s faster and more capable than FreeBSD’s tftpd.

gpxelinux.0 from the SYSLINUX project will be used as a PXE boot loader.

DHCP Configuration

A computer booting with PXE from the network looks for a boot file name from the local DHCP server. Ours is the not-yet-installed gpxelinux.0.

Some TFTP clients expect the next-server value to provide the address of the TFTP server. In this example, the DHCP and TFTP servers are on the same computer, but that’s not required.

Add these two settings to the "subnet" block of /usr/local/etc/dhcpd.conf and restart the DHCP server:

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.100 192.168.1.200;
        next-server 192.168.1.1;  # TFTP server address
        filename "gpxelinux.0";   # PXE boot loader filename
}

# /usr/local/etc/rc.d/isc-dhcpd restart

TFTP Configuration

After the PXE-booted computer receives a filename from DHCP, it has to actually load that file. TFTP is the only file transfer protocol supported by most PXE loaders.

The TFTP directory will hold all of the floppy and CD images or "live" filesystem trees from bootable utility disks. Since that directory can be quite large, this example places it in /usr. A subdirectory to hold the boot menu files is also needed:

# mkdir -p /usr/tftpboot/images
# mkdir /usr/tftpboot/pxelinux.cfg

Enable the TFTP server in /etc/inetd.conf:

tftp    dgram   udp     wait    root    /usr/local/libexec/in.tftpd     in.tftpd -p -s /usr/tftpboot -B 1024

Testing on my network showed that limiting block size to 1024 bytes provided the best performance, and prevented errors sometimes seen with larger blocks. Adding two -v options to the in.tftpd line for verbose logging makes errors easier to diagnose.

tftp-hpa can transfer large files. Other TFTP servers—or the TFTP clients used by utility disks—may fail with files larger than 32M. Alternate protocols are discussed in Advanced Options.

TFTP allows writes to files owned by the TFTP user. in.tftpd runs as user nobody by default; another user can be specified with -u. For read-only usage, make sure that none of the files in /usr/tftpboot are owned or writable by the TFTP user.

Enable inetd in /etc/rc.conf and restart it:

inetd_enable="YES"

# /etc/rc.d/inetd restart

PXE Boot Loader Installation

We’ll use gpxelinux.0, a PXE boot loader from the SYSLINUX package. Download the archive from http://syslinux.zytor.com/wiki/index.php/The_Syslinux_Project.

Copy the PXE boot loader and some other needed files to the right locations:

# tar xzf syslinux-4.01.tar.gz -C /tmp
# cp /tmp/syslinux-4.01/gpxe/gpxelinux.0         /usr/tftpboot/
# cp /tmp/syslinux-4.01/com32/menu/menu.c32      /usr/tftpboot/
# cp /tmp/syslinux-4.01/com32/menu/vesamenu.c32  /usr/tftpboot/
# cp /tmp/syslinux-4.01/com32/modules/reboot.c32 /usr/tftpboot/
# cp /tmp/syslinux-4.01/com32/modules/chain.c32  /usr/tftpboot/
# cp /tmp/syslinux-4.01/memdisk/memdisk          /usr/tftpboot/

File Descriptions

gpxelinux.0	PXE boot loader
menu.c32	text menu module
vesamenu.c32	graphic menu module
reboot.c32	reboot module
chain.c32	chain loader module
memdisk	memory disk emulator

Clonezilla

Clonezilla is a great bootable backup utility. Download the zip or ISO file from http://www.clonezilla.org. Only three files are needed from the archive: vmlinuz, initrd.img, and filesystem.squashfs.

unzip can extract those files directly into images/clonezilla (based on the instructions at http://www.clonezilla.org/clonezilla-live/livepxe.php):

# unzip -j clonezilla-live-1.2.5-17-i686.zip -d /usr/tftpboot/images/clonezilla/ \
live/vmlinuz live/initrd.img live/filesystem.squashfs

To create a menu that will be displayed after the PXE boot, create and edit the menu file /usr/tftpboot/pxelinux.cfg/default:

ui menu.c32
menu title Utilities

label clonezilla
  menu label Clonezilla
  kernel images/clonezilla/vmlinuz
  append boot=live union=aufs noswap vga=788 fetch=tftp://192.168.1.1/images/clonezilla/filesystem.squashfs
  initrd images/clonezilla/initrd.img

Menu Commands

ui	loads the menu.c32 text menu module.
menu title	sets the title shown at the top of the menu.
label	creates a menu entry.
kernel	loads the kernel; the path is relative to the TFTP server’s root directory, /usr/tftpboot/.
append	adds options to the kernel command line. Clonezilla uses initramfs, and fetch is an initramfs-specific option that tells the kernel where and how to download the live filesystem image.
initrd	points to the initial ramdisk image.

The live filesystem image will use about 100M of the target computer’s memory.

To test the configuration, boot your target computer and choose "Boot from network" or "PXE". After a few seconds, a blue text menu with the single Clonezilla option appears. Press Enter to boot and start Clonezilla.

System Rescue CD

Another very useful utility is from http://sysresccd.org/. As before, only a few files are needed from the ISO image:

# mkdir /usr/tftpboot/images/systemrescue/
# mount -t cd9660 /dev/`mdconfig -f systemrescuecd-x86-1.5.7.iso` /mnt
# cp /mnt/isolinux/rescuecd /usr/tftpboot/images/systemrescue/
# cp /mnt/isolinux/initram.igz /usr/tftpboot/images/systemrescue/
# cp /mnt/sysrcd.* /usr/tftpboot/images/systemrescue/
# umount /mnt
# mdconfig -d -u 0

The menu entry is also similar:

label systemrescue
  menu label System Rescue
  kernel images/systemrescue/rescuecd
  append setkmap=us netboot=tftp://192.168.1.1/images/systemrescue/sysrcd.dat
  initrd images/systemrescue/initram.igz

The System Rescue CD includes additional 64-bit and alternate kernels: rescue64, altker32, altker64. One of these other kernels may work on a target system that has problems booting or running the the rescuecd kernel. To add these kernels to the PXE menu, copy them from the isolinux directory of the ISO to the images/rescuecd directory and create additional menu entries in the default menu file. They all share the same initram.igz and sysrcd.dat files already present for the rescuecd kernel.

sysrcd.dat is about 200M, and can take a long time to load via TFTP. Faster protocols are discussed in Advanced Options.

HDT Floppy

Hardware Detection Tool is a bootable floppy available from http://www.hdt-project.org/wiki.

Put the hdt.img file in the images directory:

# cp hdt.img /usr/tftpboot/images/

The menu entry is uncomplicated:

label hdt
  menu label Hardware Detection Tool Floppy
  kernel memdisk
  initrd images/hdt.img

kernel is used to load the memdisk module, which allocates host memory for a disk image and then simulates an actual disk drive with the image.

initrd is used to load the HDT floppy image.

Memtest86+ 4.00 is also included on this floppy, eliminating the need for another image. It’s at the bottom of the first screen shown by HDT.

memdisk automatically decompresses zip or gzip files. The smaller files load faster and CRC checks provide data integrity.. Just compress the image and use the filename of the compressed image in the menu entry:

# gzip /usr/tftpboot/images/hdt.img

label hdt
  menu label Hardware Detection Tool Floppy
  kernel memdisk
  initrd images/hdt.img.gz

Reboot

A menu option to reboot the target computer uses the reboot.32 module:

label reboot
  menu label Reboot
  kernel reboot.c32

The Complete Text Menu

/usr/tftpboot/pxelinux.cfg/default

ui menu.c32
menu title Utilities

label clonezilla
  menu label Clonezilla
  kernel images/clonezilla/vmlinuz
  append boot=live union=aufs noswap vga=788 fetch=tftp://192.168.1.1/images/clonezilla/filesystem.squashfs
  initrd images/clonezilla/initrd.img

label systemrescue
  menu label System Rescue
  kernel images/systemrescue/rescuecd
  append setkmap=us netboot=tftp://192.168.1.1:/usr/tftpboot/images/systemrescue/sysrcd.dat
  initrd images/systemrescue/initram.igz

label hdt
  menu label Hardware Detection Tool Floppy
  kernel memdisk
  initrd images/hdt.img

label reboot
  menu label Reboot
  kernel reboot.c32

Virtual Machines

VirtualBox (/usr/ports/emulators/virtualbox-ose) is the best-supported virtual machine host for FreeBSD.

To PXE-boot virtual machines, enable the host’s VirtualBox network module in /etc/rc.conf and start the script:

vboxnet_enable="YES"

# /usr/local/etc/rc.d/vboxnet start

Set the guest’s network interface to "Bridged Adapter" and select the "PCnet-PCI II" emulated adapter. Start the VM, press F12 for the boot menu, then press L to boot from the LAN.

Advanced Options

Faster Transfers With HTTP

TFTP is not particularly fast when transferring the 200M or larger compressed filesystems used by some bootable utilities. Newer utilities can often retrieve their filesystem images by HTTP, which is much faster. One target system took three minutes to load the sysrcd.dat file by TFTP. With HTTP, it took only 18 seconds.

Installation and basic configuration of Apache 22 is, as they say, "left as an exercise for the reader".

Add a link in your DocumentRoot directory to the TFTP root directory:

# ln -s /usr/tftpboot/images/ /usr/local/www/apache22/data/

/usr/local/etc/apache22/httpd.conf is configured so image directory access is only allowed to specific portions of the network:

<Directory "/images">
    Order Deny,Allow
    Deny from all
    Allow from 192.168.1.0/24
    Options Indexes
</Directory>

The menu entries in /usr/tftpboot/pxelinux.cfg/default are then changed to use HTTP. With the images directory linked, paths will be the same as for TFTP. gpxelinux.0 can even load the kernel and initrd files with HTTP.

label clonezilla
  menu label Clonezilla
  kernel http://192.168.1.1/images/clonezilla/vmlinuz
  append boot=live union=aufs noswap vga=788 fetch=http://192.168.1.1/images/clonezilla/filesystem.squashfs
  initrd http://192.168.1.1/images/clonezilla/initrd.img

label systemrescue
  menu label System Rescue
  kernel http://192.168.1.1/images/systemrescue/rescuecd
  append setkmap=us netboot=http://192.168.1.1/images/systemrescue/sysrcd.dat
  initrd http://192.168.1.1/images/systemrescue/initram.igz

See gPXE for more information on gPXE’s capabilities.

NFS: Network File System

Memory-resident file systems can be limiting. At best, they steal memory from the operating system. At worst, a RAM filesystem may be difficult to create or not available at all.

NFS allows remote filesystems to be mounted over the network, without having to transfer a large data file or lose a large chunk of RAM on the target computer.

/etc/exports lists the directories to be network-mounted along with access limits and permissions. Export the /usr/tftpboot directory:

/usr/tftpboot -alldirs,ro -mapall=nobody:nobody -network 192.168.1.0/24

Modify /etc/hosts.allow to allow NFS access to the internal network by adding a line just before the existing rpcbind : ALL : deny line:

rpcbind : 192.168.1.0/24 : allow

NFS server programs are enabled in /etc/rc.conf:

rpcbind_enable="YES"
nfs_server_enable="YES"
mountd_flags="-r"

The NFS server will start on reboot. For now, start it manually:

# /etc/rc.d/rpcbind start
# /etc/rc.d/nfsd start

FreeBSD With NFS

The FreeBSD "livefs" image is a working FreeBSD system that can be used without installation on the target machine. An NFS-mounted filesystem can replace the CD or memory stick media for faster operation.

Download the livefs memory stick image from ftp://ftp.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/8.1/ and copy the entire directory structure to a subdirectory in the TFTP images directory:

# mount /dev/`mdconfig -f FreeBSD-8.1-RC2-i386-memstick.img`a /mnt      <1>
# rsync -aH /mnt/ /usr/tftpboot/images/freebsd/
# umount /mnt
# mdconfig -d -u 0

Note the "a" after that second backtick. mdconfig will return md0 when it creates the device. The UFS file system on that device is in the md0a partition.

Modify /etc/gettytab to autologin as root, and set the local DNS server:

# sed -i -e 's/:ht:np:/:al=root:ht:np:/' /usr/tftpboot/images/freebsd/etc/gettytab
# echo 'nameserver 192.168.1.1' > /usr/tftpboot/images/freebsd/etc/resolv.conf

This leaves a fairly large FreeBSD directory, about 849M. If the installation or package directories won’t be used, 600M of space can be reclaimed by deleting them:

# rm -rf /usr/tftpboot/images/freebsd/8.1-RC2
# rm -rf /usr/tftpboot/images/freebsd/packages

FreeBSD’s pxeboot(8) loader looks to the DHCP server for a root-path that identifies the NFS file system root, so modify dhcpd.conf again:

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.100 192.168.1.200;
        next-server 192.168.1.1; # TFTP server address
        filename "gpxelinux.0";  # PXE boot loader filename
        option root-path "/usr/tftpboot/images/freebsd";  # NFS root
}

Anything not running FreeBSD may be surprised, or more likely appalled, to find a FreeBSD file tree NFS-mounted where it was least expected. Fortunately many of the Linux systems allow a manual NFS root to be passed to the kernel.

Restart the DHCP server after changing settings:

# /usr/local/etc/rc.d/isc-dhcpd restart

The menu entry just PXE-boots the FreeBSD pxeboot loader:

label freebsd
  menu label FreeBSD LiveFS
  pxe images/freebsd/boot/pxeboot

XUbuntu With NFS

Setting up XUbuntu to use an NFS filesystem is similar to FreeBSD. NFS mounting the 643M filesystem.squashfs leaves it on the NFS server instead of copying the whole thing into memory on the target computer.

Why XUbuntu? Because shut up, that’s why!

Download the ISO image from http://xubuntu.org/ and copy the directory tree to a subdir in images:

# mkdir /usr/tftpboot/images/xubuntu
# mount -t cd9660 /dev/`mdconfig -f xubuntu-10.04-desktop-i386.iso` /mnt
# rsync -a /mnt/ /usr/tftpboot/images/xubuntu/
# umount /mnt
# mdconfig -d -u 0

The menu entry sets the netboot and nfsroot options:

label xubuntu
  menu label XUbuntu
  kernel images/xubuntu/casper/vmlinuz
  append boot=casper netboot=nfs nfsroot=192.168.1.1:/usr/tftpboot/images/xubuntu
  initrd images/xubuntu/casper/initrd.lz

Usefulness of NFS varies depending on a particular target computer’s Ethernet interface. For example, NFS-booting FreeBSD or XUbuntu fails on an Acer Aspire One D250, apparently because of driver problems with the oddball Atheros wired Ethernet controller on that system. Other computers with better-supported Ethernet boot from the same NFS server without complaint.

Graphic Menus

The VESA menu module displays menus in graphics mode, and allows background graphics. Modify the ui command at the start of the menu to load vesamenu.c32 and add a menu background command to load a background picture. The picture is PNG, 640x480, 8-bit color. Samples of the various menus can be seen at http://syslinux.zytor.com/wiki/index.php/Screenshots.

ui vesamenu.c32
menu title Utilities
menu background backgnd.png
...

Conclusion

Trying to find specific examples while setting up PXE booting was frustrating for me. The ones that were out there generally only covered the simplest situations. Hopefully the examples and specifics here will reduce some of that frustration for others.

Appendix A: Complete VESA Graphic Menu