Yo dawg, I heard you like FreeBSD Yeah, dogg. I hear ya.

This post carries on a bit about the way I build and test packages for FreeBSD. I only have one desktop machine, running FreeBSD amd64, and it needs to function as a desktop even while building and testing packages. Elsewhere, things like Project Neon and the OpenSUSE build service do something similar, on a much larger scale: building packages from various stages of development and delivering them to users. Here, though, I'm concentrating on end-to-end ports and packages testing for FreeBSD for a single computer and user.

To do this, I'm using both FreeBSD jails (containers, if you will) and VirtualBox VMs. This means everything runs on one host (although really it doesn't have to) and one monitor. The process looks like this:

Diagram of build steps and repositories Components of the build process with poudriere + VMs.

Very little of this is my original work. Previously I pointed to some poudriere tutorials; that was the nucleus of this build setup.

Build Configurations: poudriere uses ZFS in FreeBSD to manage the disk space used by the different build configurations. This makes a couple of things easy: creating and destroying configurations and keeping state or rolling back to previous known-good configurations. This means you can have several ports trees -- I've illustrated the default ports tree (which as of this writing is Qt 5.4.1, and KDE SC 4.14.3), area51 (which is Qt 5.4.1 and KDE Frameworks) and a plasma5 (which is Qt 5.5 and KDE Frameworks plus Plasma 5 Desktop).

Each ports tree is created with poudriere ports -c -p , which fetches the default ports tree. After that, I customize the tree by futzing directly with the filesystem. That can mean kdemerge (from area51), or editing ports files by hand. If needed, I can use ZFS snapshots to keep a known-good configuration around.

Each build jail is also created by poudriere, with poudriere jail -c -j . I show an amd64 and an armv6 jail here, since those are the architectures I build for. The armv6 jail takes a little more effort to set up, but that's described in the poudriere tutorial. These jails live on a ZFS filesystem too, so I can snapshot them and update if needed, or clone them cheaply.

Doing a build takes one build jail -- say amd64 -- and one ports tree -- say area51 -- and builds a particular set of packages for that combination, with poudriere bulk -f -j -p . All the resulting packages are dropped into yet another filesystem (easy to clean!); each set of packages is named for the two ingredients that went into it. The packagelist-file can be really simple: if can be a single line x11/plasma5-plasma-desktop to build the KDE Plasma 5 Desktop.

You can use any kind of web server to serve up the packages. I followed the straightforward nginx setup described in the tutorial.

But the upshot of all this is that I can grind out packages in a bunch of different configurations in a pretty straightforward fashion. Per package set the automation might be a little different, but as an example, amd64-area51 packages (preparing for the first KF5 packages on FreeBSD) go like this:

  1. rollback the area51 ports tree to a clean state
  2. update the base ports tree in the snapshot, reset snapshot to new clean state
  3. update area51
  4. merge area51 with the ports tree
  5. kick off a build

With the current gaggle of packages, this takes about 4 hours on my machine, and then I've got a new set of packages I can test.

Test Machines: that second half of the setup is doing actual testing of the packages; e.g. installing them and running stuff. For this, I use VirtualBox -- at least for the amd64 packages. It's cheap and simple and gives me a standard platform that I can recycle and use for subsequent tests.

I have one disk image, which I snapshot to a known good state. I find it vaguely amusing that I can use ZFS snapshots inside the VM (on the virtual disk), or snapshots in VirtualBox, or snapshot the disk image in its own ZFS filesystem on the host system. So many ways to achieve the same thing.

My basic disk image has XOrg server installed, twm, xterm, and has a user account configured for ssh access. The point is to have a usable system (yay twm!) on which I can configure and run the packages just built elsewhere. To try a new package set, I clone the existing machine, start it up, set the pkg(8) repository to the right URL (which is on the host machine of the VM), then snapshot it and pkg install qt5 (or whatever packages I'm interested in at the time).

Since my area51 builds only build Qt5, KDE Frameworks 5 and KDE Plasma 5 Desktop (plus dependencies that aren't already installed), it's possible that the VM needs packages that aren't built by poudriere in the build jails. So I have two pkg(8) configuration files in /usr/local/etc/pkg/repos, one to pull from the official packages repository, and one for my test repo.

[adridg@kafer ~]$ cat /usr/local/etc/pkg/repos/FreeBSD.conf 
FreeBSD: {enabled: yes}

[adridg@kafer ~]$ cat /usr/local/etc/pkg/repos/beastie.conf 
beastie: {
url: "pkg+http://beastie:8080/repo/101amd64-area51/.latest",
mirror_type: "srv",
enabled: yes
priority: 10
}

Aside from the URL -- it uses my in-house DNS to resolve the hostname of the poudriere build-machine -- the important thing in the second file is the priority: 10 line, which ensures that packages that are available from beastie will be preferred over packages from the official FreeBSD packages repository. As long as my build machine stays reasonably up-to-date, that's fine.

The standard snapshot for my VM uses only the official FreeBSD packages, so I can also do this:

  1. rollback the VM snapshot to a known-good state
  2. update all the packages (from the official packages), reset snapshot to new clean state
  3. install packages from area51 (e.g. pkg install plasma5-plasma-desktop)
  4. startx. Yay twm!
  5. futz around; I generally run startkde by hand inside the X11 session, since I don't know beforehand if it's going to work, nor if the fallback behavior is sensible.

And with all that, I can say here's KDE in FreeBSD in KDE in FreeBSD (with gobs of error messages from plasmashell, and I forgot to minimize some of the xterms so you could see the background; but the hamburger is there and KWin (X11) runs.):

Side-by-side screenshots TWM on the left, KDE Plasma 5 Desktop on the right, on a KDE 4 desktop all running FreeBSD 10-STABLE.

PS. Now is the time to start with a FreeBSD-themed splash or default background. I think Beastie could very well be turned to glass and smashed into triangular bits to fit into the look. Any interested artists?