One of the premiere tools in FreeBSD CI work is poudriere. It's a collection of shell scripts that leverage FreeBSD jails (chroot on steroids; build-in containers) and ZFS to build ports in a clean environment. You can also "cross" build for different versions of FreeBSD (e.g. on my 11-STABLE box, I can also build 10-STABLE and 12-CURRENT, although forward-compatibility to 12- can be tricky because of kernel changes). It will even truly cross build, but that's beastly slow due to QEMU (e.g. my Skylake box can just about keep up with my Pine64). And it supports multiple ports trees, so you can do builds for the official ports tree and for local experiments too.

A typical invocation of poudriere looks like this:

poudriere bulk -j 111amd64 -p github-kde -t -i -C math/freemat

That means "build packages in the 111amd64 jail, from the ports tree called github-kde; test each port, afterwards give an interactive shell, remove and rebuild a clean version of math/freemat". The -C is generally only used when building something multiple times in the same environment as you refine a port. There's also a -c "rebuild the whole world" (also known as "damn, I may as well go to bed then") flag.

Poudriere will grind away at dependencies and everything, and in the end spits out a nicely colored status line; it looks like this (here, I was rebuilding octave in order to test Qt5 compatibility, and most of the dependencies were already done).

Screenshot of Poudriere's console output
During the build, poudriere can run hooks in response to various events. Those events include build success, or failure (in various flavors). Using the hooks, it's easy to move the errors to a separate directory, to end up with the build logs of those things that actually failed. Tobias has done that, and we end up with a directory listing like this:

We call a listing like this fallout. Ports that fail during a poudriere run. Since the poudriere run is often a test-run for the upgrade of an important package (e.g. upgrading CMake is tested by building 2500+ packages), handling the fallout afterwards is important: we need to go through each failed port and figure out why it has failed.

In the screenshot above, Coin was a C++-compatibility issue; so were freeMat and ampas. Apviv is hilariously silly bad C++ code (from 2014, so only in today's context is it bad). .. and so on, and so on. Most recently, I was working from the top of the list, Tobias from the bottom, fixing actual CMake issues and optionally fixing non-CMake issues (e.g. all the dodgy C++ code). And then it struck me, we need better tool support for our very simple workflow.

Wanted!

What we need is a way to associate two pieces of data with each entry in that directory.

  • Who has "claimed" the entry (file) to look at. This is just to prevent double efforts. It should also be possible to "unclaim" an entry.
  • Tags on an entry, explaining what the problem is (e.g. "derp++", "CMake", "Upstream gone").
Screenshot showing possible design for workflow tool. Kolourpaint don't fail me now! Three failed ports, with an image indicating who has claimed them and some tags.

In a way, it's like an issues tracker, only slightly more free-form. There doesn't have to be any kind of persistence: the workflow applies to one run of poudriere and the next one is basically independent. It might be nice to have user-selectable nicknames, but the owner could be indicated as an IP address, or a hash, or a color .. perhaps the set of possible tags should persist from one incarnation to another.

Does anyone know of an existing tool that does this?

If all else fails, I may sit down with Cutelyst and see what I can do there (or in Pyramid, or whatever; most of the work is probably in the CSS and Javascript, with only a very small core to serve up the page and handle the AJAX requests).