Daemons and friendly Ninjas
There are multiple buildsystem-backends available: in regular usage, CMake generates Makefiles (and does a reasonable job of producing Makefiles that work for GNU Make and for BSD Make). But it can generate Ninja, or Visual Studio, and other buildsystem files. It's quite flexible in this regard.
Recently, the KDE-FreeBSD team has been working on Qt WebEngine, which is horrible. It contains a complete Chromium and who knows what else. Rebuilding it takes forever.
But Tobias (KDE-FreeBSD) and Koos (GNOME-FreeBSD) noticed that building things with the Ninja backend was considerably faster for some packages (e.g. Qt WebEngine, and Evolution data-thingy). Tobias wanted to try to extend the build-time improvements to all of the CMake-based ports in FreeBSD, and over the past few days, this has been a succes.
Ports builds using CMake now default to using Ninja as buildsystem-backend.
Here's a bitty table of build-times. These are one-off build times, so hardly scientifically accurate -- but suggestive of a slight improvement in build time.
| Name | Size | GMake | Ninja |
|---|---|---|---|
| liblxt | 50kB | 0:32 | 0:31 |
| llvm38 | 1655kB | * | 19:43 |
| musescore | 47590kB | 4:00 | 3:54 |
| webkit2-gtk3 | 14652kB | 44:29 | 37:40 |
Or here's a much more thorough table of results from tcberner@, who did 5 builds of each with and without ninja. I've cut out the raw data, here are just the average-of-five results, showing usually a slight improvement in build time with Ninja.
| Name | av make | av ninj | Delta | D/Awo |
|---|---|---|---|---|
| compiler-rt | 00:08 | 00:07 | -00:01 | -14% |
| openjpeg | 00:06 | 00:07 | +00:01 | +17% |
| marble | 01:57 | 01:43 | -00:14 | -11% |
| uhd | 01:49 | 01:34 | -00:15 | -13% |
| opencacscade | 04:08 | 03:23 | -00:45 | -18% |
| avidemux | 03:01 | 02:49 | -00:12 | - 6% |
| kdevelop | 01:43 | 01:33 | -00:10 | - 9% |
| ring-libclient | 00:58 | 00:53 | -00:05 | - 8% |
Not everything builds properly with Ninja. This is usually due to missing dependencies that CMake does not discover; this shows up when foo depends on bar but no rule is generated for it. Depending on build order and speed, bar may be there already by the time foo gets around to being built. Doxygen showed this, where builds on 1 CPU core were all fine, but 8 cores would blow up occasionally.
In many cases, we've gone and fixed the missing implicit dependencies in ports and upstreams. But some things are intractable, or just really need GNU Make. For this, the FreeBSD ports infrastructure now has a knob attached to CMake for switching a port build to GNU Make.
- Normal:
USES=cmake - Out-of-source:
USES=cmake:outsource - GNU Make:
USES=cmake:noninja gmake - OoS, GMake:
USES=cmake:outsource,noninja gmake - Bad:
USES=cmake gmake
For the majority of users, this has no effect, but for our package-building clusters, and for KDE-FreeBSD developers who build a lot of CMake-buildsystem software in a day it may add up to an extra coffee break. So I'll raise a shot of espresso to friendship between daemons and ninjas.
(Image credits: Beastie by Marshall Kirk McKusick on FreeBSD.org, Ninja by irkeninvaderkit on deviantart)