Calamares is a modular installer for Linux distributions using Qt and KDE technologies. It is used by dozens of Linux distro’s for that crucial step of “get the ISO onto the HDD”, or some modern variant thereof. It’s modular, so distro’s can pick-and-choose what is needed: OpenRC or systemd, for instance. But it’s hard to cover everything that Linux distributions might need, so Calamares also has an “extensions” repository for more specialised modules. Let’s take a look.

Calamares window with modules highlighted
Calamares window with modules highlighted

The screenshot here shows the first screen of Calamares (in demonstration mode, so “real” distro’s will probably have distro-specific styling). There’s a number of user-visible “pages” to Calamares: welcome, location, etc. There are modules that are responsible for each of those; the main Calamares distribution contains a configurable welcome module, and an alternative QML-based welcomeq module as well – plenty of choice for distro’s.

There are more modules than what is visible here, though: all the invisible steps like making a user, settting up the display manager, configuring OpenRC (or systemd) .. those are modules as well. For these invisible steps there are certainly some that make sense only for specialised distro’s, so while Calamares aims to be modular and address “most” of what a Linux needs to get installed, there is always one more module for that special case (like pre-installing CDE theming, or whatever).

So in the end there are two repositories for Calamares bits:

  • Calamares which is the main program, libraries, support bits, CMake infrastructure and all the modules that are generally useful (and some that aren’t, for historical reasons),
  • Calamares Extensions which contains more modules, more branding examples, and generally shows off that Calamares can be extended with “third party” code as well: distro’s can use this also to build their own collection of modules (although, as always, pull requests are welcome: I’d be happy to move as much as possible upstream if it is of interest to more than one single distro).

One extra ideas behind the extensions repo is that it is a test of how-reusable-is-Calamares-code and does-my-CMake-code-actually-work. So it is supposed to (naively) build C++ modules that use the full plugin API of Calamares. But this gives rise to a slight problem: how to shuffle the latest bits around. In a Continuous Integration (CI) setup things get built on every push. In split repositories, that becomes a little complicated unless you build all the repositories at every build: and that gets kind of expensive, computationally.

(This is by no means a surprising or new insight: KDE’s sysadmin team has roughly this in build.kde.org, KDE’s CI where one of the challenges is building KDE Applications against recent KDE Frameworks)

Drawing of CI flow
Drawing of CI flow

What I ended up doing is making a tarball of the build of Calamares, then shipping it across to CI builds of the extensions modules. To avoid excessive builds and tarball transfers, I’ve decided to do the tarballs every night, rather than on every push.

So part of my nightly CI build on KDE neon is to make install DESTDIR=stage followed by tar czf calamares.tar.gz stage to get all the bits that Calamares would have built. This is only slightly yucky: if I was more fastidious I might use CPack to build an installable package. Regardless, on pushes to the extensions repository, I can pick up that tarball as if it was a package, and unpack it to the container that is going to do the build of the extension modules. There is also a nightly build of the extensions repository that behaves the same.

What this gets me is that extensions try to build against a recent Calamares all the time, and changes to Calamares propagate to the extensions build as well. If (when) I break something, this will show up on IRC with a meaningful message.

Artifact upload is done with actions/upload-artifact@v2, one of the “standard” GitHub actions. The standard artifact download action does not know how to fetch artifacts from another repository, so I used dawidd6/action-download-artifact@v2. That does know how, so I can move the tarball around.

All this feels way more convoluted and clunky than GitLab. I’m glad KDE uses GitLab CI for some bits (and Jenkins for the rest) since all the time it feels like I’m fighting the system. Seriously, it takes a third-party action to move artifacts across repo’s? At this point I may as well be writing shell scripts again with a one-step action.

These build artifacts are never used by any distro: they’re for Calamares project consumption. Many distro’s do their own nightly builds: neon and Manjaro, for instance, build nightly packages for testing (and I’m grateful for their reports of problems as well). Why not use those packages instead? Well, getting those packages can be a bit tricksy outside of the scope of the distro-CI, and part of what I’m testing is the bits most distro’s won’t package anyway: the Calamares “SDK” for use by the extensions repo. So the various builds are complementary and catch different things.

One thing the CI is good for is making IRC lively(-ish) with notifications, I like that too.

<cala-ci> OK ci-push-xtn in calamares/calamares-extensions adriaandegroot on refs/heads/calamares
<cala-ci> .. f2e59e6 [image-slideshow] Add an example QML slideshow for images