Calamares is a modular installer for Linux distributions using Qt and KDE technologies. It is used by dozens of Linux distro’s for that crucial step of “get the ISO onto the HDD”, or some modern variant thereof. It’s modular, so distro’s can pick-and-choose what is needed: OpenRC or systemd, for instance. But it’s hard to cover everything that Linux distributions might need, so Calamares also has an “extensions” repository for more specialised modules. Let’s take a look.
The screenshot here shows the first screen of Calamares (in demonstration mode, so “real” distro’s will probably have distro-specific styling). There’s a number of user-visible “pages” to Calamares: welcome, location, etc. There are modules that are responsible for each of those; the main Calamares distribution contains a configurable welcome module, and an alternative QML-based welcomeq module as well – plenty of choice for distro’s.
There are more modules than what is visible here, though: all the invisible steps like making a user, settting up the display manager, configuring OpenRC (or systemd) .. those are modules as well. For these invisible steps there are certainly some that make sense only for specialised distro’s, so while Calamares aims to be modular and address “most” of what a Linux needs to get installed, there is always one more module for that special case (like pre-installing CDE theming, or whatever).
So in the end there are two repositories for Calamares bits:
- Calamares which is the main program, libraries, support bits, CMake infrastructure and all the modules that are generally useful (and some that aren’t, for historical reasons),
- Calamares Extensions which contains more modules, more branding examples, and generally shows off that Calamares can be extended with “third party” code as well: distro’s can use this also to build their own collection of modules (although, as always, pull requests are welcome: I’d be happy to move as much as possible upstream if it is of interest to more than one single distro).
One extra ideas behind the extensions repo is that it is a test of how-reusable-is-Calamares-code and does-my-CMake-code-actually-work. So it is supposed to (naively) build C++ modules that use the full plugin API of Calamares. But this gives rise to a slight problem: how to shuffle the latest bits around. In a Continuous Integration (CI) setup things get built on every push. In split repositories, that becomes a little complicated unless you build all the repositories at every build: and that gets kind of expensive, computationally.
(This is by no means a surprising or new insight: KDE’s sysadmin team has roughly this in build.kde.org, KDE’s CI where one of the challenges is building KDE Applications against recent KDE Frameworks)
What I ended up doing is making a tarball of the build of Calamares, then shipping it across to CI builds of the extensions modules. To avoid excessive builds and tarball transfers, I’ve decided to do the tarballs every night, rather than on every push.
So part of my nightly CI build on KDE neon is to
make install DESTDIR=stage
tar czf calamares.tar.gz stage to get all the bits that
Calamares would have built. This is only slightly yucky: if I was
more fastidious I might use CPack
to build an installable package. Regardless, on pushes to the
extensions repository, I can pick up that tarball
as if it was a package, and unpack it to the container that is
going to do the build of the extension modules. There is also a nightly
build of the extensions repository that behaves the same.
What this gets me is that extensions try to build against a recent Calamares all the time, and changes to Calamares propagate to the extensions build as well. If (when) I break something, this will show up on IRC with a meaningful message.
Artifact upload is done with
actions/upload-artifact@v2, one of
the “standard” GitHub actions. The standard artifact download action
does not know how to fetch artifacts from another repository,
so I used
dawidd6/action-download-artifact@v2. That does
know how, so I can move the tarball around.
All this feels way more convoluted and clunky than GitLab. I’m glad KDE uses GitLab CI for some bits (and Jenkins for the rest) since all the time it feels like I’m fighting the system. Seriously, it takes a third-party action to move artifacts across repo’s? At this point I may as well be writing shell scripts again with a one-step action.
These build artifacts are never used by any distro: they’re for Calamares project consumption. Many distro’s do their own nightly builds: neon and Manjaro, for instance, build nightly packages for testing (and I’m grateful for their reports of problems as well). Why not use those packages instead? Well, getting those packages can be a bit tricksy outside of the scope of the distro-CI, and part of what I’m testing is the bits most distro’s won’t package anyway: the Calamares “SDK” for use by the extensions repo. So the various builds are complementary and catch different things.
One thing the CI is good for is making IRC lively(-ish) with notifications, I like that too.
<cala-ci> OK ci-push-xtn in calamares/calamares-extensions adriaandegroot on refs/heads/calamares <cala-ci> .. f2e59e6 [image-slideshow] Add an example QML slideshow for images