Calamares Pinebook

Photo of laptop on table

Netrunner Pinebook

This week, there was the launch of Netrunner for Pinebook (1) (2). A lot of effort has gone into building the software, improving performance, getting automation and CI in place. That’s effort that benefits the wider KDE community, the wider Free Software-on-ARM community, and more. This is the laptop I’d take with me on a bicycle camping trip — clean, cheap and affordable, although heavier than a phone.

But there is an under-appreciated bit regarding images for an ARM laptop — or pre-installed Linux distro’s in general. And that’s the first-run experience. The Netrunner Pinebook image is delivered so that it boots to the Plasma 5 desktop, no passwords asked, etc. The user is called “live”, the password is “live”, and nothing is personalized. It’s possible, though not particularly secure, to use the laptop this way in a truly disposable fashion. A first-run application helps finalize the configuration of the device by creating a named user, among other things.

One of the under-documented features of Calamares is that it can operate as a first-run application as well as a system installer. This is called “OEM Mode“, because it’s of greatest interest to OEMs .. but also to distro’s that ship an image for users to flash onto (micro)SD card for use in a device.

Screenshot of desktop

Main Desktop on Pinebook

This screenshot is from Netrunner on the Pinebook, and you can see the purple-ish Calamares logo labeled “First Time Run This Installer” on the desktop — it’s partly hidden by the system information KCM. That runs Calamares in OEM mode, hardly distinguishable from the live-ISO-installer-mode that it usually has.

A Calamares OEM configuration generally:

  • Resizes the image file to fill the entire card,
  • Creates a user (the real one, personalised) with a password,
  • Performs some initial configuration of the real user,
  • Adds packages and system configuration based on that configuration, and
  • Does some cleanup (e.g. removing Calamares, since you only need it once).

Configuring Calamares as an OEM installer is no different from any other Calamares installation: make sure the config files end up in /etc/calamares, and then set dontChroot to true (that’s the real difference between OEM mode and regular).

The way Calamares development works, downstreams — that is the distro’s, and in this case Netrunner — just file feature requests for the stuff they need in Calamares, and after some back-and-forthing to make sure it’s sufficiently general, I write it. So the run up to Netrunner on the Pinebook saw some Calamares development aimed specifically at the OEM mode, and some extra bug reports. Nothing, though, that isn’t equally applicable to any other distro that needs a first-run installer.

There is a lot more that could be done in a first-run installer. KaOS has a really nice one, basically hand-holding through initial KDE Plasma Desktop setup. Pardus and Pisi Linux have something similar. A downside is that the more you do, the more specialized it becomes — it would be nice to have a good GNOME counterpart to the Plasma Look-and-Feel selection module in Calamares, but that quickly leads to a multitude of modules and dependencies (not to mention that I can’t write it all myself).

Anyway, that’s my little square inch claim to being useful to the Netrunner Pinebook project (which deletes itself).

Calamares GeoIP

Calamares is a distribution-independent (Linux) system installer. Outside of the “big five” distro’s, many smaller “boutique” distro’s need an installer, and Calamares is a highly configurable one that they can use. There’s a few dozen distro’s that I know of that use it (although I’ve only actually installed maybe six of them).

One optional feature — optional at the discretion of the distro which is deploying Calamares, that is — is the use of GeoIP location to find the time zone of the device being installed. This is used to automatically set the clock, for instance. If it’s not switched on, or there’s no network, Calamares defaults to a location defined by the distro — generally New York, although there’s an Austria-based distro that I think defaults to UTC.

For years, the default and suggested GeoIP provider has been freegeoip.net. That service is shutting down — well, being upgraded to a nicer API, at a different location, and with different access limitations. Older ISO images that have Calamares configured to use GeoIP with that service will cease to get usable GeoIP data on july 1st, 2018.

I don’t actually know which distro’s have GeoIP enabled, nor what provider they use.

However, the fact that this provider is shutting down prompted me to go over the existing code and make it more flexible and more general (prodding from Gabriel and Phil and Harald helped here as well). So Calamares is now ready to use more, and different, providers of GeoIP data. The providers can’t agree on using “time_zone” or “timezone” or “TimeZone” as the attribute (JSON) or element (XML) name where the data lives, so there’s a little bit more code and configuration needed. But you (as a distro) can now shop around and pick a GeoIP provider that respects your privacy.

CMake 3.11 in FreeBSD

The latest release of CMake has landed in FreeBSD. Prior to release we had good contact with KitWare via the bug tracker so there were few surprises left in the actual release. There were still a few last-minute fixes left, in KDE applications no less. Here is a brief summary of changes we made:

  • FindQt doesn’t match the way FreeBSD ports are built and installed, so we defer to QMake rather than looking for directories,
  • FindOpenMP gets a tweak because it won’t find the system pthreads library when gcc (for Fortran) is used,
  • FindBLAS gets a larger tweak because BLAS may need to link to libgcc_s, and which gcc_s that is needs to be figured out via ldd(1).

Some older patches have gone away because upstream has picked them up. Tweaks downstream, in package-building-terms, of cmake that were necessary:

  • check_include_files respects the required libraries, which can be a surprise when the required libraries have been found but not fully plugged into the build (e.g. missing -L flags).
  • the order of includes in automoc sources has changed, which reveals places where a C++ header file doesn’t actually include all of the headers it needs to fully define the types it uses; previously the include order might implicitly include them and the issue is papered over.

CPack now fully supports producing FreeBSD packages from a build / install tree by default, so for non-ports software which uses CMake, cpack -G FREEBSD does the right thing. Previously, this was a non-default tweak to CPack as built in FreeBSD ports.
Edit: as of CMake 3.11.0_1, CPack no longer supports producing FreeBSD packages. There were some unexplored corners of the build process that cause build failures when the FreeBSD pkg(8) support is enabled. So it’s off again until we shine some more light into those corners.

.. and a PS., CMake 3.11.1 has just been released, which reverts the change to check_include_files which I’d been working around.

Modern Akonadi and KMail on FreeBSD

For, quite literally a year or more, KMail and Akonadi on FreeBSD have been only marginally useful, at best. KDE4 era KMail was pretty darn good, but everything after that has had a number of FreeBSD users tearing out their hair. Sure, you can go to Trojit√°, which has its own special problems and is generally “meh”, or bail out entirely to webmail, but .. KMail is a really great mail client when it works. Which, on Linux desktops, is nearly always, and on FreeBSD, iswas¬†nearly never.

I looked at it with Dan and Volker last summer, briefly, and we got not much further than “hmm”. There’s a message about “The world is going to end!” which hardly makes sense, it means that a message has been truncated or corrupted while traversing a UNIX domain socket.

Now Alexandre Martins — praise be! — has wandered in with a likely solution. KDE Bug 381850 contains a suggestion, which deserves to be publicised (and tested):

sysctl net.local.stream.recvspace=65536
sysctl net.local.stream.sendspace=65536

The default FreeBSD UNIX local socket buffer space is 8kiB. Bumping the size up to 64kiB — which matches the size that Linux has by default — suddenly makes KMail and Akonadi shine again. No other changes, no recompiling, just .. bump the sysctls (perhaps also in /etc/sysctl.conf) and KMail from Area51 hums along all day without ending the world.

Since changing this value may have other effects, and Akonadi shouldn’t be dependent on a specific buffer size anyway, I’m looking into the Akonadi code (encouraged by Dan) to either automatically size the socket buffers, or to figure out where in the underlying code the assumption about buffer size lives. So for now, sysctl can make KMail users on FreeBSD happy, and later we hope to have things fully automatic (and if that doesn’t pan out, well, pkg-message exists).

PS. Modern KDE PIM applications — Akonadi, KMail — which live in the deskutils/ category of the official FreeBSD ports were added to the official tree April 10th, so you can get your fix now from the official tree.

Modern KDE Applications on FreeBSD

After the shoving is done — and it is, for the most part — it is time to fill up the void left behind by the KDE4 ports that have been shoved aside. In other words, all over the place <foo> has been shoved aside to <foo>-kde4, and now it’s time to reintroduce <foo>, but in the modern KDE Applications form. For instance, there is now a science/kalzium-kde4 (the old stuff) and a science/kalzium (the new stuff). It’s not 100% complete, but most of the applications are there.

Tobias has been the driving force behind this, with ports commits like r466877 which introduced the modern KDE applications in science/. So a huge shout-out to his work in bringing things almost-up-to-date.

Note that we now have KDE Frameworks (5.44 .. there’s an exp-run planned for this week’s 5.45 release) and KDE Applications (17.12.3) but not a Plasma 5 Desktop; so you will be running KDE Applications on the older desktop of your choice if you’re using FreeBSD ports.

The Shoving Continues

More KDE4 parts have been moved around on FreeBSD. Basically what we’re seeing is that all the existing KDE4 ports — that is, pretty much all KDE software except the KDE Frameworks 5, which are the kf5-* ports already available — are getting a -kde4 suffix. I resurrected the old filelight from KDE4 times, which we had updated to the modern KDE Applications version some time ago. That is so that KDE4 users can get the authentic (in the case of filelight-kde4, I think that also means “buggy”) experience. Users of x11/kde4 are encouraged to update and upgrade regularly these days, to catch all of the moves of packages. There are no actual updates in here, no new packages, since there aren’t any more upstream releases.

Now that we’re looking closely at the tree we find that there are some stragglers — I moved skanlite, too, for instance, which we missed in earlier shoving-rounds. Expect skanlite (un-suffixed) to be resurrected in the coming weeks, then, as the KDE Frameworks 5 based version. You can, of course, run that in a KDE4 session — or in any other desktop you like. After all, KDE software runs all over the place, it is not (generally) tied to the Plasma desktop.

So look for KDE4 to be pushed entirely to one side soon-ish, and then for something new to flow in and take its place.

CMake 3.11 (P)reparations

CMake 3.11 is here — it went through four rc’s — which means that preparatory work is underway in KDE FreeBSD land (and has been since -rc1). KDE, as the main early consumer of CMake, is the package maintainer on FreeBSD. That means that it falls to us to signal things that break due to CMake updates, and often to fix them as well. Generally the KDE ports (even the KDE4-era onces) are not a problem; modern-ish CMake was basically develop-tested in KDE. Sometimes updates in C++ bite us — recent FreeBSD releases keep updating Clang, which keeps getting more picky about C++ code (and may default to newer C++ standards than expected). But generally, KDE stuff is ok.

To test a CMake update, I build about 2000 packages on my own desktop workstation. It takes about 20 hours with all the supporting libraries and other bits — rebuilding Qt Webengine, three WebKits, five llvm’s and gcc6 kinda takes its time. Then there’s maybe two dozen packages that don’t build, and it comes down to figuring out whether they don’t build because of a change in CMake, or a change in something else, or simply because they’re already broken. But it means I end up diving into all kinds of codebases, for instance:

  • dspdfviewer had a simple C++ error, because of newer Clang being picky. (already fixed)
  • SuperTux2 has a weird #define type .. which of course breaks some C++ standard libraries where type is used as a structure field name. (already fixed)
  • sqlitebrowser has a really weird copy of QtScintilla which has both an enum and a list of #defines for the same tokens .. which breaks under some inclusion sequences. (not fixed)
  • hatari assumes that SDL_INCLUDE_DIR can only have one directory. (already fixed)
  • kicad has a typical CMake 2.6 or 2.8 setup, where some newer CMake modules from 3.0 have been copied into a cmake/ directory. This eventually breaks when the internals that the 3.0 modules rely on change. Fixing this is as easy as just deleting the old, crufty CMake bits that are shipped with the software — but it needs to be spotted before an update anyway. (already fixed)

There are not software packages I would normally involve myself with, and of these five, three have problems that are not really CMake-related.

KitWare has been great in responding to reported regressions, by the way. So those 2000 packages also serve as a testbed to spot unexpected changes in CMake’s behavior. Sometimes the best thing to do is to fix it in CMake, sometimes update the other software — or sometimes say “this has been unmaintained upstream for six years, let it go.”

There are two issues left on my list which are vexing: drawpile needs an extra include with CMake 3.11 and Wesnoth can’t find OpenMP anymore. There’s another dozen or so packages that are failing and need some work. I need to get those sorted out one way or another before the CMake 3.11.0 update hits the FreeBSD ports tree. For the over-eager, it’s already in the area51 staging tree, in branch cmake-3.11.