Calamares GeoIP

Calamares is a distribution-independent (Linux) system installer. Outside of the “big five” distro’s, many smaller “boutique” distro’s need an installer, and Calamares is a highly configurable one that they can use. There’s a few dozen distro’s that I know of that use it (although I’ve only actually installed maybe six of them).

One optional feature — optional at the discretion of the distro which is deploying Calamares, that is — is the use of GeoIP location to find the time zone of the device being installed. This is used to automatically set the clock, for instance. If it’s not switched on, or there’s no network, Calamares defaults to a location defined by the distro — generally New York, although there’s an Austria-based distro that I think defaults to UTC.

For years, the default and suggested GeoIP provider has been freegeoip.net. That service is shutting down — well, being upgraded to a nicer API, at a different location, and with different access limitations. Older ISO images that have Calamares configured to use GeoIP with that service will cease to get usable GeoIP data on july 1st, 2018.

I don’t actually know which distro’s have GeoIP enabled, nor what provider they use.

However, the fact that this provider is shutting down prompted me to go over the existing code and make it more flexible and more general (prodding from Gabriel and Phil and Harald helped here as well). So Calamares is now ready to use more, and different, providers of GeoIP data. The providers can’t agree on using “time_zone” or “timezone” or “TimeZone” as the attribute (JSON) or element (XML) name where the data lives, so there’s a little bit more code and configuration needed. But you (as a distro) can now shop around and pick a GeoIP provider that respects your privacy.

A Day on Krypton

It’s a bird! It’s a plane! No, it’s a shiny stable-yet-bleeding-edge KDE Plasma distro!

Since Calamares has to run all over the place, and is used in derivatives of all of the “Big Five” Linux distributions, I regularly switch distro’s as a development platform. Also because I inevitably blow up the VM while running Calamares, or because an update renders the system useless. At FOSDEM I had the pleasure of chatting with the folks from the SUSE stand about OpenQA and OBS.

(Note, when I originally wrote this I was going to just fiddle around a bit and then return to my Manjaro dev VM; instead it’s turned into a week and Krypton is likely to stay lodged on my VMs and spare machines for the foreseeable future.)

Last week I spent the day with openSUSE Krypton, which is a almost-bleeding-edge KDE Plasma desktop (today’s version has Plasma 5.11.5) on top of openSUSE’s rolling-release, Tumbleweed. Most of my Linux systems (e.g. the kids gaming boxes) run openSUSE of some sort, as did all my work systems at my previous job, but I have not yet used it as a development platform for Calamares. Here’s some usage notes.

Day 1 First day with a distro is usually roughly the same: install it, copy some stuff over, install tools, checkout and build Calamares. With Krypton, it’s no different.

  1. Installation looks a little wonky here and there. The installer could use a careful go-over by a designer to smooth out lines, reduce drawing glitches, etc. It may have been an artifact of installing in an 800×600 VirtualBox window, but it didn’t seem very polished, even if the installer procedure was.
  2. Install basic development tools: zypper in git cmake make gcc gcc-c++. Huh, kdevelop is already installed, that’s a good sign (except it seems like it’s broken, and can’t find the plugin KDevWelcomePage, but see below). Shame Linux systems are otherwise so poorly prepared for being development systems.
  3. Run deploycala.py on the installed system (there’s big fat warnings saying never to do that, but I’m the developer and this is a fresh VM, so nyah nyah). Fall over backwards when it turns out that apt-get exists on this system (and invokes zypper via aptitude) so that the deploy script thinks it’s on Debian and is going to do all of the wrong things. Debug the script. Figure out dependency names (e.g. it’s gcc-c++ on openSUSE, g++ on Debian and just gcc on Arch).
  4. Find there’s no PythonQt packaged; while this is a strictly optional dependency, I would like to find a distro that actually ships something usable for PythonQt (seems Arch does, and KaOS).
  5. Build Calamares.
  6. Profit!

So where does that last, profit, step come in? Well, openSUSE has Secure Boot support, while distro’s using Calamares generally don’t — for the simple reason that Calamares doesn’t support it yet. So I’ll be peeking at what, and how, openSUSE does it and massaging that into Calamares.

Day 2 Ran an update, hoping that KDevelop would be fixed by now. That’s a nice thing about rolling- and bleeding-edge distro’s, stuff gets fixed and/or broken on a daily basis. With Krypton, the underlying rolling base is touted as stable while the KDE bits are bleeding-edge. It wasn’t, but a quick question in the right IRC channel (#opensuse-kde for Krypton) got me sorted and a fix scheduled for the next build. Well done, Kryptonites.

Spent the day hacking on Calamares, mostly fiddling with other bits-and-pieces rather than doing what I intended to do, which was examine secure boot.

Day 3 Still stable. Today’s bleeding-edge update is 112MB, as KDE Plasma is updated to 5.12. I decide to do some ARM development today as well. This is obviously not ideal, since I’m then cross-compiling to aarch64 in a Linux VM running on FreeBSD, but hey. After installing cross-aarch64-gcc7 and adjusting some build instructions that assume Debian naming (e.g. CROSS_COMPILE=aarch64-suse-linux- instead of CROSS_COMPILE=aarch64-linux-gnu-), spent a thoroughly frustrating morning building U-Boot and watching it panic. That’s the downside to using very new hardware which isn’t supported by anything yet except the OEM’s binary-blob package.

Day 4 (after the weekend) A total of 733 package updates today, 810MB to download. They’re not kidding about bleeding-edge and up-to-date. In the meantime I’ve learned that my deploycala script could be much simplified by using the package-manager. Since Calamares is packaged for openSUSE, I could have done zypper mr --enable repo-source ; zypper source-install -d calamares to get the build dependencies for it.

Anyway, after a week I’ve I have not yet broken the system, it’s fast and up-to-date. I’ll be keeping this one around. (And if I was looking for something between Krypton and Leap, I’d probably go for GeckoLinux, which uses Calamares — a bit of dogfooding, as it were).

Switching Distro’s

Obviously I still use FreeBSD on the desktop; with the packages from area51 I have a full and modern KDE Plasma environment. We (as in, the KDE-FreeBSD team) are still wrestling with getting the full Plasma 5 into the official ports tree (stalled, as so often it has been, on concerns of backwards compatibility), but things like CMake 3.10.1 and Qt 5.9 are sliding into place. Slowly, like brontosauruses driving a ’57 Cadillac.

In the meantime, I do most of my Calamares development work — it is a Linux installer, after all — in VMs with some Linux distro installed. Invariably — and especially when working on tools that do the most terrible things to the disks attached to a system — I totally break the system, the VM no longer starts at all, and my development environment is interrupted for a bit.

That’s always a good moment to switch distro’s .. since I’m going to spend an hour or so re-invigorating the VM anyway, reminding myself that this time I’ll make a clone and keep snapshots and whatnot, I may as well see what others use Calamares for. I’ve left a dusty and broken KDE Neon and a Manjaro behind me, and today I’m starting on Kaos.

For the very reason that Calamares can break stuff, and because new Calamares versions need to be tested on (older) live ISO images, there’s a deploycala Python3 script that helps install all the (development) dependencies needed. This is a bit of a drag, since it’s tied to a whole bunch of distro’s specific package names, but it gets the job done. I update it whenever I hit a new distro.

Kaos Linux Logo

Kaos Linux Logo

Kaos is a KDE-leading-edge distro, and daaannnggg is it ever slick in the first 10 minutes of use. The customization of Calamares is some of the nicest I’ve seen. The release-notes page could use some work (on my part) .. if only because I could read that stuff during unsquashfs, instead of before it. First-time startup with Kaptan is a hardly-intrusive way of configuring a bunch of separate things that would require a careful trip through systemsettings.

Look for the next few Calamares releases (in particular 3.2) to emerge from the Kaos, then.

More Calamares Releases

Another month passed, just like that. I spent last week holed up with some KDE people in the hills, watching the snow come down. While they did impressive things to the KDE codebase, I hacked on Calamares. Since my last post on the topic, I’ve been running a roughly every-other-week release schedule for the Calamares 3.1-stable branch. We’ve just reached 3.1.10. The reason for these stable releases is small bugfixes, minor polishing, and occasional non-interfering features.

Each release is announced on the Calamares site, and can be found on the Calamares GitHub page.

Calamares isn’t a KDE project, and aims to support whatever Linux distro wants to use it, and to configure the stuff that is needed for that distro. But when feature requests show up for KDE integration, there’s no special reason for me to reject them — as long as things can remain modular, the SKIP_MODULES mechanism in Calamares can avoid unwanted KDE Frameworks dependencies.

One new module in development is called “Plasma Look-and-Feel” (plasmalnf) and the purpose there is to configure the look-and-feel of Plasma. No surprise there, but ther point is that this can be done during installation, before Plasma is started by the (new) user on the target system. So kind of like the Look-and-Feel KCM, but entirely outside of the target environment. The UI is more primitive, more limited, but that’s the use-case that was asked for. I’d like to thank the Plasma developers for helping out with tooling and guiding me through some of the Plasma code, and deployers of Calamares (that is, distro’s) can look forward to this feature in the 3.2 series.

Speaking of Calamares 3.2, I’m intending to put out an RC fairly soon, with the features as developed so far. There will probably be a few RCs each of which integrates a new feature branch (e.g. a giant requirements-checking rewrite) with 3.2.0 showing up for real in the new year.

Calamares releases

It’s been a quiet month for me for blogging, but one filled with unexpected and weird and not-really-bloggable things. There was a trip to Berlin, where I had the pleasure of meeing up with a bunch of KDE people whom I hadn’t seen for over a month. Long time. There was also an accident with maple syrup, I’m sure.

Anyway, there have been four (!) Calamares releases since I last wrote about it on august 23rd (two months ago). These mix various bugfixes with various regression-fixes. This illustrates that I need a better set of acceptance tests before releasing — which take a surprisingly long time to set up, since the regressions are things like “Installing Chakra with network packages fails”, not simple stuff that is OS-, hardware- and installation-independent.

The latest release fixes that Chakra (or, rather, Calamares netinstall) problem. I’m happy to report that there’s a few new Linuxen evaluating Calamares as a system installer.

Plans for the near future include a 3.2 with some new features and much nicer support from new KPMCore releases, and ongoing care for the 3.1.x series. When and how, is in the hands of the vagaries of inspiration and long sessions with pen and paper figuring out just how things should work.

Randa Approaches

Later this week, I’m leaving for Zurich, and from there I’ll take the train up to Randa (up, in the sense that I live at sea level, and Randa is the length of one million micro-SD cards laid end-to-end higher).

In Randa, I’ll be working as a KDE developer, and as a Calamares developer, and learning about accessibility tooling. There’s about 60 hacking hours in that week. I’ll also be working as the cook, for one day. There’s about 12¬†cooking hours in a day, since feeding 30 people takes a lot of vegetable-chopping, bread-slicing, and dish-washing.

That is something special about Randa, I think: the feeling of much closer “living together”, and the way the attendees work together to create an optimal hacking environment. And cooking (along with the hacking) is my way of supporting the Randa meeting.

You can support the Randa meeting, too. That doesn’t support me; it supports other attendees who need to make long trips, it defrays the costs associated with infrastructure, it brings networking to town for a week. Support the Randa meetings for this year’s theme, or for the idea of a focussed retreat for hacking.

There’s a dot story about plans for the meeting. There will be summaries as well, and blog-roundups. But blog-roundups are tricky, because of the kind of things we (attendees) tend to write about. When blogging about the Randa meeting, I’ll probably blog more about food and hikes than about the hacking. The non-hacking bits make for better stories, even if the point of being there is the hacking. The results of the long coding sessions — privilege-separating Calamares, double-checking accessibility of KDE on FreeBSD — will show up later, in a future Calamares release or KDE-FreeBSD update. That’s the long-term payoff.

 

Calamares 3.1.3 released

Calamares is an (Linux) installer framework. It’s an application with lots of pluggable modules to do system installation tasks (like partitioning, setting up users, and enabling encryption on filesystems) needed when installing Linux onto a computer system. There are modules written in C++ (with Qt), modules in Python, and modules in Python (with Qt). The middle bunch was, up until this release, untranslatable.

Calamares 3.1.2 was released yesterday, followed by a hotfix 3.1.3 (my bad), with new potentially translatable strings. I’ll admit that the last few strings were snuck in quite late, so translations are incomplete. This is no worse than they were, where the strings were entirely untranslatable. Expect more complete translations with 3.1.4, to be relased in two weeks time or so.

This is a medium-sized step for the accessibility of Calamares, now that more of the application is available in the user’s native language.

I’m also planning a much bigger accessibility step, which is adding screen reader support to the application. Actually, “adding” is probably the wrong word: I’ll stop agressively not supporting screen readers. That hostility towards screen readers comes from the setuid nature of Calamares (it’s messing around in filesystems and partitions and whatnot) and that hostility needs to end. Adaptive technologies should be usable also when installing a computer.

With that in mind, I will be going to Randa next month for this year’s Randa meeting, which has a focus on accessibility. The dot story explains the purpose of the meeting better than I can, and one of the comments presents exactly the challenge: turn off the monitor. Now boot the computer with CD in the drive, and install Linux. If Calamares can make that possible, then we’re helping everyone get Free Software.

To help the Randa meeting help everyone, consider making a donation, which helps to keep lights on in Randa (not the heating — we bring sleeping bags because it’s cold in the mountains, even in summer).

(I should note: Calamares isn’t a KDE project, although it is used by some Linux distro’s that have a KDE focus. It is also used by some Linux distro’s that focus on a Qt-free experience after installation. Calamares is an independent project, supported by Blue Systems. And yes, I do hope to install FreeBSD with Calamares at some point.)

Calamares Testing

Photo of DesktopMy project for Blue Systems is maintaining Calamares, the distro-independent installer framework. Not surprisingly, working on it means installing lots of Linux distro’s. Here’s my physical-hardware testing setup, which is two identical older HP desktop machines and a stack of physical DVDs. Very old-school. Often I use Virtual Box, but sometimes the hum of a DVD is just what I need to calm down. There’s a KDE Neon, a Manjaro and a Netrunner DVD there, but the machine labeled Ubuntu is running Kannolo and sporting an openSUSE Geeko.

I’m all for eclecticism.

So far, I’ve found one new bug in Calamares, and fixed a handfull of them. I’m thankful to Teo, the previous Calamares maintainer, for providing helpful historical information, and to the downstream users (e.g. the distros) for being cheerful in explaining their needs.

Installing a bunch of different modern Linuxen is kind of neat; the variations in KDE Plasma Desktop configuration and branding are wild. Nearly all of them have trouble being usable on small screen sizes (e.g. the 800×600 that Virtual Box starts with — this has since been fixed). They all seem to install Virtual Box guest additions and can handle resizes immediately, so it’s not a huge issue, but just annoying. I’ve only broken one of my Linux installs so far (running an update, which then crashed kscreenlocker, and now it just comes up a black screen). I’ve got a KDE Neon dev/unstable as my main development VM set up, with KDevelop and the whole shizzle .. it’s very nice inside my KDE 4 desktop on FreeBSD.

I’ve got two favorite features, so far, in Linux live CDs and in KDE Plasma installations: ejecting the live CD on shutdown (Neon does this) and skipping the confirmation screen + 30 second timeout when clicking logout or shutdown (Netrunner does this).

So, time to hunker down with the list of issues, and in the meantime: keep on installin’.