Those top Konsole Contributors

Whenever I see a post about community growth or participation — like Tomasz Canabrava’s most recent “From Nothing to top 20 contributors of Konsole in Less Than a Month” — I reach for the toolkit written by Kevin Ottens, because that makes it easy to obtain good numbers and graphs for community measurement.

Graph of Konsole's Entire History, all scrunched up and made tinyLooking at the graph (click to enlarge, but it’s still hardly readable — I’d suggest using Kevin’s tooling directly if you want to zoom in or experiment) of Konsole development since the very beginning shows a few striking facts:

  • Konsole has been in continuous development since 1998
  • Except for a gap 2006-2009, Kurt Hindenburg has been active in Konsole since roughly 2004
  • By looking at the density of commits and the length of commit-streaks, you can guess at maintainers and co-maintainers of Konsole, alongside all the occasional committers. There’s maybe five people whose histories suggest that.

Of personal interest to me is that in 2002 I contributed to Konsole for FreeBSD compatibility, and in 2009 for OpenSolaris compatibility, but nothing else in the history of the project. And Konsole spent a year licensed as Artistic, rather than the GPL2-or-later, in 1999.

Graph of Konsole's Recent HistoryRecently, you can definitely see that Tomaz has been really active, and Kurt has slightly more quiet time. But Ahmad and Mariusz are also consistently putting in work, albeit with a lower rate of commits. Judging by the commit messages, those have gone through Phabricator. That means that their commit counts are reduced because arc squashes commits (just like Tomaz, and as pointed out by Eike elsewhere). Keep in mind that commit-counts are poor proxies for contributions — that’s something we’ve been saying for years and years.

Also of interest in the history: there are 1090 commits by people not in the top-twenty; if those were grouped together they would hold second place!

So, thanks Tomaz for being noisy about what you’re doing. I think we need more of the “hey, I can improve the world” kind of noise — like KDE’s GSoC stories as well. And also, thanks Kurt and KDE community for being consistent and productive in small amounts for many years.

Trying a new Calamares workflow

I just released Calamares 3.2.1, a new-features release, and I’m trying something new for the next release. This is vaguely inspired by Paul Adams’s talk about “The Art of the Pull Request”. Immediately after the release, I’ve added several branches, one for each set of features that I want to implement for the next release. The graph displayed by QGit looks more like an octopus than ever:

Master has three immediate branches

Each branch starts with an empty commit that just says what I indend to accomplish in the branch. That way, intentions are documented inside the code repository — portably (that is, not tied up with a Pull Request or other coding-platform feature). I can of course delete the branches later if something happens to make the branches obsolete or useless. But from here, there’s a pretty clean picture of how the next release (3.2.2) is going to work:

  • Bugfixes go to master,
  • Planned feature commits go to the relevant branch,
  • Unplanned features go to new topic branches off of master.

When a planned feature is done, I will merge that branch back to master, delete the branch, and call it a day. When all of them have been merged that way, the planned features for the release are done — and it should be ready to release.

Slightly sneaky, I can even claim “FIXES #” in the initial empty commit in the branch, and assuming I’m honest with myself about the intentions of, and effects of, the branch, when I merge to master the corresponding issues in the issue tracker will be closed automatically.

Mostly I’m curious whether working this way will help me keep the total cycle time low for the next release, while also informing others where works is “needed right now”. And if it works, then in future the graph of a Calamares release will look more like an octopus, tying its feet together. Seems like a good thing for an installer framework named after squid.

(wanted) Poudriere Workflow Support

One of the premiere tools in FreeBSD CI work is poudriere. It’s a collection of shell scripts that leverage FreeBSD jails (chroot on steroids; build-in containers) and ZFS to build ports in a clean environment. You can also “cross” build for different versions of FreeBSD (e.g. on my 11-STABLE box, I can also build 10-STABLE and 12-CURRENT, although forward-compatibility to 12- can be tricky because of kernel changes). It will even truly cross build, but that’s beastly slow due to QEMU (e.g. my Skylake box can just about keep up with my Pine64). And it supports multiple ports trees, so you can do builds for the official ports tree and for local experiments too.

A typical invocation of poudriere looks like this:

poudriere bulk -j 111amd64 -p github-kde -t -i -C math/freemat

That means “build packages in the 111amd64 jail, from the ports tree called github-kde; test each port, afterwards give an interactive shell, remove and rebuild a clean version of math/freemat“. The -C is generally only used when building something multiple times in the same environment as you refine a port. There’s also a -c “rebuild the whole world” (also known as “damn, I may as well go to bed then”) flag.

Poudriere will grind away at dependencies and everything, and in the end spits out a nicely colored status line; it looks like this (here, I was rebuilding octave in order to test Qt5 compatibility, and most of the dependencies were already done).

Screenshot of Poudriere's console outputDuring the build, poudriere can run hooks in response to various events. Those events include build success, or failure (in various flavors). Using the hooks, it’s easy to move the errors to a separate directory, to end up with the build logs of those things that actually failed. Tobias has done that, and we end up with a directory listing like this:

We call a listing like this fallout. Ports that fail during a poudriere run. Since the poudriere run is often a test-run for the upgrade of an important package (e.g. upgrading CMake is tested by building 2500+ packages), handling the fallout afterwards is important: we need to go through each failed port and figure out why it has failed.

In the screenshot above, Coin was a C++-compatibility issue; so were freeMat and ampas. Apviv is hilariously silly bad C++ code (from 2014, so only in today’s context is it bad). .. and so on, and so on. Most recently, I was working from the top of the list, Tobias from the bottom, fixing actual CMake issues and optionally fixing non-CMake issues (e.g. all the dodgy C++ code). And then it struck me, we need better tool support for our very simple workflow.

Wanted!

What we need is a way to associate two pieces of data with each entry in that directory.

  • Who has “claimed” the entry (file) to look at. This is just to prevent double efforts. It should also be possible to “unclaim” an entry.
  • Tags on an entry, explaining what the problem is (e.g. “derp++”, “CMake”, “Upstream gone”).

Screenshot showing possible design for workflow tool.

Kolourpaint don’t fail me now! Three failed ports, with an image indicating who has claimed them and some tags.


In a way, it’s like an issues tracker, only slightly more free-form. There doesn’t have to be any kind of persistence: the workflow applies to one run of poudriere and the next one is basically independent. It might be nice to have user-selectable nicknames, but the owner could be indicated as an IP address, or a hash, or a color .. perhaps the set of possible tags should persist from one incarnation to another.

Does anyone know of an existing tool that does this?

If all else fails, I may sit down with Cutelyst and see what I can do there (or in Pyramid, or whatever; most of the work is probably in the CSS and Javascript, with only a very small core to serve up the page and handle the AJAX requests).

KDE on FreeBSD – June 2018

The KDE-FreeBSD team (a half-dozen hardy individuals, with varying backgrounds and varying degrees of involvement depending on how employment is doing) has a status message in the #kde-freebsd channel on freenode. Right now it looks like this:

http://FreeBSD.kde.org | Bleeding edge http://FreeBSD.kde.org/area51.php | Released: Qt 5.10.1, KDE SC 4.14.3, KF5 5.46.0, Applications 18.04.1, Plasma-5.12.5, Kdevelop-5.2.1, Digikam-5.9.0

It’s been a while since I wrote about KDE on FreeBSD, what with Calamares and third-party software happening as well. We’re better at keeping the IRC topic up-to-date than a lot of other sources of information (e.g. the FreeBSD quarterly reports, or the f.k.o website, which I’ll just dash off and update after writing this).

In no particular order:

  • Qt 5.10 is here, in a FrankenEngine incarnation: we still use WebEnging from Qt 5.9 because — like I’ve said before — WebEngine is such a gigantic pain in the butt to update with all the necessary patches to get it to compile.
  • Our collection of downstream patches to Qt 5.10 is growing, slowly. None of them are upstreamable (e.g. libressl support) though.
  • KDE Frameworks releases are generally pushed to ports within a week or two of release. Actually, now that there is a bigger stack of KDE software in FreeBSD ports the updates take longer because we have to do exp-runs.
  • Similarly, Applications and Plasma releases are reasonably up-to-date. We dodged a bullet by not jumping on Plasma 5.13 right away, I see. Tobias is the person doing almost all of the drudge-work of these updates, he deserves a pint of something in Vienna this summer.
  • The freebsd.kde.org website has been slightly updated; it was terribly out-of-date.

So we’re mostly-up-to-date, and mostly all packaged up and ready to go. Much of my day is spent in VMs packaged by other people, but it’s good to have a full KDE developer environment outside of them as well. (PS. Gotta hand it to Tomasz for the amazing application for downloading and displaying a flamingo .. niche usecases FTW)

CMake 3.12 Update on FreeBSD

CMake 3.12 has reached rc1. That means we’re testing the update on FreeBSD, and building lots and lots of packages. And, as I’ve written previously, every CMake update triggers a bunch of interesting software findings.

As a motto, I’ve got “use it, aggressively improve it” on my website (you can hire me for odd CMake and C++ jobs, too). So hitting compile issues makes me turn to fixing software outside of KDE.

  • Spring is a 3D RTS engine, with only a minor CMakeLists fix — CMake 3.12 is strict about file(GLOB) and the FOLLOW_SYMLINKS keyword, which is documented only for file(GLOB_RECURSE). Since CMake 3.5, probably much earlier, that keyword has been ignored, and now it’s an error (this is considered a regression).
  • Coin3D is a 3D toolkit, which in the version currently available on FreeBSD, doesn’t even use CMake. It hit a bunch of Clang6 compatibility issues, and after some investigation it turns out they had all been fixed already in later releases; I put in a little time to improve FreeBSD compatibility for the next release.

What I found interesting in those two was once again the variety in CMake styles — “Modern CMake Style” still needs to catch on in many places, and the wider ecosystem. Mantis bug-tracker! Mercurial! I remember being a big Mercurial fan years ago when doing KDE-Solaris and complaining how obtuse git is. (It’s still obtuse, but I’m used to it now).

There’s another four dozen ports that have fallout from this update; amusingly Kitware’s VTK 5 and VTK 6 are among them — although a first glance tells me that’s C++ problems and not CMake problems, actually. (Generally, using NULL in places where you want to write 0; older macro definitions of NULL would re-write to something that could successfully be cast to 0, but clang6 in C++17 mode, the default, uses nullptr which doesn’t cast).

Cracking the HDD

I’ve learned that IBM Travelstar 40GB drives use glass platters. I learned this the fun way, by bending one in a vice, with a big set of pliers. It went snap, tinkle — a different sound from other drives. And after that the bendy drive was usable as a maraca!

So why was I bending drives in the first place?

Well, I volunteer some of my time at a local second-hand place called Stichting Overal. It is an “idealistic” organisation that uses the revenue from second-hand sales to support various projects (generally small-scale development, like funding the construction of sanitation in schools). Like most second-hand stores, there’s clothes and ancient kitchen appliances and books and used bicycles .. and also an IT corner.

I help out Tom, a local school kid who has run the IT corner for some time. From time to time a PC or monitors or random IT crap is dropped off for re-use, and we apply triage. Yes, that is a lovely 486DX2, but it is still going into the bin. For various reasons, there’s a mismatch between supply and demand of hard drives: we end up with piles of small ATA-33 drives, and very few 80GB-or-more SATA drives.

Machines that show up and not immediately consigned to the bin are thoroughly cleaned (’cause, eww). Some machines are cannibalized for parts for others. Working, usable hard drives are wiped, and then re-triaged. Since we don’t want to leak whatever data is on the drives (even after wiping, and customers aren’t always all that careful about what they bring in either), leftover drives are destroyed.

So that’s why I was contorting a laptop drive. Here’s a Christmas ornament I have made out of a desktop 3.5″ drive.

Machines that get through this gauntlet are dd’ed with zeroes, then installed with some flavor of GNU/Linux. Even if there’s a valid Windows license attached to the machine, getting that installed correctly and legally is way more effort for us than doing something we know is right (and Free). Lately it’s been Fedora 27 with rpmfusion, and a KDE Plasma 5 desktop (I didn’t do the choosing, so this was a pleasant surprise). Frankly, I’m not convinced we’re doing a really good job in delivering the Linux desktop PC experience to the buyers, since there’s no Linux / Fedora documentation included (now I write that down, I realise we should probably check if there’s licensing obligations we need to follow up on). What it kinda needs is an OEM installer to do some post-sale configuration like setting up a user (I can think of at least one).

The World and Calamares

This last week, my desk mostly looked like this:

Photo of desk with atlas and monitors

WQHD and even more HD on paper.

There was a bug report for Calamares that Reykjavik was misplaced. And once I started looking at that, it was pointed out that Johannesburg was misplaced as well, and then I did some serious clicking around and found that most of the southern hemisphere was subtly off in Calamares.

Why?

The timezone-selection widget in Calamares was borrowed from some distro installer a long time ago, and at some point the images were resized, and some math corrected to map image points to latitude and longitude on the map. The resizing introduced some aliasing artifacts, and then the math moved locations far north — where a typical map projection is “stretched out” to the wrong spot. Some time ago I fixed up things above 65 degrees north or so. Reykjavik is at 64-and-a-bit north, so wasn’t handled then. And it was a bodge anyway.

Screenshot of timezone selection widget

Your Reykjavik is in another timezone

So this week I sat down with an atlas (because that provides a much nicer overview than any online map, and has latitude and longitude lines) and a list of places to test. The selection works better now, much better far south, but it’s not right yet. Basically, I’m going to have to re-draw this stuff, so I’m learning a little qgis to try to get this done (and otherwise I’ll ask a friend in Nijmegen who does GIS stuff professionally).

The atlas nearest to my desktop was Velhagen & Klasings Kleiner Handatlas, printed in Leipzig, Germany, in 1922. I didn’t feel like walking downstairs and grabbing a newer atlas. I figured the continents haven’t moved that much in the past 96 years.

What I hadn’t counted on was how much the world has changed. When I realised that, I kept the atlas open as an education.

It’s not really visible in the photograph above, but in 1922 Africa was almost entirely under the yoke of colonialism. What wasn’t “owned” by Europe, was blank. Germany had recently lost its colonies. Long-range air travel didn’t exist (although the Atlantic had been crossed by a non-stop flight by then). The number of telegraph cables across the Pacific ocean could be counted with a two-bit integer.

Photo of pages from atlas

Colonies and Steamship Routes

Remember that: the world wasn’t always as it is today, and needn’t be the way it is today, tomorrow. Strive to make it better.

Clicking around, I selected Harare, the capital of Zimbabwe. It’s been Harare since my earliest memories (insofar as I thought about cities in Africa as a child). Knowing the latitude and longitude I could look it up in my atlas (to check that its visual place on the map matches where it should be). Unfortunately, there’s only Salisbury in South Rhodesia there. Lagos, now 20 million people, is a small dot at six degrees north. All the struggles for independence are yet-to-come on this map. Almost everywhere I looked, a world that is very different.

So the thing I set out to fix in Calamares has been fixed, but the atlas is still on my desk, as a reminder to me of things that have changed, and things that haven’t.