KDE on FreeBSD – June 2018

The KDE-FreeBSD team (a half-dozen hardy individuals, with varying backgrounds and varying degrees of involvement depending on how employment is doing) has a status message in the #kde-freebsd channel on freenode. Right now it looks like this:

http://FreeBSD.kde.org | Bleeding edge http://FreeBSD.kde.org/area51.php | Released: Qt 5.10.1, KDE SC 4.14.3, KF5 5.46.0, Applications 18.04.1, Plasma-5.12.5, Kdevelop-5.2.1, Digikam-5.9.0

It’s been a while since I wrote about KDE on FreeBSD, what with Calamares and third-party software happening as well. We’re better at keeping the IRC topic up-to-date than a lot of other sources of information (e.g. the FreeBSD quarterly reports, or the f.k.o website, which I’ll just dash off and update after writing this).

In no particular order:

  • Qt 5.10 is here, in a FrankenEngine incarnation: we still use WebEnging from Qt 5.9 because — like I’ve said before — WebEngine is such a gigantic pain in the butt to update with all the necessary patches to get it to compile.
  • Our collection of downstream patches to Qt 5.10 is growing, slowly. None of them are upstreamable (e.g. libressl support) though.
  • KDE Frameworks releases are generally pushed to ports within a week or two of release. Actually, now that there is a bigger stack of KDE software in FreeBSD ports the updates take longer because we have to do exp-runs.
  • Similarly, Applications and Plasma releases are reasonably up-to-date. We dodged a bullet by not jumping on Plasma 5.13 right away, I see. Tobias is the person doing almost all of the drudge-work of these updates, he deserves a pint of something in Vienna this summer.
  • The freebsd.kde.org website has been slightly updated; it was terribly out-of-date.

So we’re mostly-up-to-date, and mostly all packaged up and ready to go. Much of my day is spent in VMs packaged by other people, but it’s good to have a full KDE developer environment outside of them as well. (PS. Gotta hand it to Tomasz for the amazing application for downloading and displaying a flamingo .. niche usecases FTW)

CMake 3.12 Update on FreeBSD

CMake 3.12 has reached rc1. That means we’re testing the update on FreeBSD, and building lots and lots of packages. And, as I’ve written previously, every CMake update triggers a bunch of interesting software findings.

As a motto, I’ve got “use it, aggressively improve it” on my website (you can hire me for odd CMake and C++ jobs, too). So hitting compile issues makes me turn to fixing software outside of KDE.

  • Spring is a 3D RTS engine, with only a minor CMakeLists fix — CMake 3.12 is strict about file(GLOB) and the FOLLOW_SYMLINKS keyword, which is documented only for file(GLOB_RECURSE). Since CMake 3.5, probably much earlier, that keyword has been ignored, and now it’s an error (this is considered a regression).
  • Coin3D is a 3D toolkit, which in the version currently available on FreeBSD, doesn’t even use CMake. It hit a bunch of Clang6 compatibility issues, and after some investigation it turns out they had all been fixed already in later releases; I put in a little time to improve FreeBSD compatibility for the next release.

What I found interesting in those two was once again the variety in CMake styles — “Modern CMake Style” still needs to catch on in many places, and the wider ecosystem. Mantis bug-tracker! Mercurial! I remember being a big Mercurial fan years ago when doing KDE-Solaris and complaining how obtuse git is. (It’s still obtuse, but I’m used to it now).

There’s another four dozen ports that have fallout from this update; amusingly Kitware’s VTK 5 and VTK 6 are among them — although a first glance tells me that’s C++ problems and not CMake problems, actually. (Generally, using NULL in places where you want to write 0; older macro definitions of NULL would re-write to something that could successfully be cast to 0, but clang6 in C++17 mode, the default, uses nullptr which doesn’t cast).

Cracking the HDD

I’ve learned that IBM Travelstar 40GB drives use glass platters. I learned this the fun way, by bending one in a vice, with a big set of pliers. It went snap, tinkle — a different sound from other drives. And after that the bendy drive was usable as a maraca!

So why was I bending drives in the first place?

Well, I volunteer some of my time at a local second-hand place called Stichting Overal. It is an “idealistic” organisation that uses the revenue from second-hand sales to support various projects (generally small-scale development, like funding the construction of sanitation in schools). Like most second-hand stores, there’s clothes and ancient kitchen appliances and books and used bicycles .. and also an IT corner.

I help out Tom, a local school kid who has run the IT corner for some time. From time to time a PC or monitors or random IT crap is dropped off for re-use, and we apply triage. Yes, that is a lovely 486DX2, but it is still going into the bin. For various reasons, there’s a mismatch between supply and demand of hard drives: we end up with piles of small ATA-33 drives, and very few 80GB-or-more SATA drives.

Machines that show up and not immediately consigned to the bin are thoroughly cleaned (’cause, eww). Some machines are cannibalized for parts for others. Working, usable hard drives are wiped, and then re-triaged. Since we don’t want to leak whatever data is on the drives (even after wiping, and customers aren’t always all that careful about what they bring in either), leftover drives are destroyed.

So that’s why I was contorting a laptop drive. Here’s a Christmas ornament I have made out of a desktop 3.5″ drive.

Machines that get through this gauntlet are dd’ed with zeroes, then installed with some flavor of GNU/Linux. Even if there’s a valid Windows license attached to the machine, getting that installed correctly and legally is way more effort for us than doing something we know is right (and Free). Lately it’s been Fedora 27 with rpmfusion, and a KDE Plasma 5 desktop (I didn’t do the choosing, so this was a pleasant surprise). Frankly, I’m not convinced we’re doing a really good job in delivering the Linux desktop PC experience to the buyers, since there’s no Linux / Fedora documentation included (now I write that down, I realise we should probably check if there’s licensing obligations we need to follow up on). What it kinda needs is an OEM installer to do some post-sale configuration like setting up a user (I can think of at least one).

The World and Calamares

This last week, my desk mostly looked like this:

Photo of desk with atlas and monitors

WQHD and even more HD on paper.

There was a bug report for Calamares that Reykjavik was misplaced. And once I started looking at that, it was pointed out that Johannesburg was misplaced as well, and then I did some serious clicking around and found that most of the southern hemisphere was subtly off in Calamares.

Why?

The timezone-selection widget in Calamares was borrowed from some distro installer a long time ago, and at some point the images were resized, and some math corrected to map image points to latitude and longitude on the map. The resizing introduced some aliasing artifacts, and then the math moved locations far north — where a typical map projection is “stretched out” to the wrong spot. Some time ago I fixed up things above 65 degrees north or so. Reykjavik is at 64-and-a-bit north, so wasn’t handled then. And it was a bodge anyway.

Screenshot of timezone selection widget

Your Reykjavik is in another timezone

So this week I sat down with an atlas (because that provides a much nicer overview than any online map, and has latitude and longitude lines) and a list of places to test. The selection works better now, much better far south, but it’s not right yet. Basically, I’m going to have to re-draw this stuff, so I’m learning a little qgis to try to get this done (and otherwise I’ll ask a friend in Nijmegen who does GIS stuff professionally).

The atlas nearest to my desktop was Velhagen & Klasings Kleiner Handatlas, printed in Leipzig, Germany, in 1922. I didn’t feel like walking downstairs and grabbing a newer atlas. I figured the continents haven’t moved that much in the past 96 years.

What I hadn’t counted on was how much the world has changed. When I realised that, I kept the atlas open as an education.

It’s not really visible in the photograph above, but in 1922 Africa was almost entirely under the yoke of colonialism. What wasn’t “owned” by Europe, was blank. Germany had recently lost its colonies. Long-range air travel didn’t exist (although the Atlantic had been crossed by a non-stop flight by then). The number of telegraph cables across the Pacific ocean could be counted with a two-bit integer.

Photo of pages from atlas

Colonies and Steamship Routes

Remember that: the world wasn’t always as it is today, and needn’t be the way it is today, tomorrow. Strive to make it better.

Clicking around, I selected Harare, the capital of Zimbabwe. It’s been Harare since my earliest memories (insofar as I thought about cities in Africa as a child). Knowing the latitude and longitude I could look it up in my atlas (to check that its visual place on the map matches where it should be). Unfortunately, there’s only Salisbury in South Rhodesia there. Lagos, now 20 million people, is a small dot at six degrees north. All the struggles for independence are yet-to-come on this map. Almost everywhere I looked, a world that is very different.

So the thing I set out to fix in Calamares has been fixed, but the atlas is still on my desk, as a reminder to me of things that have changed, and things that haven’t.

Other People’s Work

Most of my writing on this blog is about FreeBSD, KDE, or Calamares. So it gives a bit of a one-sided view of what I do. There’s lots of pictures of rhubarb crumble, for instance, that never see the bloggy light-of-day. But I can build more than just software! Two months ago an unusually heavy storm blew down part of the fence in my back yard, which wasn’t really good for the privacy of that yard.

Photos of fence

I could pretend this fits into the KDE privacy goals, but really I just want to show off that yes, I can dig post holes, cut lumber, hammer and fasten. While doing so I also found tomatoes, cilantro and dill growing in the yard like weeds, so that’s a bonus.

Software thingies:

Since Daniel Nicoletti keeps writing about Cutelyst, I took a stab at a FreeBSD port, since web-frameworks should be plenty portable. Well, except for the logic failure that UNIX AND NOT APPLE means LINUX. After a half-hour or so of trying to get FreeBSD’s libepoll-shim to be used, I noticed that the shim API is incomplete, so I just punted LINUX out of there. After a minor code update to deal with implicit includes, I’ve got a port file for it that needs some polishing for porter’s-handbook compliance. Expect Cutelyst in the FreeBSD ports tree within a few days.

Atelier and AtCore are active, so here’s a blue-blobs picture of what’s going on (since january). Plenty of commits from the core developer, and some incidental contributions. Lays is less visible in the contribution blobs recently, but that’s probably because of fun Free Software events.

Screenshot of activity-blobs

So that’s what everyone else has done (well, some of everyone else; I’ll leave broader coverage to Nate). Next week, come back for bicycle repairs and Calamares releases.

Watching the Detectives

There are many ways to understand a community.

For instance, Kevin Ottens has been writing about understanding the KDE community by the “green blobs” method, showing who is active when. Lays Rodrigues has written about using Gource to show Plasma growing up. Nate Graham describes the goings-on in the KDE community nearly every week.

Those are, roughly: a metric-, a visual-, and a story-based approach to understanding the community, over different timescales. But understanding of a system doesn’t come from a single dimension, from a single axis of measurement. It comes from mixing up the different views to look the system as a whole.

To that end, I’m going to apply Kevin’s and Lays’s approaches to .. well, not to something Nate has written, but to a recent this-week-in-Elisa post by Matthieu Gallien. The relevant period is april 22nd (the release of Elisa 0.1.1) to may 16th (latest post on Elisa progress).

ComDaAn is the toolbox Kevin has come up with for dealing with activity and centrality. It’s a Python3 application — it almost works with Python2, except that timestamp-formatting in Python2 does not support the %z flag (in spite of the documentation). No strange dependencies, and easy_install or system packages get everything (even on FreeBSD). The green-blobs tool is called activity.py and can take a date range to limit what is shown, for instance ComDaAn/activity.py -f 2018-04-21 -u 2018-05-16 ~/src/kde/elisa/ , which gives us this (image links to an actual HTML page with all the fanciness).

Image of contributions to Elisa

Elisa Contributors, 2018-04-22 — 2018-05-16

Gource is what Lays used to show off Plasma development. I used the same video (regenerated locally) as a blinkenlights show at the KDE booth at FOSS-North. Gource also takes a date range, for instance , which gives us this (image links to the video).

Elisa Gource Screenshot

A moment in Elisa time

The time period here is short; it’s unwise to draw any conclusions from any of these visualisations. They do support the story that Matthieu tells, and the natural order of things is that the main developer does the most commits, with features and fixes coming (ir)regularly from others. The movie shows that the structure of Elisa (or the source code, at least) remains stable over this period. Together, the visualisations along different axes enliven the story of Elisa — and running those tools over a longer period of months can help understand how the community around that application grows and changes.

[[ As an aside, there’s a really neat use of Gource out there: instead of visualising source-code changes, use it to watch other kinds of events, like those that DTrace can provide from a running system. FreeBSD users can enjoy Devin Teske’s dwatch-gource, which uses dwatch to produce logs suitable for gource, and then make a movie of what their system is doing. Maybe not great for parties, but excellent for figuring out why the desktop is suddenly slow while building Qt, LLVM and GCC in parallel (all three with -j8). ]]

Calamares on Krypton

Calamares is a Linux system installer (and some day, a FreeBSD system installer, but that is a long way off) which is distro- and desktop-independent. OpenSUSE Krypton is a live CD and installer for the latest-and-greatest .. but it already has an installer, so why try Calamares on it?

Well, sometimes it’s just to show that a derivative could be made (there is one, called GeckoLinux), or to experiment with tools and configurations.

Calamares has a script called deploycala.py, which like every gaping huge security hole is expected to be downloaded from the Calamares site, then run. It is recommended to only use this in a VM, with a live CD / ISO image running. What the script does is install a basic dev environment for Calamares, install up-to-date dependencies, and then it builds and installs Calamares. That then gives you a way to experiment, installing with Calamares from an already-set-up live CD.

The deploy script supports many different package managers and host systems, so it’s just a matter of running python3 deploycala.py -n to get started (and then wait for a while as packages are installed, Calamares is cloned, and then built). Calamares builds with no issues on Krypton (at least today, when I tried it).

Screrenshot of Calamares in Krypton

Calamares in Krypton (Qt 5.11)

Screenshot of Calamares in Manjaro

Calamares in Manjaro (Qt 5.10)

Having built Calamares, there’s a few bits I notice:

  • Esperanto isn’t supported in Qt applications (neither in Krypton, nor in Manjaro, nor in anything else I tested); QLocale has a constructor that takes an enum value specifying the language, but for a bunch of languages in that enum, it then creates a “C” locale. This is documented with the wriggly description “… if found in the database …”, but is rather unsatisfying.
  • Manjaro (Qt 5.10) does a better job of displaying Indic scripts than OpenSUSE Krypton (Qt 5.11), although this might be an artifact of installing updated packages into the live system.
  • The keyboard-layout picker displays no keycaps. It also doesn’t provide any useful debugging output. This is probably a combination of missing packages in the live system, and Calamares not providing enough useful feedback when the live image isn’t quite right. The latter, I can fix.

So, by briefly switching distro’s today, I’ve found one bug of my own, and one configuration thing for myself to document. And then from this not-using-Calamares distro, I can move on to another one.

HowTo Modern (2018) KDE on FreeBSD

Quite some time ago I wrote a how-to to install modern KDE on FreeBSD. Now that the ports are all in the main tree, things are a lot simpler. So I’m writing it out anew.

  1. VM: Let’s start over with a fresh VirtualBox VM, with a 30GB zvol for storage, 4GB RAM, 4 cores of my desktop CPU.
  2. Basic Install: Get 11.x, amd64. Don’t bother with src or with a ports tree, since this is going to install from packages. I picked auto-UFS, with MBR, the most straightforward setup. IPv4, DHCP, timezone .. for system-hardening options, I added clear /tmp and disable sendmail, nothing else. That whole process took about five minutes.
  3. Initial update: Run pkg once, to install it. Optionally, run freebsd-update to update the installed system to the latest security patches. Optionally, switch to a local pkg repository. Edit: Switch to the non-quarterly package branch, first. Thanks George.
  4. Basic packages: Run pkg repeatedly:

    pkg install xorg xterm twm # basic X
    pkg install sddm kde5 kdevelop kmail # the good stuff (edit: sddm optional!)
    pkg install virtualbox-ose-additions # only if VBox

  5. Service configuration: Add these lines to /etc/rc.conf (as suggested by the pkg-messages):

    dbus_enable="YES"
    sddm_enable="YES" # edit: if you want to
    vboxguest_enable="YES" # only if VBox
    vboxservice_enable="YES" # only if VBox

  6. Sysctl configuration: Add these lines to /etc/sysctl.conf to make sure Akonadi works:

    net.local.stream.recvspace=65536
    net.local.stream.sendspace=65536

  7. Reboot and login

Screenshot

And there’s a few gotcha’s:

  • In VirtualBox, sometimes the screen may not resize when you resize the VM window. That’s not actually a KDE Plasma problem, but you can run randr --output VGA-0 --auto from a konsole to get it to resize.
  • Akonadi may not start. There’s a warning about TIMESTAMP with implicit DEFAULT, something you can find in older KDE Forum posts, too. The fix is mkdir -p ~/.local/share/akonadi/db_data ; akonadictl start .
  • You can logout, but not shutdown from Plasma itself; you can shutdown from SDDM, but don’t count on any feedback after you click on the big red (er .. monochrome) off button.

Edit: 1. The packages are only built on the regular package branch, not quarterly, so you’ll have to switch to that if you want the packages. I didn’t catch that because (right, cheating) I have my own poudriere builds from ports HEAD. 2. SDDM is optional; you can of course use any other DM (be sure to run /usr/local/bin/startkde) or use startx and an .xinitrc file (in that case, use ck-launch-session, so that you can unlock in case of general disaster. 3. Logging goes .. somewhere. Gosh, I can’t find any useful output; we’ll document that on the KDE community wiki once it’s been tracked down. In the meantime, kdebugdialog5 --fullmode can help out. 4. As for KDevelop pulling in the old desktop environment, that’s probably solved by switching away from quarterly.

Calamares and Google Summer of Code

This year Calamares is participating in Google’s Summer of Code. While Calamares doesn’t live under the KDE umbrella — for political reasons, basically, to emplasize that it is a desktop-agnostic system installer — it has a great deal of KDE DNA inside. The maintainers (that’s Teo, then myself) have been KDE people, some of the technology is definitely KDE (KPMCore in particular). So we’re happy to be participating under the KDE umbrella in a mixed KPMCore / Calamares role.

Andrius (KPMCore) and myself (Calamares) will be working with our student Caio Jordão de Lima Carvalho on a list of advanced partitioning scheme improvements. With this, we’ll make KPMCore a more powerful library for dealing with complicated partitioning schemes, and Calamares a more powerful installer. The “advanced” schemes here are all based around LVM (Linux Volume Manager) which is something I’ve been dreading having to deal with — so I’m really happy Caio will.

He’s pushed stuff to KPMCore before, and his first pull request to Calamares arrived this week, so I feel we’re off to a good start. Look for more on the Calamares site.

Welcome x11/kde5 to the FreeBSD Ports Tree

Desktop wallpaper with Konqui and FreeBSD LogoThere is no KDE5. There are KDE Frameworks 5 (releasing monthly, now reaching version 5.45) and KDE Plasma Desktop 5 (releasing quarterly, I think, now 5.12) and KDE Applications (releasing semi-anually, called 18.04).

For the FreeBSD ports tree, there is a x11/kde5. It is a metaport, which means it collects other ports together; in this case, x11/kf5-frameworks (metaport for all the frameworks), x11/plasma5-plasma-desktop and a fistful of KDE Applications metaports (e.g. the metaport for KDE games, and the metaport for KDE graphics applications, and the metaport for what-we-consider-essential KDE applications like konsole, konqueror, dolphin, and okular). So, from a bare FreeBSD installation, installing x11/xorg, x11/sddm, and x11/kde5 should get you close to a working modern KDE Desktop experience. Throw in www/falkon and devel/kdevelop for a developer workstation, or graphics/krita for an artists workstation, and you’ve got a daily driver.

I’m not really happy with naming the port this way, but it is just an identifier that needs to distinguish it from other bits and pieces on the system. And our naming was historically such a mess that it’s taken a long time to sort out; there are still some odd corners like x11/kde-baseapps.

KDE4-style exit buttonWhat this means is that KDE4 users and modern KDE Desktop users can now be separated out effectively with the KDE packages on FreeBSD. Since there’s no more upstream releases of KDE4-era code and Qt4 was end-of-lifed long ago, I can see us going the same route as Ubuntu (and Debian, and basically everyone else) soon-ish (in FreeBSD time) and handing the whole unmaintained stack to people with an active interest in maintaining it themselves, or dropping it.

Photo of FOSDEM booth with Tobias

The elusive Tobias at FOSDEM

Special and exceptional thanks needs to go to Tobias Berner for pushing the last bits to the official ports tree and for catherding this process over the course of several years (of course, you could run a modern KDE Desktop from Area51 since 2016 or so, but not from the official ports tree).

So, what’s next? Well, in no particular order:

  • Qt 5.10 in the official tree. We’ve pushed it quickly to the KDE-FreeBSD CI systems so that git master can continue to build, but it needs to go to the official ports tree too. Main issue is dealing with WebEngine (no surprise there), so we’re looking at Qt 5.10 and the unmodified WebEngine from 5.9 — a FrankenEngine, which, for all its frightening and unnatural connotations, is probably the right name for it anyway.
  • Improving overall system integration and dealing with papercuts.
  • Chasing our CMake and KDE bug lists.
  • Bringing Wayland to fruition on FreeBSD. This in cooperation with the Mesa and GNOME teams.
  • Fixing ports and things all over the tree as we bump into things (I’ve spent some time with FreeRDP recently, and should say thanks to Kyle Evans for taking my throw-them-at-the-wall patches and making them stick).

That’ll keep us busy through 2018.