CMake 3.11 (P)reparations

CMake 3.11 is here — it went through four rc’s — which means that preparatory work is underway in KDE FreeBSD land (and has been since -rc1). KDE, as the main early consumer of CMake, is the package maintainer on FreeBSD. That means that it falls to us to signal things that break due to CMake updates, and often to fix them as well. Generally the KDE ports (even the KDE4-era onces) are not a problem; modern-ish CMake was basically develop-tested in KDE. Sometimes updates in C++ bite us — recent FreeBSD releases keep updating Clang, which keeps getting more picky about C++ code (and may default to newer C++ standards than expected). But generally, KDE stuff is ok.

To test a CMake update, I build about 2000 packages on my own desktop workstation. It takes about 20 hours with all the supporting libraries and other bits — rebuilding Qt Webengine, three WebKits, five llvm’s and gcc6 kinda takes its time. Then there’s maybe two dozen packages that don’t build, and it comes down to figuring out whether they don’t build because of a change in CMake, or a change in something else, or simply because they’re already broken. But it means I end up diving into all kinds of codebases, for instance:

  • dspdfviewer had a simple C++ error, because of newer Clang being picky. (already fixed)
  • SuperTux2 has a weird #define type .. which of course breaks some C++ standard libraries where type is used as a structure field name. (already fixed)
  • sqlitebrowser has a really weird copy of QtScintilla which has both an enum and a list of #defines for the same tokens .. which breaks under some inclusion sequences. (not fixed)
  • hatari assumes that SDL_INCLUDE_DIR can only have one directory. (already fixed)
  • kicad has a typical CMake 2.6 or 2.8 setup, where some newer CMake modules from 3.0 have been copied into a cmake/ directory. This eventually breaks when the internals that the 3.0 modules rely on change. Fixing this is as easy as just deleting the old, crufty CMake bits that are shipped with the software — but it needs to be spotted before an update anyway. (already fixed)

There are not software packages I would normally involve myself with, and of these five, three have problems that are not really CMake-related.

KitWare has been great in responding to reported regressions, by the way. So those 2000 packages also serve as a testbed to spot unexpected changes in CMake’s behavior. Sometimes the best thing to do is to fix it in CMake, sometimes update the other software — or sometimes say “this has been unmaintained upstream for six years, let it go.”

There are two issues left on my list which are vexing: drawpile needs an extra include with CMake 3.11 and Wesnoth can’t find OpenMP anymore. There’s another dozen or so packages that are failing and need some work. I need to get those sorted out one way or another before the CMake 3.11.0 update hits the FreeBSD ports tree. For the over-eager, it’s already in the area51 staging tree, in branch cmake-3.11.

Outta the way, KDE4

A comic panel from Mary Worth showing the Central Park Shover

The Central Park shover gives a bad example.

KDE4 has been rudely moved aside on FreeBSD. It still installs (use x11/kde4) and should update without a problem, but this is another step towards adding modern KDE (Plasma 5 and Applications) to the official FreeBSD Ports tree.

This has taken a long time mostly for administrative reasons, getting all the bits lined up so that people sticking with KDE4 (which, right now, would be everyone using KDE from official ports and packages on FreeBSD) don’t end up with a broken desktop. We don’t want that. But now that everything Qt4 and kdelibs4-based has been moved aside by suffixing it with -kde4, we have the unsuffixed names free to indicate the latest-and-greatest from upstream.

KDE4 users will see a lot of packages moving around and being renamed, but no functional changes. Curiously, the KDE4 desktop depends on Qt5 and KDE Frameworks 5 — and it has for quite some time already, because the Oxygen icons are shared with KDE Frameworks, but primarily because FileLight was updated to the modern KDE Applications version some time ago (the KDE4 version had some serious bugs, although I can not remember what they were). Now that the names are cleaned up, we could consider giving KDE4 users the buggy version back.

From here on, we’ve got the following things lined up:

  • Qt 5.10 is being worked on, except for WebEngine (it would slow down an update way too much), because Plasma is going to want Qt 5.10 soon.
  • CMake 3.11 is in the -rc stage, so that is being lined up.
  • The kde5-import branch in KDE-FreeBSD’s copy of the FreeBSD ports tree (e.g. Area51) is being prepped and polished for a few big SVN commits that will add all the new bits.

So we’ve been saying Real Soon Now ™ for years, but things are Realer Sooner Nower ™ now.

(The image is from Mary Worth, november 19th 2013, via Mary Worth and Me; this character is known as the Central Park Shover, and he, um .. shoves people out of the way. The Shover does not manage to steal her purse.)

Plasma 5.12 on FreeBSD

“Of course it runs FreeBSD, too” is something I said a lot in the past week (regarding the Pine64, mostly, but also about my Slimbook). I also said “Of course it runs on FreeBSD, too” a lot. Naturally area51, the unofficial KDE-FreeBSD ports tree, contains the latest in released KDE software. Plasma 5.12 and KDE Frameworks 5.42, with Qt 5.9.4. We just bumped Qt to pick up a patch from KDE’s Eike Hein to fix some weird hover behavior. So we’re all up-to-date on the KDE front, and I’ve been running it as my main desktop since the build finished in poudriere.

On the official ports front, Qt 5.9.4 and KDE Frameworks 5.42 are what we’ve got. There’s a big move coming up of KDE4 ports, which is to make room (in a sense) for KDE Applications and the Plasma Desktop ports. If you’re using KDE4 from ports, then expect package bumps and renames over the weekend (no functional change, just a lot of ports will get a -kde4 name to distinguish them from the currently-maintained, up-to-date un-suffixed ports which will land afterwards.

Virtual Machines with ZFS Volumes

My main workstation runs FreeBSD; my work on Calamares is all Linux, so I use a lot of virtual machines and do a lot of disk-swapping. My tool of choice is VirtualBox, which is really darn useful for running complete desktop environments.

But I’ve just added a new item to my toolbox. It’s called vzvol.

A very recent addition to the official ports collection, vzvol is a script that helps with, and automates, creating ZFS volumes. I saw the commit adding vzvol to the ports tree, and immediately thought “hey, that’s really nifty.” And it ticks an awful lot of boxes for me in my regular workflow.

A ZFS volume is just a reservation in a zpool (which is a storage pool, composed of one or more disks; zpools have their own administration layer) which is treated as a block device by the host OS. It’s like an image file — except it’s not a file, and the blocks in the reserved space are addressable just like any other block device. For a volume called kde-neon-dev in the pool zdata, the corresponding block device is /dev/zvol/zdata/kde-neon, and you can treat it like any other disk. For instance, you can dd(1) zeroes into it, and then run fdisk(8), or gpart(8), and experiment with disk utilities.

That’s quite different from disk images as files, where you need to loopback-mount or memory-disk-wrangle them; worse still are disk images created by VirtualBox, which are hard to manipulate from the host side with regular tools.

VirtualBox can use “write-through” devices, though — it’s a bit fiddly, but you can create a VMDK that points to a block device. Any block device, including ZFS volumes! All of a sudden, the disk blocks used by my VM are also easily manipulable from the host OS. For my day-to-day disk-wrangling, this makes a huge difference.

vzvol makes manipulating and maintaining these ZFS volumes a snap, and it’s saved me tons of time in just the past two weeks. Hey, let’s try the latest Manjaro:

vzvol -s 20G -t virtualbox -p zdata -v manjaro-test

after the test, vzvol --delete cleans it up. (And vzvol runs on Linux, too, if you have ZFS there).

A Fistful of Ports Updates

Here’s a list of KDE-related stuff (mostly official FreeBSD ports) the KDE-FreeBSD team handled recently. You could call it “a week in the life of some packagers”, packagers who are also otherwise busy with $work-work.

  • Updated otter-browser (a Qt webengine-based browser) to latest version 0.9.94
  • Added a GTK QPA
  • Updated latte-dock to latest released version 0.7.3
  • Clang6 fixes to older KDE and Qt software (out favorite #define nullptr NULL)
  • Reworked packaging of Qt 5.9 to fix a broken (generated) qconfig.h

And in area51, the unofficial ports tree where we do preparatory work (use the branch kde5-import, which supercedes the plasma5 branch; this one contains the current plan for adding Plasma5 to the ports tree and updating all the KDE Applications), we’ve also got:

  • Updated digikam to 5.8
  • Updated KDE Frameworks to 5.42 (this is waiting on an exp-run to move to official ports)
  • Improved powerdevil backend, thanks to Henry Hu
  • Added plasma5-browser-integration
  • Support Qt4 on aarch64, thanks to Fedora

As usual: you can run a full modern KDE desktop system with Plasma 5 and KDE Applications, from the area51 repository, from the kde5-import branch; official ports have the latest Qt and KDE Frameworks, but not the desktop.

Qt 5.9 on FreeBSD

Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build.

The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way.

Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it’s more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speraking of ugh, I’m still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux.

For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn’t really right anymore. The kde5-import branch is where it’s at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need.

Auditing Licenses in KDE Frameworks FreeBSD Packaging

FreeBSD is getting more serious about license metadata in the packages produced by the project — that is, the binary distribution of software produced from licensed source code. A lot of software in FreeBSD “proper” is (naturally) BSD-licensed, and a lot of Free Software packaged by FreeBSD is (also naturally) GPL licensed. But the different licenses carry different obligations, so it’s good to keep track of the exact licensing applied to each bit of software.

To this end, there’s the LICENSE= line in each port’s Makefile. Its meaning is “this software has such-and-such a license”. For conciseness, SPDX identifiers are used, so that you can write

LICENSE=LGPL21

and we know what you mean. Because licenses can carry textual obligations (e.g. the GPL expects you to receive a copy, and the BSD licenses generally require you to include the copyright notice with distributions), there’s
an additional setting to include the actual text, called LICENSE_FILES:

LICENSE_FILES=COPYING.LIB

There’s a third source of license information, and that is the headers of the sources themselves. Usually you put a copyright-and-license header at the top of each file; some licenses such as the MPL even require some administration in there. The reuse.software site (by the FSFE) provides good guidance and best-practices information for providing licensing metadata in software packages.

Anyway, for KDE Frameworks 5 I was going through the ports Makefiles and adding LICENSE information. The KDE Licensing Policy tells me that frameworks should be licensed LGPL21+, with a few variants allowed; an interesting one is (at your option) “LGPL21 or LGPL3 or any later version approved by KDE e.V.” Right now, in 2017, this choice is the same as LGPL21+ because no later versions exist, but it is not always-in-the-future-same, so I hesitate to write

LICENSE=LGPL21+

for KDE Frameworks until I’ve checked the files. The license text is usually included, but it’s not quite consistently named, so I need to look into the tarballs anyway. And as a double-check, I read a couple of source headers to see if the license named in the code, matches the license text elsewhere (e.g. some files say LGPL 2.1 only).

It’s a bit of a slow process — one which upstream (that is, the source code) could support a little better with consistent naming. It’s also a process that needs to be monitored continually, to ensure that the whole body of software remains properly and consistently licensed — hopefully following best practices, too.

Anyway, as of today only ten of the KDE Frameworks 5 ports in the official FreeBSD ports repository  have all their licensing information set, to the best of my ability to check their accuracy. As an ongoing project in keeping-license-info up-to-date it’s not very high-priority but something that gets done in-between other things.

Sayonara

Not goodbye, but hello to a new music player on my desktop. It is already packaged for FreeBSD and has a long development history.

Music players stand or fall per individual user whether they satisfy the user’s needs — no duh there, but it means I should note what my use cases are before enthusing about some music player in particular. I would be fine with playing music from the command-line, most of the time: mpg123 --shuffle /mnt/music/*/*/* is just about right (except it fails with an argument list too long error in the shell). This is for music-while-I-hack, so I don’t listen too closely, it’s not hi-fi at all, basically I want “play in a genre until I switch it off“. Tagging is largely done when ripping my CDs (I still buy physical media!) and I don’t care for album art (I can look in the jewel case if I want that, and they’re all stacked in boxes upstairs). So play, pause, stop .. and if it can avoid mixing Mahler with Morrissey and Mötorhead, that’s a bonus.

Screenshot of Sayonara main windowOff to the left here is Sayonara, which describes itself as:

Sayonara is a small, clear and fast audio player for Linux written in C++, supported by the Qt framework. It uses GStreamer as audio backend. Sayonara is open source and uses the GPLv3 license. One of Sayonara’s goals is intuitive and easy usablility. Currently, it is only available for Linux.

Clearly that’s not true, because it’s available for FreeBSD as well 🙂 . I forget how I configured it; there’s a File > Open Directory entry and I probably filled in /mnt/music and it went and found everything that gstreamer will sensibly play. The screenshot is of the minimal playlist-only view. It can be broadened to include a library view which is a bit chaotic, but which searches reasonably well (e.g. for Mudhoney). Sayonara is actually where I discovered the luxury of stay-on-genre autoplay: when I started on some Dutch punk and it kept pretty close to that, wandering off to de Kift occasionally — but no Drs.P.

I contributed a few small fixes to get Sayonara building on FreeBSD, cleaning up the CMakeLists for newer CMake, that kind of thing. Now that it runs on my Plasma desktop all day, I can spot some more  minor issues — the system tray media player control doesn’t interact with Sayonara at all, for instance. For my use-case, that’s not so important, since the fire-and-forget genre play does it for me until I hit pause in the main UI.

I’m looking forward to KDE music players Elisa and Babe, for comparison purposes: maybe they tick my requirements-boxes just as well, or better. Certainly Elisa seems to be fairly playing-music-focused. I’ve even got a FreeBSD port for Elisa ready, just waiting for Qt 5.9 to show up on my doorstep (I can get it to compile against 5.7, but it won’t run due to QML runtime thingies).

CMake 3.10 on FreeBSD

The CMake port on FreeBSD is in the hands of the KDE-FreeBSD folks (since KDE was an early adopter), and generally it falls to me to do the CMake update while Tobias is still wrestling with Plasma and Raphael massages Qt 5.9 into our way of building. 3.10 was released five weeks ago, and it took a while to update the port.

It doesn’t take long because of CMake — they’re great, the code builds flawlessly, and there has been a real effort on the part of the KitWare folks recently to absorb our downstream patches so that we have less work in future (During the delay packaging CMake 3.10.0 Kitware even put out 3.10.1, which re-started some processes). It doesn’t take long because of new FreeBSD-specific features in CMake — you can now use CPack to create native FreeBSD packages, just in case you don’t want to go through the ports system or poudriere.

Nope, the problem is the 2000-odd ports using CMake. These generally behave, but every CMake update brings with it some ports that suddenly don’t build. So I spent a day on things like openvsp and scalapack, which fall over with 3.10 (and didn’t with 3.9). The causes of these failures is diverse; sometimes bad local CMake modules that don’t add all the right linking flags, sometimes bad C++ code, sometime stuff that has nothing to do with CMake at all but just happens to be newly triggered, and therefore my problem.

CMake 3.10.1 landed in the official ports tree with r457041, just before Christmas.

Naturally, there are some cases that fall out after an update, that are not caught before the update is committed. I’m told that FreeBSD on PowerPC doesn’t have a C++11-capable compiler installed by default, so I’ll need to massage that a little. Boost 1.66 came out very shortly after CMake 3.10.1, and that leads to new kinds of breakage. I’ve spent a half day compiling and re-compiling ceph, a distributed filesystem, trying to get that to work. It’s amazing sometimes how shifting one brick can bring down a whole house, and how we ever built the house in the first place.

Switched to Plasma 5

I’ve switched to using Plasma 5 as my daily (work) desktop. “So what?” you say, since Plasma 5 has been available since July 15th, 2014. Yep, more than three years on the desktop — see Sebas’s blog for some history. So I’ve finally switched my FreeBSD desktop machine, as a sign that Plasma 5 really is coming closer to the official FreeBSD ports tree.

KDE Applications are starting to trickle in to the official ports tree. The first (I think) was FileLight, which gave a good illustration of the gotcha’s involved with the update:

  • Anything that touches baloo requires a radical update to Plasma 5 as a whole. Baloo 4 and baloo 5 are not happy side-by side, and we don’t intend to put serious effort into installing them in separate places.
  • Translations are tricky, because they have moved from (KDE 4 times) a separate i18n package, to the application package. This means that the new KF5-based application package conflicts with the old KDE4 application, but also with the entire translation package (for, say, Russian) for KDE4.

For people who want the whole thing now, and who understand that KDE4 co-existence with Plasma 5 and modern KDE Applications can be problematic, the official ports tree can be avoided, and Area51 is the repository or ports tree to use.

On my desktop, on 10.3-STABLE and with nVidia drivers, Plasma 5 is more stable than on 12-CURRENT with tricksy Intel IGP drivers on the laptop. That’s a good thing, for the desktop I spend all week in writing code.

KMail and Akonadi have decided that my local maildirs are not accessible, so I’m using mutt for the time being to look at those. All my IMAP (knock on wood) is fine, so there’s still an intriguing bug there. In Randa I showed the effect and the messages to Dan and Volker, who were likewise intrigued. In the
end, though, it needs to be debugged and fixed here, locally, and then upstreamed.

At some point, the KDE-FreeBSD team had in mind to have Plasma 5 in the official ports tree in 2017Q2. It’s looking like 2017Q4, now (with just 5 days to go in Q3), basically because the upgrade path is annoying (from a ports perspective). Not for any functional reason, which is why it’s worth repeating that Plasma 5 from Area51 is a fine, fully functional, and fully up-to-date KDE desktop.

[[ Edit 2017-09-27 14:18 because Phoronix manages to get everything wrong, all the time: Tobias has been using Plasma 5 as his FreeBSD desktop of choice for years, dogfooding the ports much better than I ever have. And while I might be the noisiest blogger of our lot, that’s .. not necessarily a position of leadership. ]]