Modern Akonadi and KMail on FreeBSD

For, quite literally a year or more, KMail and Akonadi on FreeBSD have been only marginally useful, at best. KDE4 era KMail was pretty darn good, but everything after that has had a number of FreeBSD users tearing out their hair. Sure, you can go to Trojitá, which has its own special problems and is generally “meh”, or bail out entirely to webmail, but .. KMail is a really great mail client when it works. Which, on Linux desktops, is nearly always, and on FreeBSD, iswas nearly never.

I looked at it with Dan and Volker last summer, briefly, and we got not much further than “hmm”. There’s a message about “The world is going to end!” which hardly makes sense, it means that a message has been truncated or corrupted while traversing a UNIX domain socket.

Now Alexandre Martins — praise be! — has wandered in with a likely solution. KDE Bug 381850 contains a suggestion, which deserves to be publicised (and tested):

sysctl net.local.stream.recvspace=65536
sysctl net.local.stream.sendspace=65536

The default FreeBSD UNIX local socket buffer space is 8kiB. Bumping the size up to 64kiB — which matches the size that Linux has by default — suddenly makes KMail and Akonadi shine again. No other changes, no recompiling, just .. bump the sysctls (perhaps also in /etc/sysctl.conf) and KMail from Area51 hums along all day without ending the world.

Since changing this value may have other effects, and Akonadi shouldn’t be dependent on a specific buffer size anyway, I’m looking into the Akonadi code (encouraged by Dan) to either automatically size the socket buffers, or to figure out where in the underlying code the assumption about buffer size lives. So for now, sysctl can make KMail users on FreeBSD happy, and later we hope to have things fully automatic (and if that doesn’t pan out, well, pkg-message exists).

PS. Modern KDE PIM applications — Akonadi, KMail — which live in the deskutils/ category of the official FreeBSD ports were added to the official tree April 10th, so you can get your fix now from the official tree.

Modern KDE Applications on FreeBSD

After the shoving is done — and it is, for the most part — it is time to fill up the void left behind by the KDE4 ports that have been shoved aside. In other words, all over the place <foo> has been shoved aside to <foo>-kde4, and now it’s time to reintroduce <foo>, but in the modern KDE Applications form. For instance, there is now a science/kalzium-kde4 (the old stuff) and a science/kalzium (the new stuff). It’s not 100% complete, but most of the applications are there.

Tobias has been the driving force behind this, with ports commits like r466877 which introduced the modern KDE applications in science/. So a huge shout-out to his work in bringing things almost-up-to-date.

Note that we now have KDE Frameworks (5.44 .. there’s an exp-run planned for this week’s 5.45 release) and KDE Applications (17.12.3) but not a Plasma 5 Desktop; so you will be running KDE Applications on the older desktop of your choice if you’re using FreeBSD ports.

The Shoving Continues

More KDE4 parts have been moved around on FreeBSD. Basically what we’re seeing is that all the existing KDE4 ports — that is, pretty much all KDE software except the KDE Frameworks 5, which are the kf5-* ports already available — are getting a -kde4 suffix. I resurrected the old filelight from KDE4 times, which we had updated to the modern KDE Applications version some time ago. That is so that KDE4 users can get the authentic (in the case of filelight-kde4, I think that also means “buggy”) experience. Users of x11/kde4 are encouraged to update and upgrade regularly these days, to catch all of the moves of packages. There are no actual updates in here, no new packages, since there aren’t any more upstream releases.

Now that we’re looking closely at the tree we find that there are some stragglers — I moved skanlite, too, for instance, which we missed in earlier shoving-rounds. Expect skanlite (un-suffixed) to be resurrected in the coming weeks, then, as the KDE Frameworks 5 based version. You can, of course, run that in a KDE4 session — or in any other desktop you like. After all, KDE software runs all over the place, it is not (generally) tied to the Plasma desktop.

So look for KDE4 to be pushed entirely to one side soon-ish, and then for something new to flow in and take its place.

CMake 3.11 (P)reparations

CMake 3.11 is here — it went through four rc’s — which means that preparatory work is underway in KDE FreeBSD land (and has been since -rc1). KDE, as the main early consumer of CMake, is the package maintainer on FreeBSD. That means that it falls to us to signal things that break due to CMake updates, and often to fix them as well. Generally the KDE ports (even the KDE4-era onces) are not a problem; modern-ish CMake was basically develop-tested in KDE. Sometimes updates in C++ bite us — recent FreeBSD releases keep updating Clang, which keeps getting more picky about C++ code (and may default to newer C++ standards than expected). But generally, KDE stuff is ok.

To test a CMake update, I build about 2000 packages on my own desktop workstation. It takes about 20 hours with all the supporting libraries and other bits — rebuilding Qt Webengine, three WebKits, five llvm’s and gcc6 kinda takes its time. Then there’s maybe two dozen packages that don’t build, and it comes down to figuring out whether they don’t build because of a change in CMake, or a change in something else, or simply because they’re already broken. But it means I end up diving into all kinds of codebases, for instance:

  • dspdfviewer had a simple C++ error, because of newer Clang being picky. (already fixed)
  • SuperTux2 has a weird #define type .. which of course breaks some C++ standard libraries where type is used as a structure field name. (already fixed)
  • sqlitebrowser has a really weird copy of QtScintilla which has both an enum and a list of #defines for the same tokens .. which breaks under some inclusion sequences. (not fixed)
  • hatari assumes that SDL_INCLUDE_DIR can only have one directory. (already fixed)
  • kicad has a typical CMake 2.6 or 2.8 setup, where some newer CMake modules from 3.0 have been copied into a cmake/ directory. This eventually breaks when the internals that the 3.0 modules rely on change. Fixing this is as easy as just deleting the old, crufty CMake bits that are shipped with the software — but it needs to be spotted before an update anyway. (already fixed)

There are not software packages I would normally involve myself with, and of these five, three have problems that are not really CMake-related.

KitWare has been great in responding to reported regressions, by the way. So those 2000 packages also serve as a testbed to spot unexpected changes in CMake’s behavior. Sometimes the best thing to do is to fix it in CMake, sometimes update the other software — or sometimes say “this has been unmaintained upstream for six years, let it go.”

There are two issues left on my list which are vexing: drawpile needs an extra include with CMake 3.11 and Wesnoth can’t find OpenMP anymore. There’s another dozen or so packages that are failing and need some work. I need to get those sorted out one way or another before the CMake 3.11.0 update hits the FreeBSD ports tree. For the over-eager, it’s already in the area51 staging tree, in branch cmake-3.11.

Outta the way, KDE4

A comic panel from Mary Worth showing the Central Park Shover

The Central Park shover gives a bad example.

KDE4 has been rudely moved aside on FreeBSD. It still installs (use x11/kde4) and should update without a problem, but this is another step towards adding modern KDE (Plasma 5 and Applications) to the official FreeBSD Ports tree.

This has taken a long time mostly for administrative reasons, getting all the bits lined up so that people sticking with KDE4 (which, right now, would be everyone using KDE from official ports and packages on FreeBSD) don’t end up with a broken desktop. We don’t want that. But now that everything Qt4 and kdelibs4-based has been moved aside by suffixing it with -kde4, we have the unsuffixed names free to indicate the latest-and-greatest from upstream.

KDE4 users will see a lot of packages moving around and being renamed, but no functional changes. Curiously, the KDE4 desktop depends on Qt5 and KDE Frameworks 5 — and it has for quite some time already, because the Oxygen icons are shared with KDE Frameworks, but primarily because FileLight was updated to the modern KDE Applications version some time ago (the KDE4 version had some serious bugs, although I can not remember what they were). Now that the names are cleaned up, we could consider giving KDE4 users the buggy version back.

From here on, we’ve got the following things lined up:

  • Qt 5.10 is being worked on, except for WebEngine (it would slow down an update way too much), because Plasma is going to want Qt 5.10 soon.
  • CMake 3.11 is in the -rc stage, so that is being lined up.
  • The kde5-import branch in KDE-FreeBSD’s copy of the FreeBSD ports tree (e.g. Area51) is being prepped and polished for a few big SVN commits that will add all the new bits.

So we’ve been saying Real Soon Now ™ for years, but things are Realer Sooner Nower ™ now.

(The image is from Mary Worth, november 19th 2013, via Mary Worth and Me; this character is known as the Central Park Shover, and he, um .. shoves people out of the way. The Shover does not manage to steal her purse.)

Plasma 5.12 on FreeBSD

“Of course it runs FreeBSD, too” is something I said a lot in the past week (regarding the Pine64, mostly, but also about my Slimbook). I also said “Of course it runs on FreeBSD, too” a lot. Naturally area51, the unofficial KDE-FreeBSD ports tree, contains the latest in released KDE software. Plasma 5.12 and KDE Frameworks 5.42, with Qt 5.9.4. We just bumped Qt to pick up a patch from KDE’s Eike Hein to fix some weird hover behavior. So we’re all up-to-date on the KDE front, and I’ve been running it as my main desktop since the build finished in poudriere.

On the official ports front, Qt 5.9.4 and KDE Frameworks 5.42 are what we’ve got. There’s a big move coming up of KDE4 ports, which is to make room (in a sense) for KDE Applications and the Plasma Desktop ports. If you’re using KDE4 from ports, then expect package bumps and renames over the weekend (no functional change, just a lot of ports will get a -kde4 name to distinguish them from the currently-maintained, up-to-date un-suffixed ports which will land afterwards.

Virtual Machines with ZFS Volumes

My main workstation runs FreeBSD; my work on Calamares is all Linux, so I use a lot of virtual machines and do a lot of disk-swapping. My tool of choice is VirtualBox, which is really darn useful for running complete desktop environments.

But I’ve just added a new item to my toolbox. It’s called vzvol.

A very recent addition to the official ports collection, vzvol is a script that helps with, and automates, creating ZFS volumes. I saw the commit adding vzvol to the ports tree, and immediately thought “hey, that’s really nifty.” And it ticks an awful lot of boxes for me in my regular workflow.

A ZFS volume is just a reservation in a zpool (which is a storage pool, composed of one or more disks; zpools have their own administration layer) which is treated as a block device by the host OS. It’s like an image file — except it’s not a file, and the blocks in the reserved space are addressable just like any other block device. For a volume called kde-neon-dev in the pool zdata, the corresponding block device is /dev/zvol/zdata/kde-neon, and you can treat it like any other disk. For instance, you can dd(1) zeroes into it, and then run fdisk(8), or gpart(8), and experiment with disk utilities.

That’s quite different from disk images as files, where you need to loopback-mount or memory-disk-wrangle them; worse still are disk images created by VirtualBox, which are hard to manipulate from the host side with regular tools.

VirtualBox can use “write-through” devices, though — it’s a bit fiddly, but you can create a VMDK that points to a block device. Any block device, including ZFS volumes! All of a sudden, the disk blocks used by my VM are also easily manipulable from the host OS. For my day-to-day disk-wrangling, this makes a huge difference.

vzvol makes manipulating and maintaining these ZFS volumes a snap, and it’s saved me tons of time in just the past two weeks. Hey, let’s try the latest Manjaro:

vzvol -s 20G -t virtualbox -p zdata -v manjaro-test

after the test, vzvol --delete cleans it up. (And vzvol runs on Linux, too, if you have ZFS there).

A Fistful of Ports Updates

Here’s a list of KDE-related stuff (mostly official FreeBSD ports) the KDE-FreeBSD team handled recently. You could call it “a week in the life of some packagers”, packagers who are also otherwise busy with $work-work.

  • Updated otter-browser (a Qt webengine-based browser) to latest version 0.9.94
  • Added a GTK QPA
  • Updated latte-dock to latest released version 0.7.3
  • Clang6 fixes to older KDE and Qt software (out favorite #define nullptr NULL)
  • Reworked packaging of Qt 5.9 to fix a broken (generated) qconfig.h

And in area51, the unofficial ports tree where we do preparatory work (use the branch kde5-import, which supercedes the plasma5 branch; this one contains the current plan for adding Plasma5 to the ports tree and updating all the KDE Applications), we’ve also got:

  • Updated digikam to 5.8
  • Updated KDE Frameworks to 5.42 (this is waiting on an exp-run to move to official ports)
  • Improved powerdevil backend, thanks to Henry Hu
  • Added plasma5-browser-integration
  • Support Qt4 on aarch64, thanks to Fedora

As usual: you can run a full modern KDE desktop system with Plasma 5 and KDE Applications, from the area51 repository, from the kde5-import branch; official ports have the latest Qt and KDE Frameworks, but not the desktop.

Qt 5.9 on FreeBSD

Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build.

The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way.

Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it’s more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speraking of ugh, I’m still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux.

For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn’t really right anymore. The kde5-import branch is where it’s at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need.

Auditing Licenses in KDE Frameworks FreeBSD Packaging

FreeBSD is getting more serious about license metadata in the packages produced by the project — that is, the binary distribution of software produced from licensed source code. A lot of software in FreeBSD “proper” is (naturally) BSD-licensed, and a lot of Free Software packaged by FreeBSD is (also naturally) GPL licensed. But the different licenses carry different obligations, so it’s good to keep track of the exact licensing applied to each bit of software.

To this end, there’s the LICENSE= line in each port’s Makefile. Its meaning is “this software has such-and-such a license”. For conciseness, SPDX identifiers are used, so that you can write

LICENSE=LGPL21

and we know what you mean. Because licenses can carry textual obligations (e.g. the GPL expects you to receive a copy, and the BSD licenses generally require you to include the copyright notice with distributions), there’s
an additional setting to include the actual text, called LICENSE_FILES:

LICENSE_FILES=COPYING.LIB

There’s a third source of license information, and that is the headers of the sources themselves. Usually you put a copyright-and-license header at the top of each file; some licenses such as the MPL even require some administration in there. The reuse.software site (by the FSFE) provides good guidance and best-practices information for providing licensing metadata in software packages.

Anyway, for KDE Frameworks 5 I was going through the ports Makefiles and adding LICENSE information. The KDE Licensing Policy tells me that frameworks should be licensed LGPL21+, with a few variants allowed; an interesting one is (at your option) “LGPL21 or LGPL3 or any later version approved by KDE e.V.” Right now, in 2017, this choice is the same as LGPL21+ because no later versions exist, but it is not always-in-the-future-same, so I hesitate to write

LICENSE=LGPL21+

for KDE Frameworks until I’ve checked the files. The license text is usually included, but it’s not quite consistently named, so I need to look into the tarballs anyway. And as a double-check, I read a couple of source headers to see if the license named in the code, matches the license text elsewhere (e.g. some files say LGPL 2.1 only).

It’s a bit of a slow process — one which upstream (that is, the source code) could support a little better with consistent naming. It’s also a process that needs to be monitored continually, to ensure that the whole body of software remains properly and consistently licensed — hopefully following best practices, too.

Anyway, as of today only ten of the KDE Frameworks 5 ports in the official FreeBSD ports repository  have all their licensing information set, to the best of my ability to check their accuracy. As an ongoing project in keeping-license-info up-to-date it’s not very high-priority but something that gets done in-between other things.