Debugging Tools

Today I had to repair my most important debugging tool. Here’s the result:

That’s three strands (red, white, black) from a USB-to-serial converter, soldered on to a 3-pole screw-tightened connector. Clamped into that are the serial lines (red, green and blue) which were originally crimped straight to the lines. After a few months of use, the crimping failed and the red cable (RX) broke off.

So I had to fix it, and in the process decided to make it more sturdy, more ugly, but also easier to use.

The three grey wires clamped into the connector are part of a 10-pin flat cable which I scavenged out of a 9-pin serial connector I had left in the box-of-old-parts.

The flatcable, and especially the 10-pin connector at the end, is nice and sturdy for connecting to the headers on single-board-computers, like here:

That’s why I need a serial cable for debugging. The frontmost board is a Pine64+. It runs FreeBSD and Linux; in Linux it will drive a full Plasma 5 desktop at FHD and acceptable responsiveness; FreeBSD is limited to serial console and ssh access. I use this board to test aarch64 support in FreeBSD and to test our KDE-FreeBSD packages on yet-more-unusual platforms. This turns up occasional issues in KDE code.

(The other two boards are a switch I had left over, and there’s an Odroid C2 at the back; together they form my little ARM rack, along with two planks they’re screwed on to and a power supply. Did you know the Palm m500 used the same size barrel jack for power as the Odroid C2? Yet another reason to keep old parts lying around.)

A Fistful of Ports Updates

Here’s a list of KDE-related stuff (mostly official FreeBSD ports) the KDE-FreeBSD team handled recently. You could call it “a week in the life of some packagers”, packagers who are also otherwise busy with $work-work.

  • Updated otter-browser (a Qt webengine-based browser) to latest version 0.9.94
  • Added a GTK QPA
  • Updated latte-dock to latest released version 0.7.3
  • Clang6 fixes to older KDE and Qt software (out favorite #define nullptr NULL)
  • Reworked packaging of Qt 5.9 to fix a broken (generated) qconfig.h

And in area51, the unofficial ports tree where we do preparatory work (use the branch kde5-import, which supercedes the plasma5 branch; this one contains the current plan for adding Plasma5 to the ports tree and updating all the KDE Applications), we’ve also got:

  • Updated digikam to 5.8
  • Updated KDE Frameworks to 5.42 (this is waiting on an exp-run to move to official ports)
  • Improved powerdevil backend, thanks to Henry Hu
  • Added plasma5-browser-integration
  • Support Qt4 on aarch64, thanks to Fedora

As usual: you can run a full modern KDE desktop system with Plasma 5 and KDE Applications, from the area51 repository, from the kde5-import branch; official ports have the latest Qt and KDE Frameworks, but not the desktop.

Qt 5.9 on FreeBSD

Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build.

The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way.

Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it’s more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speraking of ugh, I’m still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux.

For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn’t really right anymore. The kde5-import branch is where it’s at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need.

Auditing Licenses in KDE Frameworks FreeBSD Packaging

FreeBSD is getting more serious about license metadata in the packages produced by the project — that is, the binary distribution of software produced from licensed source code. A lot of software in FreeBSD “proper” is (naturally) BSD-licensed, and a lot of Free Software packaged by FreeBSD is (also naturally) GPL licensed. But the different licenses carry different obligations, so it’s good to keep track of the exact licensing applied to each bit of software.

To this end, there’s the LICENSE= line in each port’s Makefile. Its meaning is “this software has such-and-such a license”. For conciseness, SPDX identifiers are used, so that you can write

LICENSE=LGPL21

and we know what you mean. Because licenses can carry textual obligations (e.g. the GPL expects you to receive a copy, and the BSD licenses generally require you to include the copyright notice with distributions), there’s
an additional setting to include the actual text, called LICENSE_FILES:

LICENSE_FILES=COPYING.LIB

There’s a third source of license information, and that is the headers of the sources themselves. Usually you put a copyright-and-license header at the top of each file; some licenses such as the MPL even require some administration in there. The reuse.software site (by the FSFE) provides good guidance and best-practices information for providing licensing metadata in software packages.

Anyway, for KDE Frameworks 5 I was going through the ports Makefiles and adding LICENSE information. The KDE Licensing Policy tells me that frameworks should be licensed LGPL21+, with a few variants allowed; an interesting one is (at your option) “LGPL21 or LGPL3 or any later version approved by KDE e.V.” Right now, in 2017, this choice is the same as LGPL21+ because no later versions exist, but it is not always-in-the-future-same, so I hesitate to write

LICENSE=LGPL21+

for KDE Frameworks until I’ve checked the files. The license text is usually included, but it’s not quite consistently named, so I need to look into the tarballs anyway. And as a double-check, I read a couple of source headers to see if the license named in the code, matches the license text elsewhere (e.g. some files say LGPL 2.1 only).

It’s a bit of a slow process — one which upstream (that is, the source code) could support a little better with consistent naming. It’s also a process that needs to be monitored continually, to ensure that the whole body of software remains properly and consistently licensed — hopefully following best practices, too.

Anyway, as of today only ten of the KDE Frameworks 5 ports in the official FreeBSD ports repository  have all their licensing information set, to the best of my ability to check their accuracy. As an ongoing project in keeping-license-info up-to-date it’s not very high-priority but something that gets done in-between other things.

Retrospectacle

At the beginning of 2017, I was a programmer (mostly Python, and a little bit of C++) and spent most of my day at my desk, with an IDE open and a cup of coffee at hand. At the end of 2017, I’m a programmer (mostly C++, and a little bit of Python) and spend most of my day at my desk, with an IDE open and a cup of espresso at hand.

At some level of abstraction, not much has changed this year.

Of course, now I spend my entire day working on Free Software, in three different but partly-overlapping communities: KDE, FreeBSD, and Calamares. Basically you can track everything I do each day by looking in the relevant repositories on GitHub (read-only mirrors in the case of FreeBSD and KDE). Inspired by Michael, Krita, Matthieu Gallien (with the this-week-in-Elisa series) and Dominik Haumann, I’ve collected a list of things I did this year, in no particular order:

  • Released two versions of the proprietary application I worked on previously, and quit my job. Started working for BlueSystems and other organizations as a free-lance yet full-time programmer.
  • Updated Nethack in FreeBSD, and discovered I no longer have the Nethack chops to ascend half of the games I start. Updated Unnethack on FreeBSD, and fixed a clang-related bug upstream. Yay for roguelikes.
  • Released a dozen Calamares versions with regular bugfixes and small features. Worked together with the KDE VDG, Jens in particular, to figure out some of the UI choices — thanks for plying me with ideas.
  • Updated CMake a couple of times in FreeBSD and chased lots of weird CMakeLists code in unrelated applications.
  • Massaged KDE on FreeBSD under the expert guidance of Tobias and Raphael.
  • Attended three weeks of KDE-related sprints or meetings and a week of Akademy.

Here’s to a productive year, and looking forward to the same in 2018. You can see more of what happened in KDE in 2017 on the KDE year-end fundraiser page.

Sayonara

Not goodbye, but hello to a new music player on my desktop. It is already packaged for FreeBSD and has a long development history.

Music players stand or fall per individual user whether they satisfy the user’s needs — no duh there, but it means I should note what my use cases are before enthusing about some music player in particular. I would be fine with playing music from the command-line, most of the time: mpg123 --shuffle /mnt/music/*/*/* is just about right (except it fails with an argument list too long error in the shell). This is for music-while-I-hack, so I don’t listen too closely, it’s not hi-fi at all, basically I want “play in a genre until I switch it off“. Tagging is largely done when ripping my CDs (I still buy physical media!) and I don’t care for album art (I can look in the jewel case if I want that, and they’re all stacked in boxes upstairs). So play, pause, stop .. and if it can avoid mixing Mahler with Morrissey and Mötorhead, that’s a bonus.

Screenshot of Sayonara main windowOff to the left here is Sayonara, which describes itself as:

Sayonara is a small, clear and fast audio player for Linux written in C++, supported by the Qt framework. It uses GStreamer as audio backend. Sayonara is open source and uses the GPLv3 license. One of Sayonara’s goals is intuitive and easy usablility. Currently, it is only available for Linux.

Clearly that’s not true, because it’s available for FreeBSD as well 🙂 . I forget how I configured it; there’s a File > Open Directory entry and I probably filled in /mnt/music and it went and found everything that gstreamer will sensibly play. The screenshot is of the minimal playlist-only view. It can be broadened to include a library view which is a bit chaotic, but which searches reasonably well (e.g. for Mudhoney). Sayonara is actually where I discovered the luxury of stay-on-genre autoplay: when I started on some Dutch punk and it kept pretty close to that, wandering off to de Kift occasionally — but no Drs.P.

I contributed a few small fixes to get Sayonara building on FreeBSD, cleaning up the CMakeLists for newer CMake, that kind of thing. Now that it runs on my Plasma desktop all day, I can spot some more  minor issues — the system tray media player control doesn’t interact with Sayonara at all, for instance. For my use-case, that’s not so important, since the fire-and-forget genre play does it for me until I hit pause in the main UI.

I’m looking forward to KDE music players Elisa and Babe, for comparison purposes: maybe they tick my requirements-boxes just as well, or better. Certainly Elisa seems to be fairly playing-music-focused. I’ve even got a FreeBSD port for Elisa ready, just waiting for Qt 5.9 to show up on my doorstep (I can get it to compile against 5.7, but it won’t run due to QML runtime thingies).

CMake 3.10 on FreeBSD

The CMake port on FreeBSD is in the hands of the KDE-FreeBSD folks (since KDE was an early adopter), and generally it falls to me to do the CMake update while Tobias is still wrestling with Plasma and Raphael massages Qt 5.9 into our way of building. 3.10 was released five weeks ago, and it took a while to update the port.

It doesn’t take long because of CMake — they’re great, the code builds flawlessly, and there has been a real effort on the part of the KitWare folks recently to absorb our downstream patches so that we have less work in future (During the delay packaging CMake 3.10.0 Kitware even put out 3.10.1, which re-started some processes). It doesn’t take long because of new FreeBSD-specific features in CMake — you can now use CPack to create native FreeBSD packages, just in case you don’t want to go through the ports system or poudriere.

Nope, the problem is the 2000-odd ports using CMake. These generally behave, but every CMake update brings with it some ports that suddenly don’t build. So I spent a day on things like openvsp and scalapack, which fall over with 3.10 (and didn’t with 3.9). The causes of these failures is diverse; sometimes bad local CMake modules that don’t add all the right linking flags, sometimes bad C++ code, sometime stuff that has nothing to do with CMake at all but just happens to be newly triggered, and therefore my problem.

CMake 3.10.1 landed in the official ports tree with r457041, just before Christmas.

Naturally, there are some cases that fall out after an update, that are not caught before the update is committed. I’m told that FreeBSD on PowerPC doesn’t have a C++11-capable compiler installed by default, so I’ll need to massage that a little. Boost 1.66 came out very shortly after CMake 3.10.1, and that leads to new kinds of breakage. I’ve spent a half day compiling and re-compiling ceph, a distributed filesystem, trying to get that to work. It’s amazing sometimes how shifting one brick can bring down a whole house, and how we ever built the house in the first place.

Switching Distro’s

Obviously I still use FreeBSD on the desktop; with the packages from area51 I have a full and modern KDE Plasma environment. We (as in, the KDE-FreeBSD team) are still wrestling with getting the full Plasma 5 into the official ports tree (stalled, as so often it has been, on concerns of backwards compatibility), but things like CMake 3.10.1 and Qt 5.9 are sliding into place. Slowly, like brontosauruses driving a ’57 Cadillac.

In the meantime, I do most of my Calamares development work — it is a Linux installer, after all — in VMs with some Linux distro installed. Invariably — and especially when working on tools that do the most terrible things to the disks attached to a system — I totally break the system, the VM no longer starts at all, and my development environment is interrupted for a bit.

That’s always a good moment to switch distro’s .. since I’m going to spend an hour or so re-invigorating the VM anyway, reminding myself that this time I’ll make a clone and keep snapshots and whatnot, I may as well see what others use Calamares for. I’ve left a dusty and broken KDE Neon and a Manjaro behind me, and today I’m starting on Kaos.

For the very reason that Calamares can break stuff, and because new Calamares versions need to be tested on (older) live ISO images, there’s a deploycala Python3 script that helps install all the (development) dependencies needed. This is a bit of a drag, since it’s tied to a whole bunch of distro’s specific package names, but it gets the job done. I update it whenever I hit a new distro.

Kaos Linux Logo
Kaos Linux Logo

Kaos is a KDE-leading-edge distro, and daaannnggg is it ever slick in the first 10 minutes of use. The customization of Calamares is some of the nicest I’ve seen. The release-notes page could use some work (on my part) .. if only because I could read that stuff during unsquashfs, instead of before it. First-time startup with Kaptan is a hardly-intrusive way of configuring a bunch of separate things that would require a careful trip through systemsettings.

Look for the next few Calamares releases (in particular 3.2) to emerge from the Kaos, then.

More Calamares Releases

Another month passed, just like that. I spent last week holed up with some KDE people in the hills, watching the snow come down. While they did impressive things to the KDE codebase, I hacked on Calamares. Since my last post on the topic, I’ve been running a roughly every-other-week release schedule for the Calamares 3.1-stable branch. We’ve just reached 3.1.10. The reason for these stable releases is small bugfixes, minor polishing, and occasional non-interfering features.

Each release is announced on the Calamares site, and can be found on the Calamares GitHub page.

Calamares isn’t a KDE project, and aims to support whatever Linux distro wants to use it, and to configure the stuff that is needed for that distro. But when feature requests show up for KDE integration, there’s no special reason for me to reject them — as long as things can remain modular, the SKIP_MODULES mechanism in Calamares can avoid unwanted KDE Frameworks dependencies.

One new module in development is called “Plasma Look-and-Feel” (plasmalnf) and the purpose there is to configure the look-and-feel of Plasma. No surprise there, but ther point is that this can be done during installation, before Plasma is started by the (new) user on the target system. So kind of like the Look-and-Feel KCM, but entirely outside of the target environment. The UI is more primitive, more limited, but that’s the use-case that was asked for. I’d like to thank the Plasma developers for helping out with tooling and guiding me through some of the Plasma code, and deployers of Calamares (that is, distro’s) can look forward to this feature in the 3.2 series.

Speaking of Calamares 3.2, I’m intending to put out an RC fairly soon, with the features as developed so far. There will probably be a few RCs each of which integrates a new feature branch (e.g. a giant requirements-checking rewrite) with 3.2.0 showing up for real in the new year.

Calamares releases

It’s been a quiet month for me for blogging, but one filled with unexpected and weird and not-really-bloggable things. There was a trip to Berlin, where I had the pleasure of meeing up with a bunch of KDE people whom I hadn’t seen for over a month. Long time. There was also an accident with maple syrup, I’m sure.

Anyway, there have been four (!) Calamares releases since I last wrote about it on august 23rd (two months ago). These mix various bugfixes with various regression-fixes. This illustrates that I need a better set of acceptance tests before releasing — which take a surprisingly long time to set up, since the regressions are things like “Installing Chakra with network packages fails”, not simple stuff that is OS-, hardware- and installation-independent.

The latest release fixes that Chakra (or, rather, Calamares netinstall) problem. I’m happy to report that there’s a few new Linuxen evaluating Calamares as a system installer.

Plans for the near future include a 3.2 with some new features and much nicer support from new KPMCore releases, and ongoing care for the 3.1.x series. When and how, is in the hands of the vagaries of inspiration and long sessions with pen and paper figuring out just how things should work.