At the Randa sprint 2016, we're talking about KDE om all the platforms. And I'm here to represent the traditional packaging scheme. FreeBSD ports are created by writing a Makefile which contains some settings and then invokes some magic. The Porter's Handbook describes how.

But here in Randa there are efforts on doing Continuous Integration not only on Linux but also on Windows, and there's talk of automation of package building (packages of what? for which devices? we don't know yet). So I thought I'd take a look at a particular port Makefile and try to figure out what parts of the information in that Makefile come from -- or could be automatically sourced from -- somewhere else.

Here's the port Makefile for devel/kf5-bookmarks (that's the internal name for the port). I've numbered the lines, and will be commenting inline.

     1  # Created by: Tobias Berner
     2  # $FreeBSD: branches/plasma5/KDE/devel/kf5-kbookmarks/Makefile 12693 2016-06-13 18:31:53Z adridg $

These first lines are comments, with an SVN tag that's been expanded. Seems that I was the last person to touch this port Makefile. This is boilerplate.

     4  PORTNAME=       kbookmarks
     5  PORTVERSION=    ${KDE_FRAMEWORKS_VERSION}
     6  CATEGORIES=     devel kde # kde-frameworks

Here's a really important item: the name of the port. This is the only identifier for the port that we can find inside the Makefile. The categories are a way of organizing the ports; in this case, devel must be listed because the port is devel/kf5-kbookmarks, and kde is because it's from us. So line 4 is important, and 5 and 6 are boilerplate.

     8  MAINTAINER=     kde@FreeBSD.org
     9  COMMENT=        KF5 library for bookmarks and the XBEL format

Line 8 is boilerplate again, and then line 9! That's something that could be clipped straight from the metainfo.yaml file. After all, there's a description: line in there with pretty much the same text. So, knowing that the port is kbookmarks, if we know what repository to get stuff from for kbookmarks, we could get the metainfo and extract the information for the COMMENT.

    13  USES=           cmake:outsource compiler:c++11-lib kde:5 tar:xz
    14  USE_QT5=        buildtools_build core dbus gui linguisttools \
    15                  qmake_build widgets xml

This is dependency and build information. We use various packages for building and set options on those packages: usually the KDE tarballs are xz-compressed (not always: that's one of the unpleasant surprises packagers have to deal with; at least BSD-tar has a 'z' flag that's smart); we're doing an out-of-source cmake build. That kind of stuff.

The Qt dependencies are fairly fine-grained. This is a consequence of the BSD run-time linker, which doesn't like transitively linked shared libraries. In other words, linking an application to QtWidgets (and QtWidgets links to QtCore) doesn't link the application to QtCore. If the application also uses QtCore symbols, then it needs to explicitly link to QtCore.

I'll write more about dependencies below.

    16  USE_KDE=        codecs config configwidgets coreaddons ecm \
    17                  iconthemes widgetsaddons xmlgui
    18  KDE_DIST=       frameworks

Here, KDEDIST distinguishes between KDE Frameworks, KDE Plasma and KDE Applications, since they are delivered in slightly different ways. For instance, the Frameworks come with translations, while Applications don't. So that variable controls some settings elsewhere. USE_KDE is the list of dependencies within KDE that this port has. This list uses the same names as you'd find in PORTNAME; a later KDE Framework or Application that needs bookmarks would list USE_KDE=bookmarks. More about dependencies below.

    20  OPTIONS_DEFINE= NLS
    21  OPTIONS_SUB=    YES

Let's call this boilerplate. Frameworks have translations, and we have an option, NLS, for building and installing translations (or not). So Frameworks all have this. Applications don't, because they have translations shipped separately.

    23  .include

Line 23 is where the magic happens.

So to sum up:

  • Lines 4, 6 and 18 establish the identity of this port: it's devel/ in the filesystem, the resulting package is called kbookmarks and it's a KDE Framework.
  • Lines 13-17 are build and runtime dependencies.
  • The rest is boilerplate, though some of it depends on KDE_DIST being equal to "frameworks".

Back to dependencies. There are (at least) two ways that we can find dependenciesy information about a KDE Framework. One is to try to build it and to see where CMake complains. The other is to look at the kde-build-metadata repository. The kde-build-metadata repository contains a file where KDE documents the dependencies within the collection of KDE source repositories. So knowing that I'm working on the port for framework kbookmarks, I can query in-KDE dependencies with the tool list_dependencies.

Wouldn't it be great if CMake's find_package() also wrote a (machine-readable) record of what package was looked-for and whether it was optional or required? (And, while I'm dreaming, whether it was for build- or runtime use). That way you could get a straight-from-the-source list of dependencies.

Lacking that, list_dependencies uses a dependency file maintained by the KDE community. And using it, I find some surprising differences between the dependency information in there, what CMakeLists.txt says, and what our port Makefile says.

First off, the CMakeLists.txt for kbookmarks says:

find_package(Qt5 ${REQUIRED_QT_VERSION} NO_MODULE REQUIRED Widgets Xml DBus)

find_package(KF5Config ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5CoreAddons ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5Codecs ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5ConfigWidgets ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5IconThemes ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5WidgetsAddons ${KF5_DEP_VERSION} REQUIRED)
find_package(KF5XmlGui ${KF5_DEP_VERSION} REQUIRED)

Maybe there's more find_package calls, I didn't look everywhere. But this is a clear statement of dependencies. Matching these lists against our ports Makefile shows:

  • Each direct dependency on another framework is reflected in USE_KDE's list of frameworks. That's good!
  • Each Qt component explicitly looked-for can also be found in USE_QT5. That's good!
  • But there are other entries in USE_QT5. That's because kbookmarks links (privately) against Qt5DBus. Possibly we could drop core from USE_QT5, and there's three build-tools listed (I'm not sure linguisttools is a runtime requirement). So this is almost good: I get the feeling there's a missing find_package, and our list is a little on the bulky side.

Turning to the information in kde-build-metadata, I can ask for direct dependencies, or transitive dependencies. Here's the direct list:

frameworks/kbookmarks:
        kdesupport/extra-cmake-modules
        Qt5[stable]
        fram~/kdesrc/log/2016-06-14-10/kcalcore/cmake.logeworks/kxmlgui
        frameworks/kconfigwidgets
        frameworks/kwidgetsaddons
        frameworks/kcoreaddons
        frameworks/kiconthemes

This is roughly the same list as spotted in the CMakeLists.txt and in the port Makefile, but there's a catch (or two):

  • The Qt (sub)module information is lost. This is documented at the top of the dependency-information file, so it is not surprising, but it is a little annoying.
  • The frameworks KF5Config and KF5Codecs have vanished, in spite of being explicitly searched for in CMakeLists.txt. There's an explanation for this, because frameworks/kxmlgui depends on them (and lots else like kitemviews). So it looks like transitive dependencies have been removed here (this is not systematically done: kcmutils depends both on kxmlgui and kitemviews).

So, at least from my point of view, the dependency information could use some refinement (and/or expansion). But that's why I'm in Randa! So I can sit down with the people most-responsible and figure out what we need, and what can be provided. You can support bringing together the right people in Randa though this year's fundraiser.