I spent a day – spread out over some weeks – chasing a failure in a KDE unittest up and down the stack, in KDE libraries, Qt libraries, ical, and C-library-level timezone code.

It was worth it // going through life without a timepiece // did pay off – the Sugarcubes, Traitor (1988)

The KDE PIM code deals with lots of timezone information: after all, you want appointments to appear in the right local timezone (so the timepiece on your wrist matches the appointment) and when you communicate an appoinment to a remote colleague (say a friend in Brasil) then you want the time to appear correctly over there, as well.

Several of the unittests were failing on FreeBSD, so I got roped into checking out what the problem is. What could possibly go wrong?

Well, at the end of the day I know a lot more about timezone changes in Prague (that’s a lousy transliteration of Praha .. fie on you, English orthography!) than I feel I really need to, and have butted heads with K&R style C code, which is something I thought I left behind me in 1998 or so.

Timezone Names

Generally speaking, an application on a Linux or FreeBSD machine knows what timezone it “lives in” based on the $TZ environment variable. The “short” form is to use a timezone abbreviation, like so:

$ TZ=MST date
Tue Jul  7 01:51:00 MST 2020
$ TZ=CET date
Tue Jul  7 10:51:15 CEST 2020

While the abbreviatons are short and simple, they’re also rather ambiguous. That’s why the “longer” form is to use a zone name like America/Edmonton or Europe/Amsterdam.

$ TZ=America/Edmonton date
Tue Jul  7 02:56:05 MDT 2020

These zone names correspond to files in /usr/share/zoneinfo, which contain information about that specific timezone. More on what’s in those files later.

But what happens when a zone name is used that doesn’t have a corresponding file? It falls back to UTC (Coordinated Universal Time, which is roughly what was called Greenwich Mean Time before 1967 – Wikipedia has a bunch of history if you’re interested). You can see that in the following two examples (Wikipedia can also tell you lots of things about Muskoka):

$ TZ=America/Muskoka date
Tue Jul  7 09:06:50 UTC 2020
$ TZ=GMT date
Tue Jul  7 09:06:06 UTC 2020

“So what?” You may ask, “of course GMT displays UTC!” Except that it doesn’t: on FreeBSD, GMT does not exist as a timezone file, and so we’re seeing the fallback for UTC rather than displaying GMT itself. There’s files GMT0, GMT+0, GMT-0 and Greenwich for actual GMT displays, but not one called GMT.

Twitter tells me that this has been so in FreeBSD’s base system since 2009, and that the port misc/zoneinfo adds it back (along with, presumably, updates – although both 12-STABLE and the port currently have timezone files version 2020a).

Rolling all the way back: on FreeBSD, GMT isn’t necessarily a valid time zone name.

Failing Tests, Chapter 1

Here is a snippet from one of the failing unit tests in KDE PIM-related code (in the KDE Framework kcalendarcore):

void RecurTodoTest::testAllDay()
{
    qputenv("TZ", "GMT");
    Todo *todo = new Todo();
    todo->recurrence()->setDaily(1);
    todo->setCompleted(currentUtcDateTime);
    QVERIFY(todo->percentComplete() == 0);
}

This is heavily edited for space, but comes down to this: a Todo-item, that recurs daily (e.g. you need to do it every day until it ends, and this one has no end date) is not completed when you do it today (because tomorrow is a recurrence of the Todo).

Except if the timezone is invalid, then the recurrence rules are screwed up: there is no current date in an invalid timezone, nor is there a representation for right-now (UTC) in that timezone. So code internals end up with “there is no tomorrow” and the recurrence of the Todo item is ignored, the item is considered complete, and the test fails.

Now that I write this, I also see that the test leaks the Todo, although that’s of limited importance in a test.

Fixing Tests, Chapter 1

I’ve submitted a “fix” for this failing unit test which is to set the timezone to UTC instead. I’ve “fixed” the machines that run the CI for KDE-FreeBSD by copying Etc/GMT to GMT. That just moves the problem elsewhere.

There is an underlying problem with recurrences in invalid timezones, though – some of the unit tests were timing out when given an invalid timezone, so I think there are things to investigate still. The test ConnectDaily2 just sits there forever (or 30 seconds, which is my personal limit of patience for unit tests). This only applies when the test is run in an invalid timezone (e.g. GMT, before my “fixes”).

Failing Tests, Chapter 2

Prague Castle by Night, via Wikimedia: Karelj, Self-Photographed, Public Domain Here’s a picture of Prague Castle. At night. Maybe it’s winter. At what time was the picture taken? What time is that in UTC?

One of the unit tests in kcalendarcore tests the round-trip conversion of timezone information from iCal (which is the format used to represent appointments in an interoperable form) to Qt and back. And that test fails on FreeBSD, not on Linux.

It so happens that the test uses timezone Europe/Prague. This is an existing file. When converting timezone information back-and-forth, you end up also with a representation of the transitions. Here’s part of what we expect on Linux:

BEGIN:VTIMEZONE
TZID:Europe/Prague
BEGIN:DAYLIGHT
TZNAME:CEST
TZOFFSETFROM:+0000
TZOFFSETTO:+0200
DTSTART:19790401T010000

In April 1979, conforming to EU time zone rules (even if then-Czechoslovakia was not part of the EU, it apparently used the EU time zone guidelines for CET / CEST), there was a jump of 2 hours.

On FreeBSD, there is a 1-hour jump at the end of 1978, and another 1-hour jump in april 1979. This comes down to differences in zic (the zone-info-compiler, which takes a textual representation of the history of a timezone and spits out a binary file). With the same input, Linux zic and FreeBSD zic produce different outputs, and the test – in KDE Frameworks, which is a tremendously long way away from where the problem originates – fails because of that.

I still had a VM with a Debian installation from 2018, which shows the same behavior: there’s an extra transition in 1978. Apparently zic was changed at some point.

This is something I don’t really know how to fix effectively.

What I’ve added to the unit tests is some extra checking for the variations in the timezone file – because that is actually what is causing the problem in the test: different input files. The tests now demonstrate clearly that FreeBSD has different input files than current Linuxes.

Getting to the bottom of things means wading through the sources for zic, which is C code and fairly old C code at that and it gives me a tummy-ache. The FreeBSD version and Linux version seem to have diverged from the same codebase around 2010. So for now the root cause is going to remain unresolved, but the upper levels of the stack are slightly improved.