Personal clones on KDE infrastructure

I’m doing a little work on Tupi – the 2D animation application that joined the KDE community some months back — so that it builds on FreeBSD (the C++ code is wonderful, but the build system is qonf, which is not).
This has led me to the maze of git documentation on KDE’s infrastructure, and I’m taking notes so I don’t forget what I did. It’s also part of one of the things-to-do-at-Akademy on my list: talk to the techbase people to find out what the status and intentions are.
For the purpose of futzing with the build system, I’m using a personal clone of the repository. This is so that whatever weird-ass things I’m doing, they don’t pollute the upstream repository; eventually I hope to code-dump a CMake-based buildsystem into upstream. One that can live alongside the existing build system for platforms other than FreeBSD — although I think that in the long run having the same build system and dependency-finding-modules as all the rest of the KDE software would be good.
Basic git configuration is covered on techbase, which describes the mandatory (username) and optional (URL rewriting, colors, templates and exclusions) configurations for using git with the KDE infrastructure.
The KDE sysadmin git FAQ explains how to create a personal clone of a project repository. I’ve created a personal clone of tupi, and cloned that one locally. Then I work on that (in a project Neon VM and a FreeBSD jail).
The only thing is, my personal clone doesn’t get updates from the upstream (here, that means the original project repository) unless I pull them in myself. This is where additional remote repositories come from. In my clone, by default there’s two origin remotes (one for pull from anongit, one for push over ssh, as documented in the git configuration page).

[adridg@beastie]$ git remote -v
origin  git.kde.org:clones/tupi/adridg/tupi-cmake (fetch)
origin  git.kde.org:clones/tupi/adridg/tupi-cmake (push)

So what I need to do locally is pull in commits from the upstream, merge all of them with my local clone, and then push (which goes to my personal clone on KDE infrastructure). The git-scm book has clear instructions; for me it means adding an upstream repository:

[adridg@beastie]$ git remote add upstream kde:tupi

And then to keep up-to-date with the upstream commits and store them in my own personal copy, I need to do these four steps (this assumes that I don’t do anything in the master branch myself):

[adridg@beastie]$ git fetch upstream
[adridg@beastie]$ git checkout master
[adridg@beastie]$ git merge upstream/master
[adridg@beastie]$ git push

There’s probably a faster / better way of doing this, but at least I know I can keep up with the upstream. Now I can keep rebasing my work inside my local clone onto the latest upstream like this, until such a time as I’m ready to push my CMake branch to my personal clone on the KDE infrastructure. Once that’s done, I can ask the maintainer to pull it into upstream.

Posted in KDE | Leave a comment

Prep for Akademy

In preparation for Akademy I wanted to swap out the drive from my laptop — which is full of work-work things — and drop in a new one with stuff I actually want to have with me at Akademy, like git clones of various repositories. I spent a few hours wrestling with my Lenovo x121e (AMD) laptop and FreeBSD, which taught me the following:

  1. You can update the BIOS from a USB stick using only Linux tools, and
  2. FreeBSD does not like it when the SATA controller is in compatibility mode, and either hangs or fails to find the hard drive at all; in AHCI mode things are fine, but
  3. Even the updated BIOS cannot boot from GPT partitions, so I had to be careful during installation to manually do an MBR / fdisk-based installation (this seems to preclude ZFS as well), and then
  4. Wireless isn’t automatically detected (but the WAN modem is), and suspend-resume doesn’t resume.

This makes for  less-than-stellar performance for a conference laptop; I’ll fiddle with it a little before departing for Berlin in two weeks time (isn’t Akademy in Brno? Yes, it is, but the most effective train journey takes me to Berlin first to catch up with the trainful of KDE people at 12:46 from HBf), so I may end up being a FreeBSD person sporting an OpenSUSE laptop.

For development purposes — sort of as a quick counterpart to the FreeBSD VM where I’m doing qt5-based things for KDE applications — I installed a project Neon VM. This way, too, I can check that I’m not breaking anything on non-FreeBSD systems. What I’m seeing on the desktop in that VM is not very encouraging to me, though. As used as I am to the current KDE software on OpenSUSE or FreeBSD (4.12 or whatever), the newer software feels weird and arbitrarily changed and oddly slow. That last bit might be due to VirtualBox, I don’t really know. I’ll have to attend some of the VDG or HCI topics to get a better feeling for the (visual) changes already made.

Posted in FreeBSD, KDE | Leave a comment

My Akademy schedule

I for one applaud the decision to put Akademy-badges on the KDE community wiki.

I was afraid I was going to have to apply my awesome Kolourpaint skills again. Albert’s reminder has caused me to figure out my reasons for attending Akademy this year. So my agenda is four-or-fivefold:
  1. attend the AGM of KDE e.V.
  2. talk to Paul Adams about measuring community health. Technical aspects of measurement aside, I have some serious methodological misgivings about what he’s measuring and how he’s presenting it. This will require several beer mats of exposition.
  3. see if there’s any other FreeBSD users about. Or, for that matter, OpenSolaris people or anything else non-Linux.
  4. talk to Lydia and Valorie and other folks with knowledge of techbase and community to see if I can contribute there. There’s a lot of stuff that is undergoing a complete rewrite — but has been undergoing that for a long time.
  5. hear from the FrogLogic folks how their test-tools have evolved. It was a long time ago that we had some ideas of doing KDE HIG checks on the EBN.

There is of course also an implied 6(a) meet new people in the KDE community and 6(b) drink beer with them.

Posted in eV, KDE | Comments Off

SVN-to-git bridge (for practice)

Some time ago, I wrote that I needed to be less of a scaredy-cat about git (in particular so as to get back into KDE development, and the rat’s-nest of git repositories there was scaring me off — in that sense I’m a data point in what Paul Adams is writing about). The best way of learning is by doing, so I looked for something to do with git that would basically force me to use it regularly.
That really means “find a way to use git at work-work”, since most of my development hours happen there now (largely administrative number crunching in Python).
A bit of background: at work-work we have a central SVN repository. It has a non-standard naming scheme: trunk is called development, and branches are in the releases subdirectory. The only branches are for actual releases. For various reasons we are also using SVN 1.5, which means that we don’t have any of the more-modern merge and branch features that SVN has grown. So feature work by the developers happens in trunk directly, not in feature branches, and we end up with some pretty confusing history of interleaved commits.
I have 38 minutes on the train between Arnhem and Utrecht that I could use effectively for development of small things: typo-fixes, message improvements, adding unittests, that kind of thing. But I don’t want to end up with one big set of changes for all the little things I do on the train; I want sensible commits of one logical change after another.
So basically I want simple feature branches and offline commits. It needs to linearize history and integrate with a weird SVN setup. Previously I used Mercurial and its hgsubversion extension to get this effect; now I wanted to do the same with git, for learning purposes (and then I can futz with the KDE repositories again).
Most of what I eventually built to give me a nice git-based workflow that meshes with the central SVN repository is based on a series of blog posts
from TF Nicolaisen. They were really useful.
Anyway, my workflow now looks like this:
  • Pick a ticket N from our bug tracker (it’s TRAC, with some customized statistics modules I wrote and a handful of third-party TRAC-hacks for planning purposes),
  • Update my git repo from SVN with git pull,
  • Start a git branch for my work on the ticket with git checkout -b ticket-N,
  • Do my thing, with as many commits and experimental branches as needed, and then clean up (remove debug-commits, maybe merge some small steps), with git rebase -i,
  • Rebase onto the updated upstream SVN with git svn rebase,
  • Push the whole thing into SVN with git svn dcommit,
  • Drop the branch, since it’s in SVN now; the detached commits will get garbage collected eventually.
That’s a cromulent git workflow, and except for excessive rebasing and the push to SVN at the end, usable for regular git work as well.
The setup I’ve ended up with is illustrated here; on the server side, there are three repositories: first is the central SVN repository. This is the official and canonical source and other developers commit to SVN normally. Then there’s the git-fetch repository, which pulls revisions from SVN and puts them into git commits. This one has only the SVN commits in it. Third is a bare git repository, which is where I pull from and where the fetcher repository pushes things. This bare repository also has a few things that are not from SVN commits — I push branches here if I want to share them over git with other machines I work on, and there are a few tags in it marking some events in the history of the repository.
Client-side, there’s whatever clones I make of the bare repository.
Getting this set up was a matter of configuring and cloning the right bits; this I mostly did by following the steps described by TF Nicolaisen, except that I needed to get the authors map just right ahead of time, and I’m only git-bridging one single branch from SVN (namely, development, which is our trunk), and I’m not interested in ancient history (which is gnarly), but only fairly-recent commits from SVN. So here’s what I did:
  1. Figure out what part of the repository is interesting; for me, that was the development/ branch in SVN, from revision 28754 onwards.
  2. Figure out who is committing to the repository. SVN has the usernames, while git needs to have a name and email address. This requires a map of SVN authors to git authors. It’s also pretty much essential that the author in the mapping file matches the author and SVN username you configure in client-side git clones, or you’ll get a multitude of branches, all twisty and all very much alike. After some history examination (in SVN) and discussion on useful git author names with the other developers that might use git, we ended up with a file like this:
    adriaan = Adriaan <ade@example.com>
    bassie = Bassie <bas@example.com>

    I ended up committing this file into SVN so that it would be available — and could be updated — for general use. It lives indevelopment/.git-authors, ie. in the root of what I’m going to follow with git.

  3. Do the initial clone of the repository. Unlike the recipe here, I have an authors file (a copy exported from SVN, because it’s got to be there before the clone runs), and I don’t use the standard layout. I did this on the server, so that the SVN repository is local. The repository lives at/home/svn/project, and the fetching repo will be git-project-fetch
    cd /home/svn
    git svn clone -A author-map -r 28754:HEAD file:///home/svn/project/development git-project-fetch

  4. Set up the bare repo, which will be git-project:
    cd /home/svn
    git init –bare git-project

  5. Configure the fetching repo to push changes:
    cd /home/svn/git-project-fetch
    git remote add origin ../git-project

  6. Then modify .git/config so it reads as follows (again, this is all according to TF Nicolaisen’s recipe, only with a restricted SVN tree, a starting revision, and an author map):
    [remote "origin"]
      url = ../websites.git/
      fetch = +refs/remotes/*:refs/remotes/origin/*
      push = refs/remotes/*:refs/heads/*

    Because there’s only one branch here (namely development), there’s no need to configure which branch needs to be checked out by default.

  7. Add a post-commit hook to update the git repositories:
    if /usr/bin/lockfile -2 -r1 /tmp/project-gitsvn ; then
      ( cd /home/svn/project-fetch && /usr/bin/git svn fetch && /usr/bin/git push origin )
      rm -f /tmp/project-gitsvn
    fi

At this point, every SVN commit gets pulled into the fetcher-repo by the git-svn code and then pushed into the bare repository as if it’s a normal git repo. The server side is done. One the client side, I set up ssh access to the SVN (and now git, too) server. Then getting a correctly configured client-side clone is as follows:
  1. Use an ssh URL for access to both the SVN and the git repositories
    git clone -o mirror ssh://project.example.com/home/svn/git-project
    cd git-project/
    git checkout -t mirror/git-svn
    git svn init –prefix=mirror/ ssh://project.example.com/home/svn/project
    git svn dcommit

    That last dcommit — since the repo has just been cloned — just updates all the revision numbers. It’s not really necessary, since it will happen with the first real SVN commit from git anyway.

  2. Client side, we also need to set up the git author and the SVN authors files, so that they match with what happens on the server side. Failing to configure these consistently will cause lots of extra commits to show up in your local clone. Note I’m configuring this in the repo-local configuration, not globally, so it doesn’t interfere with the recommended KDE git setup. It uses the authors file I previously checked into SVN at the root of the development branch — now in the root of the git repo:
    cd git-project/
    git config user.name “Adriaan”
    git config user.email “ade@example.com”
    git config svn.authorsfile .git-authors

And with that, I’ve got a usable git clone that I can use for feature branches on the train, and that can easily push back to SVN. After using this for a few months I’ve finally gotten comfortable enough with git — feature branches, and sometimes futzing aroud to massage history into a usable form, and dealing with the rest of the git tools — to touch KDE git repositories again.

Posted in Bla Bla | Comments Off

Off the bike again

Welp, summer vacation is over again, cycled from Nijmegen south.

At the outset I said “we’ll go south till we have to speak French, and then turn around”, which meant “until Luik / Liége”. Didn’t make it that far, for the simple reason that campgrounds that are compatible with a family of cyclists with a tent are rare in that area.
Instead, we cycled through Belgian Limburg, across Dutch Limburg from Maastricht to Vaals, and then back up north to Venlo. I would show a map (as I did last year with Marble), but I did my GPS logging with My Tracks on Android this time, and I’d have to spend too much time mucking about to get the data off the device for processing.
Last year we rode 440km of completely flat Friesland and Groningen. This year there were hills (the Dutch Mountains!), and we weren’t sure how the kids would fare in more challenging terrain. We needn’t have worried. Mira’s “hey dad, let’s bike up the tallest mountain in the Netherlands” (322m above sea level, rising from Vaals at 160m, so this is not Tour de France material yet) was later followed by Amiel’s “hey dad, I want to ride 100km on the last day”. He’s nine years old and it took us from 10:30am to 9:00pm, but we got to the train station at Venray with 99.98km on the meter (and another 3km to go after we got off the train in Nijmegen).
Still got a few days off from work-work, so I’ve decided to pick up some KDE-FreeBSD stuff again, and Tupi in particular.

Posted in Bla Bla | 2 Comments

I believe some of us were younger then

As I crunch my way through all kinds of older data, I find lots of KDE memorabilia. Here’s the hackerheads, shot by Robert Scott for Akademy 2006. It’s just a small sample, intended (if I remember right, but there were also Curly Wurlies involved) to enliven Planet KDE. Compare the photos to Akademy 2013, and you can find a reasonable amount of overlap, showing the kind of continuity the KDE community has over seven years.

Posted in Bla Bla, KDE | 1 Comment

Yo, dogg, about those HDDs

HDD cleanup continues apace. I did like the wisecrack about semantic storage — although it’s not quite correct, since these are drives removed from otherwise decommissioned machines, or tarballs rolled of my university student account before the departmental Solaris server was decommissioned. It’s more like moving boxes never before opened (I know there’s several that I moved house with 8 years ago that are still awaiting an opening moment). In the mean time I’ve discovered a bunch of academic papers I had forgotten I had ever written and a bunch of OCaml I wrote that I no longer understand, as well as several versions of my bachelor’s thesis written in the winter of 1998.

Anyway, while futzing about with all these drives I’ve set up my main machine as FreeBSD 9.2-STABLE, running KDE 4.10 (which is as new as the KDE-FreeBSD ports are — 4.11 introduces some interesting new breakage which hasn’t been dealt with yet). I could repeat what I wrote several years ago: with KDE on top, noone knows what you’re running underneath and the applications and environment do all you need. And it’s nice having a very-close-to-vanilla-upstream desktop.
I bunged some more memory into the machine, and now I get the interesting effect that a VM plus its root disk image can fit into main memory. OpenSUSE has an amazing boot time if /dev/sda is secretly already in RAM.

But the niftiest thing I have discovered recently is VirtualBox raw disk access. This means I can (hot-) plug a drive into my machine and boot that physical drive in a VM. The setup is relatively painless, requiring only a little care with permissions and a complicated command-line. Here’s a screenshot of, left-to-right, FreeBSD dmesg showing me that I’ve switched on /dev/ada2 — a 200GB drive with Kubuntu 11.10 on it from when I was briefly a Python/Qt contract programmer — with VirtualBox and then the running Kubuntu session with its own idea of the physical disk. Hearing the physical disk rattle when I do something in a virtual machine is kinda cool, and this makes it much, much easier to figure out what files are interesting on any given drive.
But this bumps me into an interesting corner regarding KDE desktop settings and configuration. I’ve got a new desktop here. I’ve got an old desktop in the VM, and there are a bunch of settings that I’d like to transfer from the old desktop to the new one. Let’s disregard whole-desktop settings right now. Aaron Seigo recently posted something on his Google+ about Plasma-desktop settings transfer, which is promising next time I want to move my settings (focus follows mouse, yellow-and-orange wallpaper, no activities, 4 virtual desktops) to a new installation. I’m talking about application settings. For instance, I have a working Blogilo in the VM — what do I need to do to get the same Blogilo settings in another machine? That means copying the blog configurations and the passwords. So where do they live?
The userbase page for KMail has information about moving settings around; that’s the only instructions I could find for moving application settings. Those instructions are also somewhat incomplete, since passwords are probably stored in kwallet, and figuring out which stored passwords need to be copied over is a pain in the neck. Or take Blogilo. It  has a blogilorc file, but it doesn’t seem to record anything useful in there: no blog URLs, no usernames, nothing. Everything seems to be in kwallet, which exports whole wallets at a time. In other words, I can get at my blogilo configuration by exporting the wallet (unencrypted, plain XML), editing it to remove all the bits I don’t want (such as my IMAP passwords) and then importing it on the other machine. This seems error-prone and tedious — this time, it was actually just as straightforward to simply do all the configuration anew. It strikes me that there would be a market for a menu item Settings->Export all of this application’s meaningful configuration. As the computing experience becomes more fluid, moving from device to device, that might become more important — if I have configured something on my desktop, I want the same KDE application on my tablet to easily, seamlessly, take over the configuration from elsewhere.
.. and, hey, it turns out when I moved house last I used XFig to draw a plan of the house so as to juggle furniture (also in XFig) more effectively. These drives are a veritable trove of joy (and KDE configuration questions).
Posted in FreeBSD, KDE | 2 Comments

HDD Cleanup

Last week I sat down with a pile ‘o hard drives of varying vintage (although all of them SATA-era), ranging from 80GB to 500GB, which had been used in machines over the past ten years. The idea being to sort out what’s where and why and to consolidate the interesting bits. First order of business on each drive was figuring out what partitioning scheme and filesystems were on there. I suppose if you stick to one OS over time that’s a simple thing, but I have fdisk+ext2 filesystems, gpt+zfs, gpt+gmirror+gstripe+ufs and other things in the mix. Reconstructing the path from two physical drives labeled “Comice” and “Anjou” to a readable filesystem can take a while.

Anyway, interesting bits have been recovered, like an older mp3 collection of mine ripped from CDs I can no longer find (they must be somewhere in the house, in a box labeled “beware of the leopard” or something like that).
Some of the drives were once in various KDE servers or machines I gave acces to for KDE purposes, so it turns out there are various personal home directories that aren’t mine on these drives, too. This provides a bit of a memory lane. One drive, last accessed in 2007 and presumably part of the original English Breakfast Network machine, has logins for myself, and Allen Winter, but also a bunch of other users, not all of whom I can place. There are sad memories, such as /home/claire/ (empty, unused), and good times like last.fm-player-1.0.1_3.tbz  (get Last.fm with Amarok nowadays).
Anyway, if you are one of aacid bertbram bhards bille danimo dannya dfaure dhaumann fremy frerich gamaral gousiosg harald ingo jaison joerg kerian lofi padams pgquiles phil reinhold schuster scott sebas winterz zecke zenithar, and you think you might have left stuff behind, drop me a note and I’ll send you a tarball of your home directory. I’ll try to notify those logins I can still map to an email address as well.
After some grace period, I’ll destroy whatever data is left so we can say that those machines are well and truly tidied up.
Posted in KDE | 4 Comments

Cycling (with a few bits of KDE)

So while y’alls were frolicking at Akademy (with Dot coverage), I was on vacation. The choice between vacationing with my family or attending Akademy was a tough one: we also spent some time looking into getting the whole family to the conference, but the price tag and “what will the kids do there” tipped the scales. Maybe next year Akademy will be somewhere that fits in my summer vacation and is amenable to cycling.

Anyway, I picked up an Asus Fonepad before leaving on the trip and left my trusty N900 behind. Battery life and screen real-estate were the motivations there, and I wanted only one device to take along. The whole “you’ll look like a tool making calls with it, but it works” thing appeals to me as well. It’s mostly a fine device and served its purpose well. I’ll gripe about it elsewhere — suffice to say that I’d be quite happy to have something other than Android running on the Intel Atom inside it.
I didn’t use the Fonepad as a GPS tracker while riding around — it spent most of the travel days neatly packed away, while an Amaryllo GPS logger (writes to an SD card) was strapped to the handlebars with a bunch of elastic bands.
The logger writes NMEA sentences to the SD card, which I convert to KML with GPSBabel (because the tracker is occasionally flaky and I leave it on all day, even when taking a nap in a ditch, I use a filter set to position,distance=10m,time=120 to compress the log). That gives me something that Marble can work with. I can import the tracks for each day (File->Open will read KML) and that zooms to the area I’ve biked through. After every load I need to switch off “places of interest” in the Legend tab, though, or I get a gazillion meaningless tags (numbered) cluttering the view. This may be an artifact of my KML generation; I’ll have to look at the GPSBabel manual more closely some time. Once loaded, Marble’s “export map” feature produces a nice PNG so I can show you where I went on my summer vacation:
That’s about 440km total, which the kids biked with nary a complaint (almost). There’s gaps where the GPS flaked out or ran out of battery. Just to reinforce your preconceptions of the Netherlands: along the entire route it was flat, there were tulips, windmills and cows.
Another KDE angle, though: most of the trip I wore the KDE hoodie that I bought at FOSDEM — blue and snuggly an warm it is, and emblazoned with a big KDE logo. At one campground, a dad sitting out front of his tent asked me, “hey, that K-thing, that’s a Linux desktop, isn’t it?” Bringing the number of times someone “in the wild” has recognized KDE stuff to two, for me. (“In the wild” means not a KDE person, not at a Free Software conference, and not a university CS student) So don’t get the KDE hoodie for the recognition it’ll get you in the train, get it to support KDE (and to be warm and snuggly).
Posted in Bla Bla, KDE | 3 Comments

Badges? We don’t need no badges.

It’s one of those yearly things, scheduled for less than two months from now. Frankly, I’m a little surprised that no one else — Paul Adams is a usual suspect — has bunged up some badges for this year yet. So here’s my entry for Akademy 2013 in Bilbao, showcasing, as always, my most excellent kolourpaint skills. And, like it says on the tin (brass? what material are badges made of anyway), Akademy doesn’t fit my schedule this year either. My only remaining hope is to integrate the conference with a three week train-and-bicycle vacation for two adults and two kids.

Posted in KDE | 2 Comments