Maßnahmenkatalog für die Bundesregierung nach den NSA-Enthüllungen

Täglich erfahren wir neue Details über die elektronische Totalüberwachung unseres Landes und die vollständige Erosion unserer Privatsphäre. Unsere Regierung verharrt in Lethargie, spielt auf Zeit und schiebt den Schwarzen Peter im Kreis herum. Es ist klar, dass wir uns von diesem Marionettentheater keinerlei Abhilfe versprechen dürfen.

Nichtsdestotrotz möchte ich hier den Versuch machen, einige durchgreifende Maßnahmen gegen unsere Totalüberwachung und die Verletzung unserer Grundrechte zusammenzustellen. Eine Art TODO-Liste für unsere Regierung, anhand deren Abarbeitung man den ernsthaften Willen zu einer nachhaltigen Verbesserung der Situation festmachen könnte. Continue reading Maßnahmenkatalog für die Bundesregierung nach den NSA-Enthüllungen

Keyboard layout using XKB for coding

Keyboard layouts are something that should be ergonomic, but most of them aren’t. And as we’re all big creatures of habit, we don’t switch to better ones. Me included. Nevertheless, even if I’m using python a lot and brackets there are not as widespread as in other languages, I still got annoyed about them enough coding it with my German keyboard layout, to look for some improvements. English layout only partly improves the situation, as there is a bunch of brackets still not comfortably reachable. I was also always quite jealous on vim and emacs users, that they could tell their editors to use some keyboard shortcuts for cursor movement, but eclipse is too retarded for this. Nevertheless, keyboard shortcuts in vim also suck quite a lot, as they are adjusted for keyboards from stoneage. E.g. vim uses HJKL for left, up, down and right, so it’s shifted exactly by one key to the left compared to the standard position. Using ESC instead of TAB also seems to origin from that times. Changeing this consequently for all derived commands seems to be a quite a messy adventure deep into vims config. So it also sucks.

Fortunately using Linux the solution is easy. Some XKB configuration does the job:

xkb_symbols "coding" {
    include "de(basic)"
    name[Group1]= "German (with extra brackets and cursor)";

    key 	{ [         i,          I, Up ] };
    key 	{ [         o,          O, bracketleft ] };
    key 	{ [         p,          P, bracketright ] };

    key 	{ [         h,          H, BackSpace ] };
    key 	{ [         j,          J, Left ] };
    key 	{ [         k,          K, Down ] };
    key 	{ [         l,          L, Right ] };
    key 	{ [odiaeresis, Odiaeresis, parenleft ] };
    key 	{ [adiaeresis, Adiaeresis, parenright ] };

    key 	{ [    period,      colon, braceleft 	] };
    key 	{ [     minus, underscore, braceright, emdash] };

    include "level3(lwin_switch)"
    include "level3(caps_switch)"

The pasted 20 loc in the file /usr/share/X11/xkb/symbols/de plus according lines in xkb/rules/evdev and xkb/rules/evdev.xml fix the issue. What it does is to throw away the totally useless CapsLock functionality and rebind it with the function of AltGr. This way you can easily switch to level 3 key functions with your left hand, and we can throw in a bunch of new functionality to the right hand side. There we have cursor functionality for the keys IJKL, [ ] on OP, ( ) on ÖÄ and { } on , and -. We also have a new backspace on H.

I’ll probably also add some more stuff for the left hand side, as AltGr is quite nicely reachable with the right thumb, and I’d actually like to reduce the load on the left pinky some more. Doing lots of text marking, copying, inserting, holding down shift and pressing control a lot, it’s much more stressed when editing code, then when writing plain text. So the weakest of all fingers has to do most, which is quite weird and definitely awaits another fix.

Hello Vietnam

Speedy numpy replacement for matlab accumarray

I regularly have to translate some matlab code into python. Most functions there translate fairly well to numpy functions, but the accumarray receipe, that I used to use up to now, sucked quite hard performance wise. So I was looking for some more elegant solution. Unfortunately, there is not too much around, and I was already about to write something together, to ask it at stackoverflow, when I had the Idea for this little snippet:

def accum_np(accmap, a, func=np.sum):
    indices = np.where(np.ediff1d(accmap, to_begin=[1],
    vals = np.zeros(len(indices) - 1)
    for i in xrange(len(indices) - 1):
        vals[i] = func(a[indices[i]:indices[i+1]])
    return vals

Careful: This quick hack only works with contiguous accmaps, like 222111333, but not 1212323. Every change from one number to another will be seen as a new value. This avoids the slow sorting.

Benchmarking shows, that it’s more than 18x faster than the previous solution:

accmap = np.repeat(np.arange(100000), 20)
a = np.random.randn(accmap.size)

timeit accum_py(accmap, a)
>>> 1 loops, best of 3: 16.7 s per loop

timeit accum_np(accmap, a)
>>> 1 loops, best of 3: 887 ms per loop

For completeness, here the timings with octave:

accmap = repmat(1:100000, 20, 1)(:);
a = randn([numel(accmap), 1]);
tic; accumarray(accmap, a); toc
>>> Elapsed time is 0.05152 seconds.

Which actually makes me think of using some bigger guns for the problem now.

So after some days of hacking around with scipy.weave.inline now, I’m down to this:

timeit accum(accmap, a)
1 loops, best of 3: 27 ms per loop

This seems pretty reasonable now, when comparing it with octave.

The new implementation comes with fast implementations written in C for most common functions (sum, prod, min, max, mean, std, …), and falls back to a pure numpy solution for everything less common. It comes with a complete test suite, but if you should face some issues with it, please let me know!


This blog post is quite outdated right now. In collaboration with @d1mansion, all this developed further into some nifty little python package called numpy-groupies available at PyPI and on Github. For more info on this topic, usage details and benchmarks, see the project page at Github.

Lebenslaufschönen für Fortgeschrittene

Pressefreiheit quo vadisWie heise berichtet, scheint Herr Lindner kein großer Fan von Meinungs- und Pressefreiheit zu sein und damit den Namen seiner eigenen Partei gründlich zu karikieren. Dem Bericht nach hat der stets meinungsflexible Heißsporns des 3%‑Lobbyvereins zunächst seinen Lebenslauf bei Wikipedia schönen lassen, und geht nun anschließend juristisch dagegen vor, dass jemand das Kind beim Namen nennt. Erstes Opfer dieser Posse wurde dabei ein durchaus lesenswerter Artikel der Wirtschaftswoche, der von der eigenen Redaktion allerdings aufgrund mangelnden Stehvermögens mittlerweile entfernt wurde. Wer den Artikel als symbolgewordenen Ausdruck der Presse- und Meinungsfreiheit ebenfalls vermissen sollte hat Glück im Unglück: vergisst nichts!

Wikivoyage parser with heuristics

While travelling through Vietnam, I was using wikivoyage quite intensely as a travel guide and so I started to contribute to some articles there myself. When editing an article, especially cleaning up and structuring long semi-formatted lists of hotels and restaurant was quite annoying, but given the semi-structured shape of the lists, it’s not straight forward, to automate the formatting.

Being annoyed enough by the editing, I took it as a challenge, and wrote a parser making use of a bunch of heuristical rules to classify the list entries, split them into chunks, apply formatting rules on the chunks, and merge it together again into a nicely formatted list entry. So some ugly unstructred listing like

* '''Birmingham Buddhist Centre''', 11 Park Rd, Moseley (''#1, #35 or #50 bus''), ''+44 121'' 449 5279 (''[]''), []. A centre run by the Friends of the Western Buddhist Order'' .

* '''Hotel Indah Manila''' 350 A J Villegas St. Tel: ''+63 2'' 5361188, 5362288. [] Rates start at ₱2000 for this modest 76-room hotel. Facilities include Café Indah and conference and function rooms. Airport and city transfers, tour assistance, and laundry service are available.

becomes nicely formatted into

* {{vCard| type=sight| subtype=religious| name=Birmingham Buddhist Centre| address=11 Park Rd, Moseley| directions=#1, #35 or #50 bus| phone=+44 121 449 5279|| url=| description=A centre run by the Friends of the Western Buddhist Order.}}

* {{vCard| type=hotel| subtype=hotel| name=Hotel Indah Manila| address=350 A J Villegas St| phone=+63 2 5361188, 5362288| url=| price=Rates start at ₱2000 for this modest 76-room hotel| description=Facilities include Café Indah and conference and function rooms. Airport and city transfers, tour assistance, and laundry service are available.}}

I wrote it as a library and gave it a web frontend using CGI or as a standalone version using bottle. After using python intensely for several years, it’s actually the first time, that I used it to display some web content instead of PHP, and I was a bit surprised, how straight forward it was. So, give it a try, and let me know what you think about it! The source code is available at github.

Public letter to coffeebean Manila

Dear coffeebean team,

I’m referring to my message on your facebook page, which I was writing sitting in coffeebean M.Adriatico yesterday. I explicitly don’t want to blame your personel there, it’s a general thing. And it’s a problem not just with coffeebean, but with a lot of gastronomies in Philippines, so I’ll explain my complaints in some more detail.

Self service is something, that was invented, to provide cheap food to the lowest possible price. To save money on waiters, McDonalds and friends just skipped good service in order to provide a lower price. That’s how they became big initially. People who go there are either in a hurry or want to have cheap food and give a shit about service, as they plan to leave right after quickly swallowing the food there. You don’t sit around in a fast food store half the afternoon, because it’s not a pleasant atmosphere. Continue reading Public letter to coffeebean Manila