Jonathan Lange

Echo and Bounce: Finns and Greeks

noreply@blogger.com (jml) — Mon, 20 Feb 2012 23:44:30 +0000

Following the resolutions in my last post on reading, I've finally finished two of the heavier books that I'd started: The Art of Rhetoric by Aristotle, and The Kalevala, a cycle of Finnish epic poetry. Some reflections:

The Kalevala

I actually only read the first volume, which is the first twenty songs
For what it is, it's surprisingly readable and engaging. Good stories, lots of vivid poetic images, even some humour
The introduction compared it to other great ancient epics, such as the Iliad
It's written in trochaic tetrameter (tum-ti-tum-ti-tum-ti-tum-ti) and translated that way too. That metre is particularly infectious and I found my internal monologue beating that way after reading a section

The Song of Hiawatha is written that way too, apparently inspired by The Kalevala

I knew nothing about Finnish mythology and now I know next to nothing but here's some stuff:

There's no pantheon and not many obvious gods, just one high God, "Ukko"
Most of the characters don't seem like ordinary men and women. The translation says "heroes", which fits, but you need to think bigger than Odysseus or Achilles
Birch trees, some constellations, the moon and the sun all are persons, in some fashion
Mothers matter way more than fathers

Really wish I knew more about its provenance. It's hard to tell how much is truly ancient, how much has been influenced by Christianity and how much was made up by the compiler.
Lots of singing. If you hated Tom Bombadil or the Silmarillion (Mikey, I'm looking at you), then you should probably skip this one
I'm probably not going to read Volume 2 any time soon

The Art of Rhetoric

This is the first Aristotle I've ever read. I'm really interested in the subject, initially because I wanted to be conscious of when I'm using rhetorical tricks and when they are being used on me.

The book is more than just a series of tricks. It's about the philosophical and psychological underpinnings of persuasive speech: what kinds of arguments there are; how to come up with them; which ones are best suited for which purposes; what things work regardless of what kind of speech you are making; how emotions fit into things and so forth. What I had previously thought of as rhetoric – rhetorical style – is a smallish bit at the end.

It's a really tough, technical read. Lots and lots of it didn't stick, and I am sure it would repay a deeper study. The translator tries his best, and his introduction was helpful, but even he tended to be cumbersome. He once described something as being "not inimitable". I don't know what that means.

Even though it's really tough and technical, I was impressed by Aristotle. When I was growing up, I mostly knew of him as the guy who got science completely wrong and helped screw up the Middle Ages. Reading this, he comes across as very sharp, creative and interesting. Also, since he and the translator frequently refer to his other works, he comes across as very prolific. It's amazing how many different fields he contributed to. I realize this isn't exactly news.

It also makes me realize just how much cool stuff was going on in Classical Greece, and how modern much of it seems to me. I guess I'm late to this game too, but when I read things like this I feel as if our culture has this super-interesting older cousin with an excellent music collection who we never talk to because we can't be bothered. Or something.

I wish I had a classical education, or at least that I could go off and study classics for a while. I really think I would learn things that would be useful today. Or perhaps our culture has absorbed everything that is useful or good or relevant about their culture and it would all be a mildly diverting exercise in academic self-gratification.

Mere Code: testtools 0.9.14 released

noreply@blogger.com (jml) — Thu, 16 Feb 2012 10:55:29 +0000

testtools's sister project, subunit, was using a private API that we deleted in the 0.9.13 release. This release restores that API in order to smooth out the upgrade path.

If you don't use subunit, then this release won't matter very much to you.

Echo and Bounce: Reading

noreply@blogger.com (jml) — Wed, 15 Feb 2012 12:15:14 +0000

A few quick reflections on my recent reading:

Have decided to apply kanban principles and limit my books in progress and my queue of next books
Have set myself a rule that I'm not going to start any new books until I've cleared off the ones on my bedside table and the ones in my "In progress" collection on my Kindle
Just finished "I sleep in haysheds and corners". Some stories are better than others. I particularly liked the opening story for its Tasmaniana and Will Cox's story.
Also just finished "Why Classical Music Still Matters". Many good ideas, but ultimately disappointing as the author insists on florid prose in the tradition of critical theory. This makes some points impenetrable and others seem like literary word games. Would love to read a book on this topic that was more direct.
I don't keep a reading log. Benny & Jean-Paul both do, and I was taught to by my old mentor Martin Chance, but I don't. I should. I want to. How does one start so late in the game? What format does one use?

Also, I'm more active on Google+ these days than on my blog. You might want to follow me there.

Mere Code: testtools 0.9.13 released

noreply@blogger.com (jml) — Mon, 06 Feb 2012 15:55:12 +0000

It has been a while, but testtools 0.9.13 is finally out! Lots more matchers and bug fixes, as well as improved error reporting. Full release notes on Launchpad.

Thanks to James Westby, Graham Binns, Francesco Banconi and Robert Collins for making this release possible.

Mere Code: Simple made easy

noreply@blogger.com (jml) — Wed, 01 Feb 2012 13:50:38 +0000

Rich Hickey did a great talk at Strange Loop called "Simple Made Easy". You should watch it.

When I tried to explain the talk to someone, I stumbled a lot and it was obvious to me that I didn't really understand it. So I'm going through it again and turning it into a blog post, purely for my own gain.

This is roughly the first half of the talk. Not much of my own analysis or opinion is inserted, and I've pretty much stuck with Hickey's illustrations and phrasings. Thus this post is pretty derivative. Oops.

Simple vs Easy

"Simple" means one thing, "easy" another. Simple is the opposite of complex. A thing is simple if it has no interleaving, if it has one purpose, one concept, one dimension, one task. Being simple does not imply one instance or one operation: it's about interleaving, not cardinality. Importantly, this means that simplicity is objective.

Easy is the opposite of hard, or difficult. A thing is easy if it's near to hand, if it's easy to get at (location), if it's near to our understanding (familiarity) or skill set or if it's within our capabilities. This means that ease is relative.

Speaking English is dead easy for me, but that doesn't mean that speaking English is intrinsically simple. I find French quite difficult. Little French children speak French all the time, and there's always a part of me that thinks, "Boy, those kids are clever, being able to speak a foreign language at that age", but that's silly. It's easy for them, it lies near to them.

This distinction between simple & easy is good one, and is useful in all sorts of areas. But how does it relate to software?

Constructs vs Artefacts

As programmers, when we make software we are working on constructs: source code, libraries, language concepts and so forth. Rich contends that we focus on the ease of use of those constructs: How many lines of code? How much boilerplate? Will new developers be familiar with our technology?

But all of this is secondary. What actually matters is the artefact, the running programs that users actually use. Does it do what it's supposed to do? Does it do it well? Can we rely on it working well? Can we fix problems when they occur? Can we change it? You know, the interesting problems.

Thus we need to be assessing our constructs – our code, our technology choices – based on the attributes of the artefacts that we'll create, not based on the experience of typing code in.

Limits

We can't make something reliable if we don't understand it. And, actually, everyone's understanding is pretty limited. We can all only hold a small number of things in our head at once.

When things are complex, many parts are tied together by definition. You can't pull out just one piece and consider it because it's intertwined with other pieces. This creates an extra burden to understanding a system and thus makes it difficult to reason about the system.

You do need to reason about a system, both to know what to change and to be able to do so without introducing defects. Tests, refactoring, rapid deployment and all that are great, but to make a change to the system safely & without fear still requires you to be able to reason about it. Every bug in your product that was found in the field passed the type checker and passed all of the tests. Your type system doesn't tell you what change to make next in order to get the software you want any more than guard rails on a highway tell you how to get to Grandma's.

Speed

Focusing on ease and ignoring simplicity means that you'll go really fast in the beginning, but will become slower and slower as the complexity builds.

Focusing on simplicity will mean that you'll go slower in the beginning, because you'll have to do some work to simplify the problem space, but making sure that you only have intrinsic complexity means that your rate of development will remain at a high constant.

There are no actual numbers for this.

Complicating constructs

Many complicating constructs are available, familiar, succinctly described and easy to use. But none of that matters to end users. What matters is the complexity they yield. This complexity is incidental, it's not intrinsic to the problem.

If we build things simply, then the resulting system is easier to debug, easier to change and easier to understand.

Compare a knitted castle to a castle made of Lego. The knitted castle might have been great fun to make, and might have been really easy if knitted using a loom and cutting edge knitting tools, but there's no way that it's easier to change than a Lego castle. It's not about the ease of construction, it's about the simplicity of the artefact.

How can we make software easier?

Well, we can install it to make it easier by location. We can learn it and try it to make it easier by familiarity. We can't do much about our capabilities though. If we want software to be easier to comprehend, we are going to have to bring it down to our level. We have to make it simpler.

Take Lisp as an example. It's hard for many people because they don't have a Lisp installed, or their editor doesn't support paren matching, but they can make it easier by installing a Lisp and getting a plugin for their editor. It's also hard because it's unfamiliar. Who'd have thought that parens could go on that side of the function? But you can gain that familiarity quickly enough.

But parens in Lisp are used for functions and for grouping data. That's hard to get your head around, and that's because it's complex. It braids together two distinct notions.

Mere Code: Undistract me

noreply@blogger.com (jml) — Mon, 23 Jan 2012 17:44:37 +0000

Here's a thing that happens a lot to me: I'm doing some work, and as part of that work I need to run a command in my terminal that takes a little while. I run the command, look at it for about a second and then switch to doing something else – checking email, perhaps. I get deeply involved in my email checking, and then about twenty minutes later I switch back to the terminal and see the command has finished. For all I know, it finished nineteen minutes ago, and I was just too engrossed to notice it.

This is a big productivity sink for me, especially if the command happened to fail and need retrying. I'm not disciplined enough to just sit and watch the command, and I'm not prescient enough to add something to each invocation telling me when a command is done. What I want is something that alerts me whenever long running commands finish.

Well, that thing now exists, thanks to glyph's script that provides precmd and postcmd support to bash and a lot of help from Chris Jones of Terminator.

To use it right now:
$ bzr co lp:~jml/+junk/shell-tools
$ . shell-tools/long-running.bash
$ notify_when_long_running_commands_finish_install

You'll see that if you run a command that takes over 30 seconds to complete, it will pop up a notification, which should hopefully take you away from whatever it was you are doing and back to the task at hand.

If you look at the code, you'll see that it installs two hooks: precmd and preexec. preexec runs just before the shell launches a command, and precmd runs just before it prompts for the next command. Our preexec stores when the command was launched and the precmd checks to see if it finished within a certain time frame. If not, it sends out a notification.

Currently, you'll get a notification when you finish reading a long document, since the command finishes a long time after the command starts. Obviously this isn't ideal. I think the fix is to only send notifications when the shell doesn't have focus. Unfortunately, that's a little tricky and I think is going to be highly terminal specific.

Anyway, I'm a total shell newbie, so I'd love to know if there's any way this could be done better. Also let me know if you find this useful, or you know of someone who has already done this.

Mere Code: What are my projects?

noreply@blogger.com (jml) — Mon, 26 Dec 2011 04:54:40 +0000

Launchpad doesn't really have any good mechanism for letting you review a list of "your" projects.

That's partly because there are a lot of different ways that it could do it. You could be the maintainer of a project, or its driver, or its bug supervisor, or you might have commit access to its trunk branch. All of this could be direct, or through membership of a team. It's tough.

Since it's that time of year when I review my projects, responsibilities, goals and the like and start to figure out what I want to do next year, I want to figure out what are my projects on Launchpad.

Luckily, Launchpad has an API. I can't use it to figure out what projects I'm the maintainer of, but I can use it to figure out what trunk branches I have commit access to. The link has the Python code for the script.

Really, Launchpad should allow me to curate my own list of projects, with input to that list coming from all of the sources mentioned above, as well as arbitrarily selecting projects.

Mere Code: pyflakes update

noreply@blogger.com (jml) — Mon, 21 Nov 2011 14:29:05 +0000

Thanks to radix, exarkun & dash, my branch to pyflakes to warn about duplicate definitions of classes finally landed. I did the work a year ago as an outrage-powered, opportunistic fix after I saw a co-worker struggle with tests weirdly not failing (Turned out it was a huge test module and there was another class at the bottom with the same name). I'm very happy to see it landed.

For those who haven't been paying attention, official pyflakes development is now taking place on Launchpad as part of the divmod.org project. The trunk of that project now has the best version of pyflakes known to man.

pyflakes is the best static Python checker. It's fast, and has very few false positives.

Update: A couple of people have asked me about lp:pyflakes. It's dead. It died when the divmorg.org trac instance died ages ago. Don't use it. To get pyflakes or any other divmod project, use lp:divmod.org.

Mere Code: Automatic packaging update

noreply@blogger.com (jml) — Thu, 10 Nov 2011 12:40:33 +0000

I just blogged on developer.ubuntu.com about the work we're doing with automatic packaging. Read it!

Mere Code: How to feel better (or, some tips on refactoring)

noreply@blogger.com (jml) — Fri, 04 Nov 2011 18:29:19 +0000

A few months back I gave a lightning talk at the Launchpad Thunderdome about how I do refactoring. It's very opinionated, and mostly applies to big, old code bases, but worth writing up anyway.

The core idea here is that very few things make me feel as good as deleting code. I love cleaning up code and the clean code base that results, and I'm sure that many others feel like me. As such, this is a guide on how to feel better.

1. Know your enemy

“Functionality is an asset, code is a liability”. Truer words were never spoken. Every line of code is a potential source of bugs and a barrier to understanding, and thus carries a maintenance cost.

Maintain an awareness of things that need refactoring. Here's a quick and incomplete list:

unused code – this can be deleted
boilerplate – this should become a function or class
wrong documentation – these should be updated
two ways of doing something – perhaps there should be one
bad names – change them to something that makes you think less

2. Keep a "yak stack"

Yak shaving is "any seemingly pointless activity which is actually necessary to solve a problem which solves a problem which, several levels of recursion later, solves the real problem you're working on." (Jargon File)

A few of us have extended the concept beyond the "actually necessary" to include anything that's making the task at hand more difficult and less fun but is not worth fixing right now. Hence the yak stack. Here's how it works:

Whenever you come across something in your code base that is difficult to understand or that slows you down: make a note of it. When you've finished the task at hand, fix the problem. If you encounter other things that slow you down, write them down. Work through the list.

3. Every option is wrong

In a big, old code base, there are probably many, many areas that need refactoring. Don't worry about which is the "best" place to start – there is no best place.

4. Start from green, stay green

Never, ever refactor while your tests are failing. Refactoring is about changing the shape of code while preserving its behaviour. It's much harder to be sure you're keeping the behaviour if you are comparing one set of thirty tracebacks with another set of thirty tracebacks. Better to compare a passing ("green") test run with another passing test run.

Run tests frequently. More often then you think you should. Commit often – think of it like quick save in a tough level of a video game. It frees you up to experiment more and means you have less in your head at any one time.

5. Do not nest

Don't begin a refactoring while you are in the middle of another refactoring. If you find you must, use tools like 'bzr shelve' to store your current diff and then work from the clean head of your branch.

6. Keep moving, leave a trail

Don't get bogged down in details, otherwise you'll never finish. Literally. Someone will come along and distract you and before you know it, three months will pass and your refactoring branch will be full of conflicts. If you see something you are unsure of, mark it with a XXX or a FIXME or a TODO or whatever works for you and then continue with what you are doing.

Tools like 'bzr todo' can make it really easy to check to see if you've added any XXX comments.

7. Translate tests with care

As said above, refactoring is about changing the shape of code while preserving its behaviour. When you update tests, you risk changing your definition of the system's behaviour – so be careful.

8. Confront uncertainty with destruction

If you see some code and you are not sure if it's needed, delete it. Doesn't matter if it's a whole function or just an odd line. If you have a test suite, and it was important, that will catch the failure. If you have version control, and it was important, one of your collaborators will notice and revert the change.

If it was important and neither of these happened, then your whole project has learned something new about itself, and that's probably worth the hassle. (Oh, add tests & better docs after this happens.)

9. Good grep tools

Remember that symbols aren't only referenced from files that match *.py. In big code bases there are often other sorts of files that refer to symbols in the main code. In Launchpad, for example, we have ZCML and doctest files that refer to symbols. When you want to know how something is used or you want to rename something, make sure you use a grep command that actually finds everything.

Ideally, you should be able to run this command faster than you can think about wanting to do it.

Personally, I use 'bzr grep' a lot. Others recommend 'ack'.

10. There will be failures

Mentally prepare yourself for the fact that the first two or three full test runs after your refactoring will fail. This is especially important for code bases that have multi-hour test run times.

If you think this way, then you won't be as discouraged when it actually happens.

11. Finish the job

Busy people refactoring a big code base are often tempted to apply a refactoring to only a single part. For example, some useful function is extracted from repeated boilerplate in a few other functions. However, many, many other instances in the code base continue to use the repeated boilerplate.

This is almost worse than just leaving the repeated boilerplate. There are now two idiomatic ways of doing the activity. Further, other developers who work on other parts of the code base probably won't find out about it, and might end up repeating your refactoring. Ugh. How is anyone new expected to get to grips with this code?

Similarly, if a class, function or concept is renamed, rename it everywhere, especially in the documentation.

It's difficult and often tedious, but it really is worth taking refactorings to completion. Apply them to the whole code base, or not at all.

Note that I'm referring to completeness, not perfection. If you block on perfection, you never get anything useful done. If you aim for frequent incremental improvements, you soar.

12. Read these books

I highly recommend "Refactoring" by Martin Fowler and "TDD by Example" by Kent Beck. I stole many of these ideas from them.

Over to you

This was very much just a dump of how I do refactoring when hacking on Launchpad. I'm always keen to learn more, and would love to hear about what works for you.

Echo and Bounce: Outcomes of meetings

noreply@blogger.com (jml) — Thu, 13 Oct 2011 15:24:23 +0000

At Canonical, we normally work in a distributed environment. As such, when we do get together, it's either for intense, multiple day meetings or intense, multiple day hacking sessions. Either way, there's a bunch of useful talk and progress, and this often needs to get written up.

While minutes and daily reports are a barrel of fun, for actually doing things and telling others how much you've done, you only really need to record three types of thing:

Next Actions – what you're going to do next, and who's going to do it
Decisions – you've had discussions and come to conclusions, what were they?
Achievements – things that actually got done during the meeting / hacking session

You can remember these with the convenient acronym, NADA.

Echo and Bounce: Time isn't money

noreply@blogger.com (jml) — Wed, 28 Sep 2011 10:19:31 +0000

According to The Economist:

Money has three main qualities:
as a medium of exchange, buyers can give it to sellers to pay for goods and services;
as a unit of account, it can be used to add up apples and oranges in some common value;
as a store of value, it can be used to transfer purchasing power into the future.

Can time be a medium of exchange? I guess sort of, but it's a little like barter for services. Much better to use money for that.

I guess it can be used as a unit of account. It took me as long to write this as it takes me to fold my laundry and clean my room. Learning French to conversational levels might take as much time as playing through Arkham Asylum and the next Assassin's Creed. Useful things to know.

Turns out you can't store time though. Those few minutes of spare time you have can't be put into a time bank and then redeemed for extra spare time in the future. It goes stale very quickly.

Mere Code: testtools 0.9.12 out

noreply@blogger.com (jml) — Wed, 14 Sep 2011 11:01:19 +0000

testtools 0.9.12 has just been released!

It's a huge release, this one. We normally try to release much more frequently, but for this release I wanted to wait until all of our known unicode handling bugs were fixed. Today, Martin [gz] finished off his heroic branch to fix up assertThat, and I figured it was time to release.

In addition to all of the unicode fixes, we've really cleaned up the way test failures are displayed. A lot of the boilerplate around the traceback has been removed, a lot of levels of the stack are gone (although you can get them back if you want), and assertThat is way less repetitive. This all really adds up. If you're using an old release, you want to upgrade right now. Honest.

For me, this release is the one where using matchers becomes really and properly fun. As such, all of our assertFoo methods are now implemented in terms of matchers.

Of course, we have our usual raft of fixes, helpers and new matchers too. The full changelog is on Launchpad.

In addition to Martin [gz], thanks to Christian Kampka, Robert Collins and Canonical for making this release what it is.

If you're new to testtools, it's basically a way to do serious, tasteful unit testing in Python.

Documentation lives at http://testtools.readthedocs.org/en/latest/.
PyPI: http://pypi.python.org/pypi/testtools
All development takes place on Launchpad: https://launchpad.net/testtools
Join us on #python-testing on Freenode

We don't currently have a release PPA, but we do have a daily builds PPA. The trunk is kept stable using Jenkins, so it's fairly safe to use.

Echo and Bounce: Date picked!

noreply@blogger.com (jml) — Tue, 13 Sep 2011 21:24:36 +0000

For all of those readers who might be interested, and foreign friends who might be in London at the time, let it be known that Joliette & I have picked the date of our wedding. Let it be May 12th, 2012.

Echo and Bounce: Engaged!

noreply@blogger.com (jml) — Sat, 10 Sep 2011 13:36:59 +0000

A couple of weekends ago, Joliette and I were on holiday in Barcelona, visiting our friends Jamu & Jenn.

While we were there, the four of us walked up to the top of Tibidabo, a mountain overlooking Barcelona. We walked to the very top where Joliette and I separated from our hosts to take a closer look at the church. After looking around at its interior a bit, we went outside. I drew Joliette aside underneath some trees away from the crowds, got down on one knee and asked her to marry me.

She said yes.

And then I started breathing again, and gave her the ring that I had been carrying around like a fool in "Europe's pickpocketing capital".

Since then, we've told our families, announced on Facebook and are mere hours away from deciding on a date. Wedding planning has begun in earnest. As Joliette's family is mostly in America, and mine is mostly in Australia, and we're marrying in London, we have our work cut out for us.

Mere Code: doctest really isn't very good

noreply@blogger.com (jml) — Fri, 09 Sep 2011 17:39:48 +0000

I just got sick of trying to decode obtuse doctest diff errors when using NORMALIZE_WHITESPACE and ELLIPSIS options. Although doctest gives you hooks to do something about this, it's really hard to actually write the logic.

See, NORMALIZE_WHITESPACE also normalizes line breaks, and '...' can match across multiple lines. That means that you can't take a line-based approach, which makes it really hard to clean up a diff. Anyway, here's my attempt: https://code.launchpad.net/~jml/testtools/better-doctest-output-checker/+merge/74842

It's a pain, because both of them would be very useful without their line-spanning behaviour. I guess without the line-spanning behaviour of '...' there'd be no way to indicate multiple lines of crap in output.

My suggestion in the mean time? Don't use doctest.

Mere Code: Progress on auto packaging

noreply@blogger.com (jml) — Tue, 23 Aug 2011 18:09:54 +0000

Instead of a junk branch, the auto packaging code now has a proper home on Launchpad. I've filed bugs for all of the things that I could think of that were wrong with it. Actually, that's a lie. When I spot something wrong in code or behaviour I make a note, usually a XXX comment. The bugs up there are the ones I got from grepping my notes.

Anyway, minor rant on effective use of brain and the gift of literacy aside, I've also done a reasonably detailed specification. It's been a long time since I've done a spec this technical & detailed, but I hope it works out. If you have any comments or questions, ask here or on the spec itself.

Echo and Bounce: Coffee

noreply@blogger.com (jml) — Fri, 19 Aug 2011 18:09:26 +0000

I'm not a morning person. At all. I really don't like waking up, or indeed doing anything before 11am. Sadly, not being possessed of a fortune and thus able to live life at the mellow tempo of my choosing, I have to put up with it.

I've been doing an experiment to see if I can hate waking up less. The idea is to give up coffee and all other drinks with caffeine and see if I can draw from my own latent energy. After all, most of the "buzz" that regular coffee drinkers get is actually the relief of the withdrawal symptoms that they've developed over night. Or so I'm told.

Last week I gradually cut down my coffee drinking from "way too much" to "two shots of espresso" to "one shot" and then to no coffee at all. No tea, no coke, no other sources of caffeine besides the occasional square of dark chocolate.

I had major headaches on day two of the "two shots" stage, and felt mildly lethargic most of the week. Especially had a great deal of trouble focusing.

The weekend was my first "no coffee" days. Mild headaches & severe lethargy. No need to focus. Thanks to a very patient Joliette for two great days.

This week, the headaches and the lack of focus have been pretty tough. Today I've been headache free but very much craving a coffee as a way to signal, "right, time to knuckle down & get to it". The focus-fog seems to be worst in the late morning and best in the afternoon.

I've also been going to sleep earlier, although I've been doing more exercise, which may be a part of that.

I'm still waiting for the day when I bounce right out of bed, eager to face the world, and for a day where my brain focuses like it does on caffeine. I'm hoping the day turns up next week, because I'm definitely going to have a cortado or three when I'm in Barcelona.

Mere Code: How dependency guessing works

noreply@blogger.com (jml) — Thu, 18 Aug 2011 12:18:20 +0000

Although in my last post I talked about our proof of concept tool, I didn't actually explain how to use it. That's mostly because it's not quite ready to be used by others. There's a good reason.

The most interesting thing that pkgme-binary does is to guess the dependencies of an application given only its tarball. The way it does this is by finding all of the ELF objects (executables, shared libraries etc.), reading the symbols from them, and figuring out the packages based on those.

Canny readers are already thinking, "Yes, that is what dpkg-shlibdeps does" – which is true. As some background, Debian packages that export symbols for linking also provide explicit metadata about those symbols and what versions they appear in, in order to allow Debian packagers to figure out dependencies easily using dpkg-shlibdeps. Fascinating reading can be found in the Debian policy manual, a spec on improving dpkg-shlibdeps and the guide on using the new symbol files.

Unfortunately, dpkg-shlibdeps can only query packages that are installed on your system. If you have a binary with symbols that are not provided by any library on your system, you cannot use it to calculate dependencies. Debian (or the Debian flavour of your choosing) might have a perfectly good package to satisfy that dependency, but you won't be able to find it.

So, to do dependency guessing properly, you need to have a database mapping symbols back to packages. Debian already has something like this in its mole database. Ubuntu doesn't have anything like this that I know about.

Assembling and maintaining such a database is pretty much a simple matter of programming, but it's still work that I wanted to avoid for the proof-of-concept. Instead, I used the mole database in Debian for the calculations. That means we'll get wrong answers for Ubuntu, but it demonstrates that the concept works.

Since I have a copy of that mole database locally, and since it's about 1.5GB, I'm not distributing it with the branch. Which means that the automagic binary packaging branch won't actually work for you.

Mere Code: Automagic packaging of binary apps for Ubuntu

noreply@blogger.com (jml) — Thu, 18 Aug 2011 11:57:39 +0000

For the last few weeks I've been working on a tool to automagically package binary applications for Ubuntu. The idea is that anyone who wants to distribute a binary app on Ubuntu should be able to do so without having to learn how to package it.

I've used pkgme to build a proof-of-concept: which you can look at here: lp:~jml/+junk/pkgme-binary.

Or, you can watch the demo:

Right now, the tool assumes that the tarball:

is an application
contains ELF objects that can be scanned for symbols, which determine the dependencies
has one main executable file
that its contents can be copied into /opt/$PACKAGE/ and can be run from there by an ordinary user
comes with a JSON file specifying extra metadata

Our hope is that 90% of the binary applications we get will meet these requirements.

The plan is to take this proof-of-concept and turn it into something that will run server-side behind the Ubuntu Developer Portal. Next steps are to spec it out, create a project, start filing bugs and fix the bugs I already know about.

Watch this space for more updates.

Thanks to James Westby and Adam Conrad for their help in doing the proof-of-concept and to the dozen or so people who helped me get the darn screencast out.

Mere Code: launchpadlib helper

noreply@blogger.com (jml) — Tue, 02 Aug 2011 11:05:11 +0000

I often need to muck around in a Python interpreter to figure out what I need to do with Launchpad's API. As it is, I write out the same commands over and over.

Since repetition is a stupid boring job that we should force machines to do while they are still subservient, I wrote a harness that, when run, gives you a Python interpreter and a few useful objects for playing with the Launchpad API.

For example:

$ ./lpharness.py 
lp:     authenticated Launchpad object
me:     logged in user
anon:   anonymous Launchpad object
errors: launchpadlib.errors
uris:   launchpadlib.uris

pprint: pretty printer
>>> pprint(list(me.searchTasks(status='In Progress')))
[<bug_task at https://api.launchpad.net/1.0/launchpad/+bug/240067>,
 <bug_task at https://api.launchpad.net/1.0/launchpad/+bug/418932>,
 <bug_task at https://api.launchpad.net/1.0/hydrazine/+bug/535414>,
 <bug_task at https://api.launchpad.net/1.0/hydrazine/+bug/574981>,
 <bug_task at https://api.launchpad.net/1.0/hydrazine/+bug/612641>,
 <bug_task at https://api.launchpad.net/1.0/tarmac/+bug/683351>,
 <bug_task at https://api.launchpad.net/1.0/wikkid/+bug/695232>,
 <bug_task at https://api.launchpad.net/1.0/launchpad/+bug/721166>,
 <bug_task at https://api.launchpad.net/1.0/ensemble/+bug/728320>,
 <bug_task at https://api.launchpad.net/1.0/bughugger/+bug/731075>,
 <bug_task at https://api.launchpad.net/1.0/tarmac/+bug/807785>,
 <bug_task at https://api.launchpad.net/1.0/pkgme/+bug/809447>,
 <bug_task at https://api.launchpad.net/1.0/ubuntu-archive-tools/+bug/805634>]

Script: http://paste.ubuntu.com/657088/

You are free to do whatever you like with it. I hope that someone puts it into some useful, maintained, centralized place for doing stuff with Launchpad. Perhaps launchpadlib itself.

Mere Code: Ubuntu on my iMac

noreply@blogger.com (jml) — Sun, 10 Jul 2011 13:52:30 +0000

Long time readers will know of my attempts to get Ubuntu running on my iMac. Now that I'm working within the Ubuntu Engineering team at Canonical, I figured it would be a good time to try again.

Installing natively fails on 11.04 in the same way that it failed on 10.10: the installer sends output to the Mini DisplayPort rather than the actual screen. I don't have an external monitor that I can hook up to the DisplayPort, so that's pretty much that.

I can still run a VM, right? VirtualBox is pretty good, and I got me a running install of Ubuntu with the Unity shell without too much fuss. Problem comes with keyboard configuration. The literal keypresses Cmd+H and Cmd+Q are not passed through to the guest OS. With a Dvorak layout these become Cmd+D and Cmd+", and if one maps Cmd to Meta then you cannot easily kill-word in Emacs. That means for me to have a useful environment, I'd have to make do with Super and Meta swapped around from the usual PC layout.

VM Fusion doesn't have this problem and has a nicer UI, but I can't run Unity or compiz under it. This makes Ubuntu feel really old and clunky, and I'd rather not do that.

Parallels, well, at this point I'm running out of the will.

Think I'll settle on VirtualBox for the time being.

Echo and Bounce: Super powers

noreply@blogger.com (jml) — Sun, 03 Jul 2011 10:56:34 +0000

It used to be that if you asked me which super-power I wanted, I would tell you, "instantaneous, effortless, accurate, error-free teleportation". Something very much like Jumper, except with fewer determined foes and better acting. After all, that would mean no more flights, no more uphill walks, no more being late, no more forgetting things and plenty more holidays.

But recently, I've changed my mind. I want to be Video Game Man. I want the super-powers that every video game character actually has that aren't even treated as super-powers.

Infinite stamina: Practically every video game character can run, jump, climb and fight forever without becoming tired, experiencing fatigue or slowing down in any way. This would make kickboxing much easier.

Self-sustaining: No need to eat, drink or sleep. Well, maybe if I got injured I could eat an apple to cure me. Think of the time saved.

Pause: When things get too confusing or fast-paced, or when a difficult decision must be made, I could pause and take as much time as I wanted to think about it. Of course, it would only be thinking.

Quick save, quick load: Attempt difficult situations over and over again. Accidental death would not be a problem. Negotiations could be optimized. Future knowledge could be used to enhance past decisions.

Friend or foe: Know instantly who the bad guys are. No more moral ambiguity for me, thank you very much!

Know thyself: Clear knowledge of all of my own capabilities – no more Dunning-Kruger effect for me! – and also of my own status, health, wealth etc. Always obvious what I could improve and how much I could improve.

Maps: Always knowing where I am, and where I need to go.

Unbreakable: Although I could be injured, it would be impossible to break my limbs, blind me, deafen me, or even slow me down. No possible accident could reduce or remove any of my innate capabilities.

Looking over this list, I'm reminded of The Culture, a utopia designed by Iain M. Banks and the little I know of the Reality is Broken meme. Which reminds me, most reader of this blog should look at "Don't Play Games With Me".

Oh, and if you're able to get me that teleporting gig, I guess I could be convinced to settle for that.

Mere Code: Leaving Launchpad, going to work on Ubuntu

noreply@blogger.com (jml) — Mon, 20 Jun 2011 11:10:17 +0000

After four and a half years of working on Launchpad, I'm moving on within Canonical, taking a position on the Ubuntu Engineering team, working to get more and better apps on Ubuntu.

I am very excited about the change, but also a little sad, since it will mean leaving what has been the best team I've worked on. They are a wonderful bunch of people who not only get a lot of great stuff done, but consistently surprise me with their constant drive to get better at what they do.

Over the last couple of years, we have changed the way we are organized so that we can focus on building a small number of features at a time, the features that we do churn out are much, much better, we are rolling out new versions of Launchpad multiple times a week and we are actually paying off technical debt.

And they've needed to: Launchpad is a massive, ambitious application. I've never worked on something so big that aims to do so much, and with so few people actually hacking on it.

I still care a lot about Launchpad, It's crammed with potential and has so much to offer the open source world. It's going to be hard for me to move on.

But, wow, what a place to move on to, and what a time to do it. Ubuntu is aiming to go from about 20 million users to about 200 million, and to do that is going to need way, way more apps than they have now. To do that, we'll need a spiffing developer programme, and that's where I come in. I'll be loading up on information, plans and more concrete goals in a couple of week's time in Dublin, and will share them as I can.

Oh, and if you want to guide the world's best open source development collaboration platform into the future, do drop me or Francis Lacoste a line.

Mere Code: testtools 0.9.11 released

noreply@blogger.com (jml) — Sun, 12 Jun 2011 13:08:41 +0000

While I slept, Robert Collins released testtools 0.9.11. This release is mostly a bug fix release, but the fixes are very much worthwhile.

In particular, Martin [gz] got our Python 3 support going again, Gavin Panella fixed up a dodgy Matcher and fixed a bug in how we feed information from fixtures into test results.

Also in this release, Tim Cole and Martin Pool made their first ever patches, for which they have my hearty thanks and congratulations. (The congratulations are also hearty.)

As always, it's just so darn pleasant to see so many fine people working together to make an open source project work.

Mere Code: testtools 0.9.10 released

noreply@blogger.com (jml) — Mon, 11 Apr 2011 12:44:58 +0000

There was a bug in testtools 0.9.9 that prevented it from being easy_installed. Tres Seaver has fixed this bug, and I've rolled out a new release: testtools 0.9.10. Sorry for the inconvenience folks.

Mere Code: testtools 0.9.9 released

noreply@blogger.com (jml) — Fri, 08 Apr 2011 12:00:17 +0000

testtools 0.9.9 has been released today. For me, the twin pillars of this release have been our vast documentation improvements and the new matchers, which make it very, very easy to make meaningful assertions about complex objects. Well worth checking out.

Thanks to Christian Kampka, Robert Collins, Aaron Bentley, Jelmer Vernooij and Michael Hudson-Doyle their contributions.

Mere Code: mdz on listening to users

noreply@blogger.com (jml) — Tue, 08 Mar 2011 09:02:09 +0000

Matt Zimmerman writes about whether we should listen to users. It's a great post, and you should read it, but the summary is:

If you want inspiration for a new product, don't listen to users, watch them
To test whether a product is a good fit, ask users for feedback and listen to it
If you want to incrementally improve an existing offering, don't listen, gather data

I'm over-simplifying, you should read the original post. I also wonder whether it should be "when is it safe to not act on user feedback", rather than "not listening" per se.

Echo and Bounce: Sun day

noreply@blogger.com (jml) — Sun, 20 Feb 2011 13:41:37 +0000

Winter is dragging on in London. There's a tree outside my window, perhaps I've told you about it before. When I first moved here a year and a half ago, its branches were bare and through the gaps between the ends I could see a dead building made of black and glass and a solid grey sky. It's like that now. Every time I look out my window I hope to see flowers budding or leaves growing, telling me that Spring is really here.

It's odd to care so much about it, and to look for those signs. In Australia, so many plants are evergreen, and you have to be as far south as Tasmania for seasons to mean much. Flower buds and singing birds are the sort of metaphors you read in books and have to be explained to you, in much the way that the rain-in-a-dry-land metaphors in the Bible have to be explained to the British.

I've spent the morning replying to personal correspondence, most of which has been sitting in a virtual drawer for months and months. I confess that a part of me saw it as a duty that had to be done, but that part quickly shrivelled once I actually started to re-read and reply to the emails. What I dreaded from a distance as a chore quickly became a pleasure when up close, and I truly am blessed to have such friends.

Now there are lazy Sunday chores to do before I'm off to church. Cleaning, organizing, reviewing lists, making lists and carefully avoiding becoming lost in code, books or video games.

Echo and Bounce: Twitter

noreply@blogger.com (jml) — Wed, 02 Feb 2011 12:11:00 +0000

There are a lot of interesting people on Twitter, and it seems like a good way to be part of a broader conversation. I quite like the short format of Facebook status update, so it figures that tweeting should come naturally.

The problem is I always slide off Twitter like a sock off a warm glass window (try it sometime!). I just can't stick to it. Anyone else had this problem? Is it a tech thing or an attitude thing? Am I better off leaving it alone?

Mere Code: Music is too hard

noreply@blogger.com (jml) — Wed, 02 Feb 2011 12:04:01 +0000

It's way too hard to keep all of my music stored on one machine in a lossless format, have it backed up to another machine, synced to my iPod in MP3 and have it pleasingly playable on my OS X computer. Way too hard. Really.
Especially since fixes to track listings and so forth seem to vanish like smoke in the breeze. Banshee and Rhythmbox both pretend to help, but that's only so they can mess with my life more.

Echo and Bounce: The time is out of joint

noreply@blogger.com (jml) — Sun, 30 Jan 2011 12:28:46 +0000

I mentioned a while ago how it felt like I'd been living in some pretty big times. Since then, subjective time has been torn out of the shoulder socket of habit and things have been in a bit of a mess. A good mess, but a mess nevertheless. There's nothing for it but to recount what's happened.

Buckets of fun were had over Christmas. I spent the Eve and the Day with Ian and Rowan, two friends from church who shared a wonderfully family-like Christmas with me. Ro even made a stocking with my name on it. I lost every single board game I played, but there was nice port. I also got a chance to have a video call with most of my Tasmanian family. Funny to think that was the first time I saw my parents in over a year.

After that, I returned home and played video games for a week solid, while Joliette and practically everyone else were away. Enslaved is very good, and Batman: Arkham Asylum is even better, but you probably knew that already.

Square-eye fun ended, the new year rolled around and I packed my bags and went to Dallas for three weeks. Now, there are many fine people who live in Dallas, many of whom have been kind to me. But it's a hole.

Granted, I spent most of the time in a hotel in conference rooms without natural sunlight talking about some of the more arcane aspects of distributed open source software development, but even so, it's not one of my top ten favourite cities. In each of those cities, you do not need a car to cross the street.

The first week in Dallas was spent doing the single geekiest thing that has ever been done in that city. Our old group from Tassie was re-formed, along with special guest James Whelan. My monk kicked arse.

Eventually, after both play and work were done, I left Dallas (yay!) and went home to London. Jetlag has thrown the time out of joint in a different way, although if I had more discipline I would recover faster. Returning to kickboxing after six weeks out of practice is an exercise in humility, as was watching some of the clips from my grading session.

J & I went to see Hamlet on Wednesday. It's my first time watching it on stage, and the first performance J has ever seen. I was very pleased. I had forgotten just how good it is. The production emphasised the constant surveillance of the Danish court, Kinnear played the Dane as a hyper-sensitive introvert who shields himself with wit, and Calder did an excellent Polonius.

Driving goes well. After warming up a bit I seemed to be as good as I was six weeks ago. Apparently, I've just got to master maneuvres, roundabouts and the motorway before I'm ready for my test.

Also, did my first ever yoga class yesterday. I was expecting it to be challenging because I'm not very flexible. It turned out to be challenging because I'm not very strong. I'm surprised that the male yoga practitioners I know don't look substantially beefier, to be honest. Also, I hadn't expected to break a sweat. Nice that there wasn't much of a "spiritual" element to that, as there are certain teachings that I cannot submit to nor appear to submit to.

That's it, I think. Hope all is going well with you.

Mere Code: Expressing agreement and disagreement numerically

noreply@blogger.com (jml) — Tue, 25 Jan 2011 18:02:02 +0000

There's a habit within Canonical on expressing agreement and disagreement numerically. If we agree with something someone says in email or IRC, we'll often say "+1", which means "I agree!" or "if we were to vote about this, you would have my vote". I sometimes hear people actually say the words "plus one" in conversation.

Some have extrapolated from this to say "-1" to mean "I disagree", which is a pretty natural thing to do. However, it confuses me a lot.

You see, the "+1" thing is a little bit like the Apache voting system, which I met in my early days of open source, back when I lurked on the Subversion development mailing list. The voting rules there were burned into my mind, and I foolishly assume that they are equally fundamental to everyone else.

In the Apache system, "-1" means "veto", which is a very strong way of disagreeing. When people on the Launchpad say "-1", I never really know what they mean. I don't blame them, "-1" is a bit of a silly way of vetoing a proposal. I bet if keyboards had a key for ∞, there'd be a much more sensible way.

I guess the moral of the story is, "ambiguous shorthand is ambiguous".

Incidentally, how does the Apache voting system work out in practice these days?

Mere Code: Larval prototype for Launchpad dashboards

noreply@blogger.com (jml) — Thu, 23 Dec 2010 16:09:57 +0000

I mentioned a while ago that I really want to see something like a dashboard in Launchpad, some kind of view that shows you everything that you must do, everything that you are waiting on from others and all of your current work-in-progress. Launchpad could do this really well, since it has rich, inter-linked data about what's going on and since it can show you this information for all of your projects.

Today, I knocked up a very quick-and-dirty prototype for this. It shows all of the work-in-progress for a person across all of Launchpad, grouped by project.

The code lives at lp:~jml/+junk/whip and you can see examples of my work-in-progress and jelmer's work-in-progress online. You should be able to make your own with './bin/whip $LP-NAME > wip.html'. Note that there'll be some PYTHONPATH shenanigans.

Hackers, I'd love to see if you could turn this prototype into a web-app, or even a page on Launchpad. There's a NOTES file in there with whatever ideas I've had.

Designers, there's got to be a better way of showing this data than what I've picked. Take a look at the examples and see what you can turn them into.

Echo and Bounce: Big times

noreply@blogger.com (jml) — Wed, 22 Dec 2010 20:11:02 +0000

We've had some big times over the last couple of weeks.

Two weeks ago, my mind was focused on grading, on the day when I would be tested to see if I would be awarded a blue belt for kick-boxing. Everything reminded me of grading, or was bad because it might harm my chances, or was good because it would help, or was too big to think about before grading. All plans and thoughts passed through the nexus of that cold Sunday morning in a large indoor dojo.

On the day, I felt this strange mix of confidence and uncertainty. I was healthy, I had trained hard, I knew all of the techniques, my teachers said I was ready and I really wanted that belt, but I had never done anything this physically demanding or "sporty" before and I didn't know if some rebel part of my mind would tell me to quit half-way through.

The grading was hard, but I passed. I've been awarded my blue belt, and I have to say that there are few achievements of which I am more proud.

If all I'd done over the last two weeks had been to get this belt, it would have been a big two weeks.

But of course, I also went ice skating. It's not the first time I've been in ice skates on a rink, but it's the first time I've actually skated in those ice skates on a rink. It's fun, but it's stressful in the same way learning to drive in London is stressful, something is always veering out in front of you daring you to maim it with your beginner's ignorance.

I have also done my woefully late Christmas shopping, a great Pyrrhic victory in which I lost hours and sterling and self-respect. I wish I had a better knack of demonstrating affection through the giving of gifts.

Christmas shopping, ice skating, kick-boxing, what else? Oh yes, Barcelona. Joliette and I spent the week there with friends and it was wonderful. I got a little ill during the middle of it, which meant that we lolled about indoors as much as we strolled about outside. Even so, we saw some great things.

Sagrada Familía has been renovated since I last saw it. All of the interior scaffolding has been taken down and inside it's almost complete. To describe it would only be to belittle it. We were speechless and amazed for ten full minutes after we crossed the threshold.

There's more, of course. Over the last couple of weeks I've walked among ancient wonders, seen Herbie Hancock play live, heard beautiful and pure voices sing carols at the Royal Albert Hall, had an actual decent plate of eggs benedict in London and drank a mug of the most chocolatey hot chocolate that ever was.

Sadly, I never got to try the panther milk.

Mere Code: testtools 0.9.8 released

noreply@blogger.com (jml) — Wed, 22 Dec 2010 19:19:03 +0000

The announcement is a few days late, but I thought that you'd like to know that testtools 0.9.8 has been released.

This is one of our biggest releases, we have fixed a lot of bugs, added experimental support for running tests inside Twisted's reactor and added a stack of new matchers and convenience methods. It's well worth upgrading.

We've also got some good stuff in the pipeline, including a full re-working of our documentation, better error messages and still more matchers.

The "more matchers" thing is significant. More and more people are starting to use testtools because of the way our matchers let them build domain-specific assertions with rich, useful error messages. We continue to get contributions for basic matchers and for new ways of combining matchers. Also, projects like James Westby's soupmatchers show just how useful matchers can be. It makes me think that eventually matchers will become part of the normal way that people write tests in Python.

Thanks to Robert Collins, Martin [gz], Jelmer Vernooij, Michael Hudson-Doyle and James Westby for making this our best release ever.

Mere Code: testtools bug update

noreply@blogger.com (jml) — Fri, 10 Dec 2010 23:03:10 +0000

Hello testtools fans.

If you are wondering what all the recent bug mail was about, wonder no more, for I shall explain. Thus.

We're using only three different levels of importance for testtools bugs: Critical, Medium and Wishlist. Critical is reserved for release blockers: regressions, test failures and other disasters. Medium is for genuine defects and other things that we think are important. Wishlist is for everything else. I think of them as Critical, Important and Someday.

All of this is in preparation for the 0.9.8 release, which promises to be a doozy. Stay tuned.

Echo and Bounce: A theory

noreply@blogger.com (jml) — Wed, 01 Dec 2010 12:10:23 +0000

Anyone else notice how December has sneaked up on us this year? Normally it just stands there, proud, forbidding and distant, waiting for us to come to it. This time though, December has thrown off its gaudy robes, donned a black ski mask and starting running silently through bushes and backflipping over rooftops in order to trap us in a stealth ninja attack.

I have a theory. I don't actually think December is some kind of rogue assassin months with training in the mystical arts – that would be silly. I think that the G8 has been secretly selling off bits of October to the Chinese government.

Think about it. Everyone knows that October has thirty-one days, because otherwise there wouldn't be a Halloween, but it always feels like it should be a thirty day month. Winter came early this year. Why? Because there were actually fewer days in October to separate it from Summer. A day here, a day there, sliced off, packaged and sent to China where they use it to dissipate some of the excess energy from their over-heated economy.

All I'm saying is, don't be surprised when next year you are sitting back, feeling all smug and self-congratulatory in your August armchair and then BAM! it's mid-November and you are wondering what just happened to the year and trying to figure out when you'll find time to do your Christmas shopping.

Don't ask me how it works, I'm not a Communist.

Echo and Bounce: Lots on my mind

noreply@blogger.com (jml) — Mon, 29 Nov 2010 23:42:42 +0000

I just got back from three hours of kick-boxing, and am pretty exhausted – "cream crackered" as they say over here. It's the first time I've been in over a week, since I had a cold for much of last week.

Apart from the obvious physical demands, kick-boxing is quite demanding mentally, at least to the beginner. It's a little bit like being a beginner at driving, in that there are many things you have to remember to do correctly all of the time, there is no time at which you can forget any of them.

You have to keep your guard up at all times, fists clenched at your temples. Your fists have to be clenched properly and tightly with the thumb tucked away over the fingers. You must keep shuffling: moving around, bouncing, bobbing, weaving at all times – even when listening to instructions. But you mustn't shuffle during a combo. During the combo, pivot for reverse punches, hook punches, uppercuts, side kicks and roundhouses, but stay straight for front kicks, jabs and back hands. Ki-ai (i.e. shout, for me it's sort of a "Tsuh!" sound) at the end of your combo. Keep shuffling. Don't punch or kick too hard, just "sting" the pad / sparring partner. Return hands to guard as quickly as possible after a punch.

It's a lot to keep in mind. The nice thing is that, like driving, more and more of it becomes automatic as you keep doing it. Still, I'm a bit worried that I'll have a brain-related failure during my grading on Sunday week.

Echo and Bounce: Work-in-progress

noreply@blogger.com (jml) — Mon, 29 Nov 2010 12:44:37 +0000

I've been thinking more recently about keeping the amount of work I have in progress to a minimum. Having a small number of things on the go at once means that you actually finish those things quicker, and finishing things is great.

GTD isn't great at helping me do this, because new projects just go onto my projects list and new actions on my actions list. What I really need is a backlog, a list of projects that are queued up waiting for me to finish one of my current projects.

This is a little like the someday/maybe list of GTD, feels at least to me to be conceptually different.

Any other personal organization geeks thought about this? Any tips?

Mere Code: testtools manuals

noreply@blogger.com (jml) — Mon, 29 Nov 2010 00:31:59 +0000

I wrote a whole pile of documentation for testtools over the weekend. You can read it here.

I would very much appreciate it if you could read over the docs there and tell me if anything is unclear, wrong or otherwise unhelpful. Telling me what's good would also be appreciated.

Please leave whatever feedback you have either as comments on this post or on the merge proposal.

Echo and Bounce: Envy and equality

noreply@blogger.com (jml) — Sun, 28 Nov 2010 11:53:15 +0000

Just came across this post that I had started a while ago and never finished. I thought it was interesting enough to share.

I ~~recently~~ watched Alain de Botton talk about a gentler, kinder success, and I recommend that you do the same.

I was surprised to hear echoes of earlier writing in his arguments. Indeed, it showed remarkable discipline to be able to talk about the great role that chance plays in personal success without quoting Ecclesiastes 9:11.

De Botton also discussed envy. Envy, he argues, is largely created by a belief in equality. I think he's right, and that he's right to say it. However, I could have sworn I read a similar thing in something C. S. Lewis wrote, and it turns out I did: Screwtape Proposes a Toast. It is one of Lewis's poorer pieces where he writes about the English education system and small government and other claptrap under the guise of writing about the human spiritual condition.

Lewis's essay feels horribly dated and rather cantankerous. De Botton's talk is friendly and witty and warm. Ecclesiastes is beautiful, concise and somewhat disheartening. True things stay true, and bear repeating in new ways and new contexts.

Mere Code: Tests that print stuff

noreply@blogger.com (jml) — Fri, 26 Nov 2010 16:01:35 +0000

Tests that print stuff to stdout or stderr annoy me. They make a successful test run much less satisfying to watch, and they clutter logs when you are trying to debug a failure.

Launchpad has quite a few tests that print stuff. I was going to fix them up today, but I've got a Launchpad branch in progress and I've promised myself that I'll have only one in progress at a time. Besides, the build is broken.

Anyway, I've written a quick hack that lets you analyze a subunit stream to see which tests print stuff.

To get a subunit stream including the output itself:

$ gzip -dc testtools-experiment-r11753.subunit.gz | grep -v time: |  python tests-with-output.py

To get a list of the tests:

$ gzip -dc testtools-experiment-r11753.subunit.gz |  python tests-with-output.py  | subunit-ls --no-passthrough

For those interested, Launchpad has roughly one hundred such tests.

As with my last post, I'm left with the feeling that subunit is great but needs a lot more polish. I guess my next step is to figure out how to upstream these.

Mere Code: Boiling kettles, unit tests and data

noreply@blogger.com (jml) — Fri, 26 Nov 2010 13:34:23 +0000

The Launchpad test suite takes way too long to run: somewhere between three and four hours. Incidentally, Rob has just started some work that ought to make it run a lot faster, for which I am truly thankful. Anyway, over the last three (almost four!) years, I've watched the test suite run many, many times, waiting for it to finish or at least reveal a failure.

It has been easy to see that not all tests are created equal. Some are very slow, others are very fast. Zope's terrible layer mechanism means that these are often grouped together: slow with slow, fast with fast. Watching the test suite churn away for the umpteenth time made me wonder, exactly how are the tests distributed over time.

Luckily, thanks to subunit, a bit of Python glue, and OpenOffice it's really easy to get a picture:

Graph of how many Launchpad tests are run in each minute of the test run

That's a graph of the number of tests that complete within each minute of the Launchpad test suite run. What it means is that you are more likely to see an actual test result if you watch at a random time during the first half of the test run. It also means that we have a relatively small number of very slow tests.

Sometimes these graphs are more useful at lower granularity, so I made this one:

Graph of how many Launchpad tests are run in each ten minute block of the test run

And this one:

Graph of how many Launchpad tests are run in each hour block of the test run

The raw data is taken from a successful run of one of my recent branches. If you download it and open it up, you'll see that the subunit output includes a large number of "time:" statements. Each of these indicates that all subunit statements following it should be considered as taking place at the time given in the original statement.

I used a small Python script to generate the CSVs, running it like this:

gzip -dc testtools-experiment-r11753.subunit.gz | python test-distribution.py 60 > /home/jml/Desktop/tests-each-minute.csv

Where "60" is the number of seconds of granularity.

It was about twenty minutes work all up, and subunit made much of it dead simple. subunit has its flaws, but it's a really good idea.

That said, I can't help but feel that it should have been less work. Partly, subunit should have a way to convert data to a format more amenable to analysis by third party tools. Mostly though, those third party tools ought to exist and be known to me.

I can easily imagine a tool where I wouldn't have to run the script each time to get a different level of granularity, but rather I could use a slider in a UI and watch the graph update itself in real time. That would be cool, and relatively easy. How come nobody has done that yet?

Also, although OpenOffice's graphing thing is fairly nice, why isn't there a better tool? One that makes it easier to share graphs as images on the web without having to take crummy screenshots.

Mere Code: Big or small?

noreply@blogger.com (jml) — Wed, 24 Nov 2010 16:12:24 +0000

I've been thinking a bit about whether it is better to have one big code base that has a lot of different components and features, or whether there should be many small code bases that each do one thing well.

I don't have any answers, but perhaps these half-formed thoughts will help: positive, negative and interesting things about having many small related projects. These thoughts are mostly inspired by working on a bunch of different testing-related projects.

Positive

"Do one thing and do it well"
Enforces a certain kind of interface discipline
Avoids/postpones scaling problems with big projects

test suite run times
documentation navigation
bug triage
forking mailing lists etc.

Newcomers only need to "buy in" to one idea at a time
Aligns with conceptual understanding of the problem
Better separation of commit privs etc

Negative

Release overhead
Duplication of infrastructure

buildbot / hudson / pqm
bug tracker
mailing list

Duplication of license / copyright games
Harder for newcomers to see the big picture
Problems caused by interactions between different versions
Depending on multiple libraries is a pain on many platforms
Lag with commit privs etc

Interesting

Perhaps smaller & self-contained means easier to upstream
Some projects like adding only small dependencies, other projects like adding few dependencies

Do these make sense? What would you add?

Twisted is in a sense the opposite of the small/many paradigm, in that it includes a great deal of extra features along with its core.

Echo and Bounce: Learning to drive

noreply@blogger.com (jml) — Sun, 21 Nov 2010 13:43:04 +0000

I've been eligible to apply for a driving license for a little over a decade now, but I still haven't got one. Partly it's because of an unusual schooling, partly because I've lived in walking or public transport distance of everywhere I wanted to go, partly because I just don't get off on cars the way some do but mostly because I'm lazy.

However, driving is an important life skill, since a decade is long enough and since not having a car is a pain when I visit Tasmania, I am learning to drive.

Of course, this means learning to drive in London. Here's something of how it goes.

I wake up at a time carefully calculated to be as late as possible, but to give me enough time to consume and absorb my usual pot of coffee before the lesson starts. I leave the house and look around to discover whether today is going to be a driving in fog day or a driving in the rain day. I then turn up the mews and carefully navigate the maze of construction work (at one point vaulting a guard rail) just so my driving instructor can pick me up in a spot that's outside of the congestion zone.

As he pulls up in the middle of the busy main road just off Paddington station, I dive into the tiny car, shake his hand and we drive off to the canals some four or five minutes away. Then we swap around, I take the wheel and do my best to enjoy two of the more stressful hours of my week.

I spend those hours in a constant state of crisis management. London grew up for people, horses and carts, not automobiles, so the streets are winding and narrow. This is partly what I love about the city, but it does make learning to drive that much harder.

The roads have lanes in much the same way that Britain has a constitution: they exist; they are not written down; they are generally followed and your safest bet is to do the same thing as whoever comes before you.

A two-way street inevitably has cars parked along both sides, making it wide enough for only one care at a time. Halfway through there will be some kind of road works, just beyond which a mother will be pushing a pram out to cross the street. If I manage to dodge the road works and heroically save the child's life, a delivery van will reverse out of its parking spot to cross both of the lanes. Then there are the buses and cyclists and cab drivers who are all deliberately trying to get me to have an accident.

The real highlight is when I get to enjoy the freedom of the open road: no traffic, no obstructions, just me zooming along at a reckless thirty miles an hour (50km/h). I tell you, I feel almost giddy.

Though each week I'm getting a little bit better. I'm really hoping to be able to do the test and pass before I head to Australia this April, so I can pootle along the Southern Outlet behind the logging trucks and find my way into the comforts of Hobart.

Mere Code: "Don't Make Me Think", thoughts for Launchpad

noreply@blogger.com (jml) — Fri, 19 Nov 2010 15:29:25 +0000

I just read Don't Make Me Think, by Steve Krug. It's a good book. Sadly, I read it over a fairly long chunk of time so a lot of the punch has been spread out.

Here are my thoughts about how we can apply some of the ideas to Launchpad.

Tighten up our visual design
Launchpad doesn't look too bad, I don't think, but it could look a whole lot better. Doing that would cover over other sins, make people more comfortable and is just the right thing to do.

Error recovery & undoability
We're not great at this. You can't rename teams, you can't rename PPAs, you can't rename branches that other branches are stacked on. You can't delete bug tasks. Many forms just say "Invalid value" when you enter bad input.

Keep up with the user testing
We do a fair chunk of user testing, mostly thanks to Matthew Revell. We should do it more often, and more readily. Also, we should wave some kind of magic wand that makes it easier for developers to observe the user tests as they happen.

Home page
Make it better.

Remove more stuff
Way too often in our design discussions, we talk about adding stuff to the page: widgets, links, explanatory text. We should lean toward removing before we add. Especially with text: fewer words. Be suspicious when the solution to a usability problem is "more documentation".

Discuss things less
Sometimes, we really do talk too much about whether a thing should be one way or it should be another. Krug leads me to believe that most of things we agonize about aren't actually the things that are important to users.

Mere Code: What else have you got?

noreply@blogger.com (jml) — Thu, 18 Nov 2010 18:16:16 +0000

I recently mentioned some of the very important things we're doing on Launchpad (performance, privacy, derived archives and desktop integration) and some of the equally important things that we're still doing (making links, importing upstream translations, daily builds). We've also got a few other things on the hob.

In particular, there's some corking great work going on in making our build farm more scalable. The build farm is used for building Ubuntu, PPAs, building recipes and doing upstream translation imports, so it's a critical part of our infrastructure. We've been working to make better use of the machines we have, and make sure that we don't actually degrade in performance as we add more machines. There's also been a lot of good work here in making the code more understandable and robust.

Since we're now using the build farm for more things that just building packages, we've had to give the UI some love, particularly in making generalized build histories for the builders. This just means making builder and archive pages on Launchpad show all of the builds nicely, regardless of what type of build they are. We're kind of finished here, we just need that lousy product strategist to get around to reviewing the feature.

On the bug tracker, we're adding a new facility that will disable the automatic duplicate detection on a package-by-package basis. The dupe finder is one of my favourite parts of Launchpad, and I would never disable it for anything I do. However, the packages that take care of providing sound, graphics and networking get a lot of bugs that have the same symptoms but are actually completely different, and only an expert can tell them apart. In these cases, auto dupe detection does more harm than good, so we're providing a way to turn it off.

Perhaps most excitingly of all the "non-core" work, we're working on giving you far more control over your bug mail. Launchpad has long been known for sending way too much email. We're changing that.

There's also some great stuff being done outside the Launchpad team:

DKIM-based mail authentication (aka "Change bug statuses from GMail")
Pre-forking lp-serve (aka "Shave two seconds off every bzr push to Launchpad")
Bugzilla components (aka "One step closer to smooth upstreaming of bugs")
Branch merge queues

And that's all of the stuff that's going on in Launchpad, as far as I know. Actually, there's a fair bit of work going on to make it so we can deploy Launchpad way more often and work on features for as long as we want without exposing them to unsuspecting users. And we're also working on simplifying our branch landing machinery. But that's it.

As you can see, there are a lot of things. That means that progress on any one thing is much slower than anyone would like. However, progress is being made. Give us a few months and we might be working on only three things at once, rather than over a dozen.

As always, if you're keen on following Launchpad development, you can join #launchpad-dev on Freenode, or our developer mailing list.

Coming soon: ideas for the not-so-distant future.

Edit: Oops! Forgot merge queues.

Mere Code: Reviewing specs, rock on!

noreply@blogger.com (jml) — Thu, 18 Nov 2010 18:14:40 +0000

I just finished reviewing a bunch of LEPs, which are the specs Launchpadders write for doing new feature work. I've been looking at a bunch of the privacy LEPs and one on merge queues. It's been great to see such good, clear thinking, and also to see people thinking and writing in terms of constraints and requirements ("What do we want? Why?") instead of implementation or potential solutions.

Basically I've got a bunch of scribbled-on bits of paper right now. Tomorrow I'm going to update the wiki & schedule calls with the developers so we can kick them along.

Echo and Bounce: Things difficult and easy about kickboxing

noreply@blogger.com (jml) — Tue, 16 Nov 2010 18:36:02 +0000

A few months ago I started kickboxing in London (terrible website, great people). I go to a great school that meets in various church halls and gyms around central London. It's not boxxercising or tai-bo or whatever, but actual kicking-people-in-the-head kickboxing.

Of course, I don't actually kick people in the head. Unless they are really short. And our school is "semi-contact" – we hit people as lightly as we can get away with.

Some things are harder than I expected, surprisingly so:

bouncing around on the spot ("keep shuffling!") while keeping guard up
kicking; it's hard work!
remembering whether to attack or defend during sparring drills
fitting all your gear into one bag
remembering to put your mouth guard in before putting on gloves
doing your ki-ai through a mouth guard
untangling hand wraps that have just been through the washing machine

Here's what has been surprisingly easy:

getting into the training session and forgetting about my day
actually keeping it up

I've been genuinely surprised by how much I'm enjoying it and how friendly the teachers and other students are. I'm aiming for my first belt in December, and have strange mix of fear, excitement and determination churning around in me.

I'll keep you posted.

PS. Apologies for the long hiatus. Hope I'm back now.

Mere Code: Having an extra feature never hurts, rebutted

noreply@blogger.com (jml) — Tue, 16 Nov 2010 18:10:14 +0000

On the Launchpad user list, we're having a bit of a chat about whether or not to remove team polls. One commenter suggested that "having an extra feature never hurts". Curtis's reply is so good I want to quote it here in full:

Actually, It always hurts. It is always bad:

     1. The bad UI experience reflects poorly on all Launchpad.
     2. A user tried to mess with URLs to avoid the UI and caused an
        oops. I spent a couple days fixing an issue that I personally do
        not think should ever have been a priority. Certainly there are
        at least a 1000 more important bugs than the oops, but we
        believe oopses are always high priority.
     3. Pages are slower to load because the application looks for poll
        data.
     4. We must update the code as libraries and infrastructure changes.
     5. We need to triage and retriage all the poll bugs.
     6. We need to answer user questions on Lp Answers and IRC.

It is better that Launchpad provides 5 really great features than to
provide 10 mediocre features. I prefer to remove the feature, but if the
community believes that it is important, we might work on this in the
upcoming Launchpad bugjam to close as many bugs as possible in a week. I
cannot foresee this feature being compelling in the net two years
without a contributor dedicating time to it.

The situation is eloquently stated by Antoine de Saint-Exupéry

        A designer knows he has achieved perfection not when there is
        nothing left to add, but when there is nothing left to take
        away.

That is all.

Mere Code: And then what?

noreply@blogger.com (jml) — Tue, 16 Nov 2010 15:03:34 +0000

By now, you know where Launchpad is at. You know what our top priorities are, you know the other important stuff that we're still working on, you know the other bits and pieces we've got going. You might even know why.

What you might not know, and what this post hopes to tell you, is what we are thinking about right now. Humans are plan-making creatures, and product strategists perhaps more so than most. What follows is a brief description of the half-formed thoughts and plans that I have for Launchpad. (Also, have I mentioned the roadmap yet?)

Strategically speaking, there are two major approaches we can take to our work on Launchpad. The first is working on things that bring new users and new projects to us. This would mean finding something that we can provide that alternatives don't and potential users want, or assuming that we already have such a thing, find the blockers that stop people from using Launchpad and remove them (the blockers, not the people).

For example, if we thought that open source developers wanted their hosting site, say, to be part of the semantic web, we could go and do that and be relatively confident that no one else will. That would be finding a competitive advantage.

Or, to take another example, we could say that our awesome bug tracker, our translation support and daily builds are heaps plenty good advantages, but people don't use Launchpad because we only have full support for one VCS and we provide no place to host a project shop-front web site. That would mean adding Git, Subversion and Mercurial support, and adding some kind of wiki / webhosting. Personally, I find this much more appealing than semantic web shenanigans.

All of that's one strategic approach: attract new users. There's another major approach that we could take: make things better for our current users. All things considered, I think I want to take the second approach.

There's so much we could do here, partly because Launchpad itself is so broad. Some key things stand out though.

First, there is a cluster of problems related to the way that people approach the website. Although we tout cross-project collaboration as one of our big things, we don't make it particularly easy. We need to provide each user and each team with a view that shows them all of the things that are interesting to them across all of Launchpad. In fact, we probably need to give them two views: one that's historical and chronological, and another that's forward looking and more dashboard-like. In my head, I've been thinking of this as walls (ala Facebook) and dashboards.

If we dig into this, there'll be other changes: better, fairer karma; more social interaction; better person, project and team pages; a "waiting for" section in the dashboard so you can see who to hassle; a better experience for new users; and probably death to the home page for users who've got accounts. Lots of goodness here.

Second, it's becoming increasingly clear that blueprints are a good idea with a sub-standard implementation. Persuaded largely by Matthew Paul Thomas, I think that we should merge the blueprint tracker with the bug tracker to make a combined issue tracker. A lot of people get worried when I say this, because they really like blueprints and they understand that bugs and blueprints are actually two different things. Please don't worry. There will almost certainly always be something in Launchpad like a blueprint or specification that is visibly different from a bug. However, I think we ought to change what lists they appear in, what statuses they have and many other things. The loss of distinctiveness will be more than made up for with an increase in capability.

Third, the way we search and the way we display results of searches needs to be revamped to be faster and more flexible. I want to see easy forms, a good search string mini-language, sexy URLs, searches that can be saved and shared.

There's so much more that we could do, that we ought to do: make mailing lists rock; inline commenting on merge proposals; eliminate all of the silly "refresh this page" business that we still have; go to series and milestones with a sharp axe and a clear conscience; provide top-notch tools for QA teams; show way more cool stuff based on our data; make filing bugs upstream a complete no-brainer; clean up the terminology we use for naming things; provide an event sending interface for launchpadlib so that people don't have to poll. I could go on for a very long time.

I call out the three things above – cross-project views, merging bugs and blueprints and better search – because I think they are things that will make Launchpad better for absolutely everyone who uses it. In particular, done right, good cross-project views turns a weakness into a strength. It takes the vast, daunting size of Launchpad and instead turns it into a huge realm of opportunity.

Now, none of this is fixed in concrete. There's still plenty of time to discuss, plan out, change our minds and what not. But it's what's in my head right now.

That's it. Right now, we're working on privacy, performance, derived archives and desktop integration as our top priorities. We're still plugging away at daily builds and making links between distributions and upstreams, and we're working on a host of useful, important things. In the future, if my plans go as … planned, then we'll have dashboards, activity walls, a combined issue tracker and excellent search.

What do you think?

Mere Code: Make it really easy to fix bugs on Ubuntu

noreply@blogger.com (jml) — Mon, 15 Nov 2010 14:36:53 +0000

One of the best things that anyone ever said was, "not enough gets said about the importance of abandoning crap". Mr Glass was probably talking about writing, but his words could well have been aimed squarely at any well-established software development process.

Right now, it's too hard to fix a bug in Ubuntu. There are a lot of things that we can do to make it easier, let me tell you about mine.

If you see a small bug in a program on your desktop, you want to something like this:

Figure out what package it's in
Look for an existing bug, if not file one
Get the source
Build it, try to reproduce the bug
Actually fix the bug
Test to see that it's actually fixed
Submit it then shepherd it through whatever processes come next

Much of this is monkey work if you know how to do it, or incomprehensible arcana if you don't. Either way, it's important that we abandon (or automate) this essentially uninteresting work.

In that spirit, at UDS I wrote a script that a lot of us have been talking about for a long time. I called it start-hacking, because I am terrible at names and wanted to make a project. Right now, when you run the script on your Ubuntu desktop, your cursor becomes a cross-hair. When you click on an application, start-hacking will tell you the source package that the application belongs to and where you can get the source (both Ubuntu source and latest upstream if available). It's very simple, but that's because it builds on a very large amount of work that Canonical has done, mostly through Launchpad and Apport.

Eventually, we want to make the script a much more beautiful application that gets you a built, runnable copy of any Ubuntu application or library in a test environment. Our goal is to make fixing a typo (or typo-level bug) as fast and as smooth as possible for any application on the desktop.

All we need is a bit of time and a few volunteers.

Mere Code: Ubuntu in a VM on OS X?

noreply@blogger.com (jml) — Sat, 06 Nov 2010 11:28:52 +0000

I'm trying to get Ubuntu 10.10 running in a VM on my iMac. It mostly works with either VirtualBox or VMWare Fusion, but I can't get 3D acceleration working. Under VirtualBox, I can get compositing happening, but it's very slow. Under VMWare Fusion, Ubuntu won't even acknowledge that I have a 3D card. Help.

Edit: If you can point me to a working guide for dual booting Ubuntu on the iMac 27", then that would also be great.

Mere Code: Still going

noreply@blogger.com (jml) — Thu, 04 Nov 2010 17:24:19 +0000

Last post, I talked about the top priorities for Launchpad development right now: performance, privacy, derivative distributions and desktop integration. In this post, I want to talk about the other things that we are doing, and why we are doing them at all.

In September last year, we decided that we needed to put a lot of work into something that is at the core of the very idea of Launchpad: bridging the gap between Linux distributions (specifically Ubuntu) and upstream open source projects. Launchpad has been intended from the very beginning to smooth out and accelerate collaboration between these two areas of endeavour.

Much of that work has already been done, but we have three initiatives still going: making links, daily builds and importing upstream translations.

Making links is all about making it incredibly to add useful, usable information about package / upstream relationships. Launchpad has always had the ability to store such information, but now it's much, much easier to use because of a whole suite of changes: automatic suggestions for upstream / distro links; simplified project configuration; automatic linking to projects when marking a bug as upstream; just-in-time project registration etc. There are a few more bugs to go before we're happy to call it done.

Daily builds are a very cool feature that's still in beta. You provide us with a "recipe" that tells Launchpad how to combine branches into something resembling a source package, then we take that recipe, assemble the source package, build it and then publish it into a PPA of your choice. Then, if the branches change, we'll provide a new build each day. The feature mostly works right now – Project Neon are using it to get nightly builds of Amarok – but the UI needs polish, there are a few system-level glitches and we're asking questions about the recipe format. I'll be posting later about how to get involved with the beta.

Importing upstream translations into Ubuntu makes it much easier for Ubuntu to be the best-translated Linux distribution ever. Right now it's already possible to import translations from upstream projects into Launchpad. Soon those translations will be imported directly into the equivalent Ubuntu packages too, which means that Ubuntu translators can choose to use the upstream translation or instead provide a better one. Before we can do that, we'll need to do some scalability work. After that's done, we'll also want to do some UI work to make it very easy to connect the Ubuntu package translations to the imported upstream translations.

All in all, three very neat initiatives to bring Ubuntu and its upstreams closer together.

So that's it right? What with performance, derivative distributions, privacy and desktop integration as well as making links, daily builds and importing upstream translations, the Launchpad team must have its plate full? Indeed not! Stay tuned, and I'll let you know what other things we are currently working on, as well as the future plans that are bubbling away.

Mere Code: What to do, what to do

noreply@blogger.com (jml) — Wed, 03 Nov 2010 09:03:05 +0000

One of the many things I love about going to UDS is the opportunity to talk to so many different people about a relatively small set of things. Having the same conversation over again from various angles helps jostle a few lazy thoughts into rank, allowing the now ordered thoughts to execute a disciplined march on the bastion of uncertainty. That is, explaining what I'm thinking makes my thinking clearer. Rather selfishly, I'd like to start explaining here what I've been thinking recently.

The most important things for Launchpad development right now are performance, derivative distributions, privacy and desktop integration. Everything else we want to do is less important than any of these. I used to include software center support, but we've done that already, all the others are in progress.

A faster and more snappy Launchpad will make the lives of all of our existing users happy, whether they are dentists hacking in their spare time or field engineers in Taipei. Improving performance makes the user experience better by having less of it.

Ubuntu is being shipped by default on more and more devices. Each version of Ubuntu on those devices can quite legitimately be thought of as a derivative distribution. We want to give our hardware enablement teams the same top-notch tools that our platform teams get to use. Also in the highly competitive world of device manufacturing, many things need to be kept secret, so Launchpad needs to get its privacy act together.

Finally, I'm convinced that developing apps and fixing bugs in Ubuntu needs to be as easy and smooth as possible. I'm also convinced, alongside the Ubuntu One folk, that we can do some pretty neat things when we integrate the desktop and the web. Thus I've been pushing for a long time to improve Launchpad's desktop integration. Unlike the others it's not an immediate need, but I think the work we do now will pay off very quickly.

All of these four things are in progress right now. I'd love to write more about some of the other things we have in progress and about my own dim picture of the future, but this is enough for one post.

Echo and Bounce: Inboxen

noreply@blogger.com (jml) — Tue, 02 Nov 2010 15:16:43 +0000

I have a bunch of places where all of the input into my life gets funneled:

My Gmail inbox
My physical in-tray
Files on my desktop
Files in "Downloads"
My Remember the Milk inbox

Every day I empty every one of these.

Mere Code: Launchpad and UDS-N

noreply@blogger.com (jml) — Tue, 02 Nov 2010 14:23:01 +0000

Yesterday I got back from the Ubuntu Developer Summit where hundreds of people got together to plan the next release of Ubuntu, the "Natty Narwhal".

For Natty, the Ubuntu Platform team have five foci:

Unity
2D Experience
Software Center
Testing
Community contributions

If you have a problem with these, talk to the hand.

We on the Launchpad team want to do everything we can to help Ubuntu achieve these goals. Partly, this will be by continuing to make Launchpad a joy to use. However we'll probably be able to offer some specific help with the last three foci: Software Center; testing and community contributions.

I don't know of any concrete actions yet – I've yet to trawl through the many blueprints to find them – but for the next few months, I'm hoping that Launchpad developers will be watching for opportunities to help make Natty the best release ever.

Mere Code: The art is long, the life short

noreply@blogger.com (jml) — Thu, 28 Oct 2010 19:41:09 +0000

I love the energy at UDS. I wish I could go away from it and sit down and do nothing but hack for three months so I could finish all of the little, awesome things that Launchpad could do.

Mere Code: testtools compatibility

noreply@blogger.com (jml) — Mon, 25 Oct 2010 11:37:55 +0000

We've recently been getting a few bugs filed on testtools about compatibility with various versions of Python. I really want to support as many versions as I can, but I simply don't have them on my system.

What resources are available to me if I want to set up a Hudson instance or similar that runs testtools tests against Python 2.4, 2.5, 2.6, 2.7, 3.0 and 3.1? A Windows run would also be nice.