I have been told that not enough information about the Plone user interface sprint at Informaat is getting out. That is not due to a non-disclosure agreement for attendees; all of us have just been too busy working and enjoying ourselves to get to such mundane things as writing reports or blog posts.

This sprint is somewhat different than normal sprints: the goal is not to produce as much code as possible, but to lay a foundation for the user interface for future versions of Plone. This requires a lot of thought and prototyping to figure out the right concepts, both in the user interface itself and the backend code on top of which it is built. A lot of this work is done on top of the work done by the little known deco group.

There are a few guiding ideas behind the work done at this sprint:

• separate all the knobs and toggles related to content editing and page manipulation out from the normal view.
• there are three different modi of interactiving with a page: view, edit, and construction (modifying layout, managing tiles). These require different types of user interface and should be handled separately.
• make the interface as simple as possible. If possible try to use a progressive complexity approach: keep things as simple as possible, and only expose more advanced options once users are ready to use them.
• always try to use simple semantic markup. There is very rarely a need to add extra elements of pollute the markup with layout or positioning-related information; that should all be done in CSS.

We divided the activities in five areas: content editing, control panels, page assembly, backend and tests.

The first day was almost entirely spent on brainstorming, which is very rare for a Plone sprint. This was extremely useful, and provides us with a solid foundation for the rest of the sprint, and ongoing improvement of the Plone user interface.

If you ever used TinyMCE and jQuery in a project chances are that you have seen this error:

tinyMCE is not defined

This might be browser dependent: I see this in Firefox, but Safari has no problems at all. At times this can be bad enough that Firefox will block loading the page, refuses to reload it and needs to be restarted. Doing a few searches revealed that others have noticed the same problem, but I could not find an answer. It turns out the solution is simple: you have to scope your javascript properly. This breaks:

 $(document).ready(function() {
    tinyMCE.init(....);
}

but this works fine:

(function($) {
    tinyMCE.init(....);
})(jQuery);

After a very long period of not working on it I finally got around to looking at pyrad again. Even though I do not use it much anymore myself it is still being used by a lot of people and I get regular feature requests for it, so it certainly deserved a bit of love.

My approach to software projects has changed a bit since I started with pyrad and one difference is that these days I write tests for all code. Tests help a lot in making sure your code does what it is supposed to do and any change you make will not have unexpected side-effects. So I sat down and wrote tests for pyrad. A lot of them and together they now cover 100% of the pyrad code. While writing the tests I found a few APIs problems and one real bug. Considering that so far there were no tests at all I am very happy with how stable the code proved to be.

The other visible change is that pyrad is not also distributed as a python egg, making it possible to install it using the standard python setuptools (easy_install is your friend) and it is listed in the python package.

After those changes pyrad finally deserved a 1.0 release. And here it is

With the recent release of Debian etch (usability note: why can't that URL be per-release? All the info on sarge suddenly is not reachable at the location people bookmarked anymore) I figured I would try upgrading some servers to it. Debian has always had excellent in-place migration handling, so this should be easy, right? If only..

• Post-upgrade courier IMAP no longer worked. Or to be more specific: IMAP worked, but just did not find my mailboxes anymore. And no warnings, errors or anything else in the logfiles. Some hand-merging of configuration files and a restart fixed that.
• Apache2 LDAP authentication has changed with the upgrade to Apache 2.2, which means it broke for all my sites. Apache now uses quiet different configuration options to configure LDAP so I can not really blame Debian for it. But still, it's all part of the Debian upgrade experience.
• After the upgrade my apache was suddenly doing disk-caching of all sites, leading to various strange problems with outdated data being served, which broke several applications. There was no warning during the upgrade process for this change.
• I use iTerm as terminal emulator. After upgrading a server terminal emulation was suddenly horrible: lines would run too long, wraping them to the next line in the terminal, scrolling would not work and other general madness. I ended up copying /usr/share/terminfo from a Debian sarge machine to /etc/terminfo on the etch machines to fix this.
• And now for the clincher: you can no longer use python 2.3 with edge. To ascertain this the python package now conflicts with all versions of the python2.3 package, except for those in unstable. Since I run a fair amount of Zope instances which use python2.3 that means that I simply can not upgrade machines to etch. I am not the only one to notice that: I browsed the debian-python archives and noticed that other people have brought that up on that, but nothing has changed since.

I will stay with sarge for a while longer.

If you work on open source projects and hang out on irc changes are you are familiar with CIA. CIA provides a convenient way to stay in touch with what is happening in a source code repository by reporting on all activity real time.

Of course there is much more to a project than just source code changes: issue tracking is just at least as important, if not more. So wouldn't it be nice if CIA can report on activity there as well? If you are using trac now you can, with this patch: trac-cia.diff This adds a couple of new option to your trac.ini file which confifure the trac-CIA interface:

[notification]
send_to_cia = true
cia_project = YourCiaProjectName
cia_server = http://cia.navi.cx

This patch tells trac to report all newly opened tickets to CIA, which will show up like this:

<CIA-13> wichert * r #5725 Issue tracker/: [Infrastructure] trac/cia
	 testing ticket

It has been argued that it makes sense to use a seperate ZODB to store a catalog for Zope sites. So far I have not been able to find any benchmarks that substantiate this claim.

Without doing any benchmarks I can think of four reasons why this makes sense:

• Packing: a catalog has no need for history for its contents, so you can pack it without keeping any history. For normal content having some history available can be very useful.
• Optimization: catalog objects can be much smaller than normal objects so you can cache a larger number of them.
• Performance: the catalog gets updates a lot more (due to things like Archetypes doing multiple reindexing when updating objects) which can stall access to the main data.
• Stability: with Zope 2.7 you could easily get a read conflict error when reading while someone is writing data. Using multiple ZODBs reduces the chances of that happening. With more recent Zope (ZODB really) releases this is not an issue anymore though.

It would be nice if someone can extend this list and do benchmarks proving that this ZODB split indeed helps (or that it does not in which case a lot of us can simplify our setups).

Update: alecm mentioned a very good reason: catalog searches, which can wake up a large amount of small objects, should not cause other objects to be pushed out of the cache.

As many other people, I have become addicted to the trac system. After having used it for a few personal projects and deciding that it really is cool technology and talking about it with a few other people I switched Plone from its old issue tracker to trac and the result has been marvelous: it runs smoothly with over 5000 tickets and produces great overviews such as this 2.1.x release issue status overview.

However for our projects at Amaze we are missing an essential feature: the ability to have a testing status for issues. This is a seperate step between active and resolved where an issue should be resolved, but has not been certified to be resolved by full testing, which is (should) be done by other people.

To support this I created a patch which added a new ticket status called 'testing', modified the milestone and roadmap to show that tickets in a testing status seperately and updated the unittests to reflect the new possible actions.

In order to make this easier to use I also modified the trac-post-commit subversion hook: I added a --testing parameter which tells the hook to mark an issue as testing instead of fixed. The default is set to fixed in order to preserve backwards compability, and since I suspect that most trac installs are for small projects which do not have a need for a seperate testing status.

If you are curious: you can download trac testing patch from here or look at trac ticket 2511. The patch is based on the current svn trunk for trac.

Update: seems the trac folks already closed the issue since there is a larged project in progress which provides the same functionality (and more). However since there is no assigned milestone or estimate when it might be merged, so until then I still consider this patch to be a very useful addition to trac.

It's that time of year again: the anual 3 october festivities in Leiden, celebrating the end of the Spanish siege in 1574.

Since I happen to life in the centre of town I setup a webcam again.

Thanks to Telegraaf Media ICT Debian got a new server today: tartini. It will be used to reduce the load of klecker a bit and make sure we always have a standby machine in case one of the machines breaks down.

Alioth meanwhile is doing reasonably fine; new hardware has finally been ordered and should arrive in October. Unfortunately account syncing for Debian accounts has been broken since July 21, so new Debian developers will not get there an account. Debian-admin has been notified three times in the last three weeks so with some luck it will start working again this month.

And in more server news: all arrangement for a new server for plone.org are close to being finalized. It is just a matter of a little paperwork and getting the physical machine now.

I finally entered this decade and looked a bit at bittorrent. Being a big fan of tools to make my life simpler I found TorrentFlux, which provides a nice web interface in which I can drop torrent files and forget about them, knowing the system will take care of downloading them, seeding others for a bit and then shutting down.

Only downside was that it had a hard dependency on mysql, while my systems all use postgres and I do not want to run two database servers. Luckily fixing that was quite simple, so here is a patch adding postgres support to TorrentFlux.

Working with serial ports always makes me feel like gone several decades back in time.

Following a reasonably massive systems upgrade at work a fresh release of python-dhm which I already used there. Besides the usual bugfixes this includes a event module which implements a C# event type as well as reworked SQL wrappers with a more pythonesque interface.

New release of sict. It now has the abilite to merge configuration data from different sources which makes it trivial to support multiple config files and do things like type checking or default values without having to hardcode them.