stick's corner

Last weekend I attended FOSDEM 2010 and it was a blast! I’ve never seen such a high number of tracks on a conference. Kudos to the organization team! During these two days I was sending a short messages via identica and twitter, because as @amedee said “the crew provided a fiber optic network, so people can compress their thoughts in char[140] blurbs”. I still decided to write a short report, because not everything could fit into these blurbs and not everyone uses these microblogging services. So here we go!

Saturday

The first keynote I attended was the “Evil on the Internet” by Richard Clayton. The Janson room totally crowded as can be seen on the photo below. I expect there were around 1200 people in the audience. Richard spoke about various tricks how to identify scammers and one of the interesting points was that you can use Google Street View to check scammers’ (often fake) addresses.

After the keynote I went to see KDE SC 4.4 demo by Jos Poortvliet. It turned out to be a good choice, because I finally learned why the “Rotate widget” feature is useful. Imagine a multitouch-table with 8 people around, each working with his/hers own set of widgets. Pretty cool idea!

I was interested how Maemo and Fedora manage their communities, so I attended the respective talks in the Distributions track. Maemo uses karma to measure the activity of its community members. They have 6 masters who take care of their field related issues and 5 members of community council (see http://wiki.maemo.org/Maemo.org_team team page). Max Spevack of Fedora surprised me that he had no prepared slides, but he is a good speaker so the talk was still very good. He spoke about Fedora governance “mountain” which is: Individual, Regional, SIG, Project, Board, Project Leader. One good thing about SIGs is that they can miserably without bad impact on the distribution as a whole (but they tend to be wildly successful). In the end Max recommended us reading The Starfish and the Spider book.

Then I went to see the Ruby+Rails devroom. More than 70% of people had MacBooks there, but this could be expected. Nicolas Jacobeus gave us 25 tips for Ruby and Rails development and Francois Stephany told us about how Ruby is still being inspired by Smalltalk even today and used pretty funny examples to demonstrate it:

I wanted to see the last 2 XMPP talks, but their devroom was desperately full, so I went back to our stand to meet the rest of the openSUSE gang and have dinner with them.

Sunday

Second day had even more visitors and most of the smaller rooms were full. I couldn’t get to Miguel’s talk about Mono Edge, so I went to see NixOS talk. They use very interesting package management and configuration storage. See bottom of this page for more info.

After that it was my turn to give a talk on RPM packaging collaboration. It went quite well although the battery in my microphone died, so the recording will be probably fubared. I got valuable feedback from Fedora and Mandriva folks, even from Jeff Johnson. Let’s see if it raises the level of discussions on rpm.org wiki, mailinglists and IRC. The slides are already available from my Projects page.

I was finally able to make it into the Mono room where Miguel shortly presented the Pinta paint editor and Alp showed us Moonlight player which used fully-managed Theora codec to play the movie. These demos were followed by series of in-browser and desktop Moonlight demonstrations by Andrea Gaita.

Then I rushed to see Evan presenting his StatusNet project (you might know it under ther former name laconi.ca), but I was able to catch only Q&A at the end. Shortly after GregKH appeared on stage and gave very funny (as usual) guide how to contribute to Linux kernel. He also talked about the coding style and gave a perfect explanation why to care about it (of course, not only in kernel, but generally): If you have a coding style, code patterns will start to emerge and you are able to see the “metadata”.

This was the last FOSDEM talk and all I had in front of me was a looooong travel home, but definitively worth it!

(Twitpic photos by: @jaom7, @Cimm, @vyruss).

… and I’m looking forward to meeting you all! I will also give a talk about RPM Packaging Collaboration so remember to show up on Sunday if you are interested!

If you are closely following Ruby development and especially the situation around ruby gems, you might already know of Gemcutter. It is a new service, which provides a very easy way how to publish gems and also a good API to deal with them. It is not trying to replace RubyForge as whole, just its gem hosting (+ now defunct GitHub gem hosting) and will soon become the central and the only place for Ruby gems. The whole site is MIT licensed and the code is available on GitHub.

During the winter holidays I wrote a simple script which utilizes the Gemcutter API and prints versions of rubygem-* packages in our devel:languages:ruby:extensions Build Service repository compared with the corresponding gem versions on Gemcutter. Using this script and a great gem2rpm (more particularly gem2rpm-opensuse command which applies openSUSE template and is available from rubygem-gem2rpm package), I was able to update nearly a hundred of gems in just two hours. Rails rubygems have a specific packaging in openSUSE, so I left them out, but more than 90% of the rest didn’t need any changes in autogenerated spec file.

This brought me an idea. If only Gemcutter had an option to somehow send out notification that a new gem has been pushed, we could automate the process and have up-to-date rubygems in our devel:languages:ruby:extensions repository almost instantly. (We would still need to keep the list of “dirty” rubygems that need to be updated manually, though. For example, Rails packages I mentioned earlier, where we keep multiple versions, or others where we need to add a patch replacing /usr/local/bin/ruby with /usr/bin/ruby in scripts).

Few days later, Gemcutter gained RSS feed support, but only for the gems one is interested in. I didn’t find the option to have RSS feed for all gems. This could have helped in creating such mechanism, but that won’t be needed anymore because …

… yesterday Nick Quaranto of Gemcutter announced webhook support. I’m really excited, because that’s exactly what we need! When one registers a webhook, Gemcutter emits a POST request on a certain URL when a gem is pushed or updated. This request is a JSON document containing the info about gem. What we need is to create a mechanism that:

• receives notification via POST JSON request
• checks whether the package is not “dirty” → exit if it is (and probably send some email …)
• fetches the package from the Build Service or create a new one
• fetches the new gem, removes the old one
• runs gem2rpm-opensuse to create a spec file replacing the old one
• adds changelog entry
• pushes the updated package back into the Build Service

Last but not least: If Fedora and Mandriva had gem2rpm templates in a perfect shape too, Build Service could provide packaged gems also for their distributions.

So what do you think? Any volunteers for this? Right now, I’m off to fix some small bugs I found in gem2rpm while fiddling with it …

I’m sure you all heard about the openSUSE Conference 2009 that took place in September in Nuremberg. Not so many know about the RPM Summit that was a part of the conference during its first two days. Idea to create something like this started at LinuxTag 2009 when Zonker invited Florian to Nuremberg. One thing lead to another and in the end we managed to get several important people into one room (or to join us via conf call) and we kept them there until all problems with RPM and package management in general were solved. Well, not really, there was a lot to discuss and there still is, but you got the point.

And who were our victims^Wexperts? Here’s the list:

• Michael Schroeder, SUSE RPM maintainer (Novell)
• Florian Festi, RPM upstream developer (Red Hat)
• Panu Matilainen, RPM upstream developer (Red Hat)
• Klaus Kaempf and Stanislav Visnovsky, SUSE system management folks (including software management, e.g. libzypp) (Novell)
• Seth Vidal and James Antill of the YUM fame (Red Hat)

We were also joined by others interested in RPM development: Ludwig Nussel, Olaf Dabrunz and Jan Engelhardt. Most of the time we were working through a quite long list of issues we prepared in advance in the wiki and it turned out that there is not much difference in the view on the different topics. I tried to summarize the summit in the following 10 points:

1. Virtual triggers

   Triggers on virtual symbols (provides) do not work at the moment. This is considered a bug (or missing feature) and will be fixed.

2. File triggers

   Panu is working on file triggers feature. This will help packagers to get rid of a large number of scriptlets (e.g. %post/%postun -p /sbin/ldconfig) in spec files, but he wants to be sure that it doesn’t break anything else. File triggers are already used by Mandriva, but Panu wants to implement them in a different fashion. (See Mandriva wiki for the details about their implementation).

3. Soft dependencies

   Soft dependencies keywords that are already used by SUSE (Recommends, Suggests, Supplements, Enhances) will be recognized in the future versions of RPM. RPM will not do anything with them except of storing in the database and reporting to higher levels of package management stack. There is an ongoing discussion whether and how to implement soft dependencies in YUM (they are already implemented in the zypp stack).

4. Update scriptlets

   We agreed to introduce two new scriptlets called %preup and %postup. These will be called when updating the package (%preun, %postun, %pre and %post will not be run in this case). This is a way how to fix the broken uninstall scriptlets and one doesn’t have to do the weird if [ "$1" = "0" ]; stuff in the scriptlets anymore to detect whether the operation is upgrade or uninstall.

5. Scriptlets arguments

   Package scriptlets will get more information (e.g. NEVRA of the packages) about currently run transaction in environment variables. This will make it easier for scriptlets to handle special situations or to detect more precisely what is happening. (One could for example convert some config files when upgrading from package of version 1.x to 2.x and to leave them intact otherwise).

6. DeltaRPM

   DeltaRPM sources will stay in gitorious at the moment. They might appear in RPM upstream git repository later in the future but there is no reasoning for this right now. Jindrich Novy is working on a new diff algorithm, we’ll see if it will be merged into DeltaRPM or replace it completely.

7. New payload format

   Currently used format is CPIO but it has a filesize limit of 8 GiB. This is not enough when one tries to distribute very large files, for example appliance images, packaged as RPMs. Developers will need to find a way how to enhance CPIO or try another format soon. TAR is really not an option …

8. Tilde in version

   The special character tilde (~) will be available for use in version representing a negative version token. This will simplify complicated rules which abuse the version and the release tags. (For example, using foo-2.5.99.2 instead of the foo-2.6~beta2). There already exists a patch but it needs more test coverage.

9. Easy way to add or remove autogenerated dependencies

   This feature is wanted but is not implemented yet. Currently this is workarounded by redefining the __find_provides/__find_requires/… in spec file. Example from the open-vm-tools package:

   # exclude AMD PCnet32 LANCE pci.id from Supplements list [bnc#397554]
   %define __find_supplements sh -c '/usr/lib/rpm/find-supplements %{name} | grep -v pci:v00001022d00002000'

10. Logging and recovery

    There is a plan to include a transaction log to RPM which will allow recovery after a broken transaction. RPM log could also replace similar features in YUM/zypper and will be more reliable as it would also work from rpm cli.

That’s it! I had a great time during the conference and was really happy to see the synergy between the developers, even though they are working for two competing companies. In case you have any questions feel free to join us on Freenode IRC channel #rpm.org or use the appropriate mailing list.