stick's corner

Yesterday, we needed with darix to obtain a dependency graph of the package you are about to install. I knew that something similar was planned in zypper, so I went to Jano Kupec to check the status of it. Unfortunately, I learned that this feature is not implemented yet. I think it should not be very hard to enhance the zypper package list with some eye-candy, but I haven’t looked into it yet. I would love to have these outputs similar to Gentoo ones (colors and simple ASCII art dependency trees). Btw, zypper already has color support, so if you want to start hacking, there is source code in gitorious.

Jano also showed me a neat trick how to obtain the results we wanted anyway. If you are in similar situation, just follow these steps (they are of course not suitable for everyday use, but still better than nothing):

1. install package libzypp-testsuite-tools
2. run zypper install --debug-solver pkg
3. cd into /var/log/zypper.solverTestCase
4. open solver-test.xml in your favorite editor
5. add <graphic/> tag just above the </trial> closing tag
6. run /usr/lib/zypp/testsuite/bin/deptestomatic.multi solver-test.xml (as normal user, you won’t get any graph when running as root)
7. you can pan the graph, rotate it with the right click or even save it to disk!

The resulting image can be seen here:

I’m sure you all heard about the openSUSE Conference 2009 that took place in September in Nuremberg. Not so many know about the RPM Summit that was a part of the conference during its first two days. Idea to create something like this started at LinuxTag 2009 when Zonker invited Florian to Nuremberg. One thing lead to another and in the end we managed to get several important people into one room (or to join us via conf call) and we kept them there until all problems with RPM and package management in general were solved. Well, not really, there was a lot to discuss and there still is, but you got the point.

And who were our victims^Wexperts? Here’s the list:

• Michael Schroeder, SUSE RPM maintainer (Novell)
• Florian Festi, RPM upstream developer (Red Hat)
• Panu Matilainen, RPM upstream developer (Red Hat)
• Klaus Kaempf and Stanislav Visnovsky, SUSE system management folks (including software management, e.g. libzypp) (Novell)
• Seth Vidal and James Antill of the YUM fame (Red Hat)

We were also joined by others interested in RPM development: Ludwig Nussel, Olaf Dabrunz and Jan Engelhardt. Most of the time we were working through a quite long list of issues we prepared in advance in the wiki and it turned out that there is not much difference in the view on the different topics. I tried to summarize the summit in the following 10 points:

1. Virtual triggers

   Triggers on virtual symbols (provides) do not work at the moment. This is considered a bug (or missing feature) and will be fixed.

2. File triggers

   Panu is working on file triggers feature. This will help packagers to get rid of a large number of scriptlets (e.g. %post/%postun -p /sbin/ldconfig) in spec files, but he wants to be sure that it doesn’t break anything else. File triggers are already used by Mandriva, but Panu wants to implement them in a different fashion. (See Mandriva wiki for the details about their implementation).

3. Soft dependencies

   Soft dependencies keywords that are already used by SUSE (Recommends, Suggests, Supplements, Enhances) will be recognized in the future versions of RPM. RPM will not do anything with them except of storing in the database and reporting to higher levels of package management stack. There is an ongoing discussion whether and how to implement soft dependencies in YUM (they are already implemented in the zypp stack).

4. Update scriptlets

   We agreed to introduce two new scriptlets called %preup and %postup. These will be called when updating the package (%preun, %postun, %pre and %post will not be run in this case). This is a way how to fix the broken uninstall scriptlets and one doesn’t have to do the weird if [ "$1" = "0" ]; stuff in the scriptlets anymore to detect whether the operation is upgrade or uninstall.

5. Scriptlets arguments

   Package scriptlets will get more information (e.g. NEVRA of the packages) about currently run transaction in environment variables. This will make it easier for scriptlets to handle special situations or to detect more precisely what is happening. (One could for example convert some config files when upgrading from package of version 1.x to 2.x and to leave them intact otherwise).

6. DeltaRPM

   DeltaRPM sources will stay in gitorious at the moment. They might appear in RPM upstream git repository later in the future but there is no reasoning for this right now. Jindrich Novy is working on a new diff algorithm, we’ll see if it will be merged into DeltaRPM or replace it completely.

7. New payload format

   Currently used format is CPIO but it has a filesize limit of 8 GiB. This is not enough when one tries to distribute very large files, for example appliance images, packaged as RPMs. Developers will need to find a way how to enhance CPIO or try another format soon. TAR is really not an option …

8. Tilde in version

   The special character tilde (~) will be available for use in version representing a negative version token. This will simplify complicated rules which abuse the version and the release tags. (For example, using foo-2.5.99.2 instead of the foo-2.6~beta2). There already exists a patch but it needs more test coverage.

9. Easy way to add or remove autogenerated dependencies

   This feature is wanted but is not implemented yet. Currently this is workarounded by redefining the __find_provides/__find_requires/… in spec file. Example from the open-vm-tools package:

   # exclude AMD PCnet32 LANCE pci.id from Supplements list [bnc#397554]
   %define __find_supplements sh -c '/usr/lib/rpm/find-supplements %{name} | grep -v pci:v00001022d00002000'

10. Logging and recovery

    There is a plan to include a transaction log to RPM which will allow recovery after a broken transaction. RPM log could also replace similar features in YUM/zypper and will be more reliable as it would also work from rpm cli.

That’s it! I had a great time during the conference and was really happy to see the synergy between the developers, even though they are working for two competing companies. In case you have any questions feel free to join us on Freenode IRC channel #rpm.org or use the appropriate mailing list.

I finally implemented very important feature of scout: It is now able to search for binaries in all enabled zypp repositories. SAT-solver files are used for this, so user does not have to install any external index files. I really would like to thank Klaus Kaempf for his exhaustive help with python bindings for satsolver. Another great news are that Werner Fink applied command-not-found patch for bash package, so 11.1 will probably contain this feature working out of the box! Current early implementation has one problem though: it is pretty slow comparing to older use-own-sqlite-database approach (2 seconds compared to 0.2 seconds). But it indexes more repositories at once (I have 14 enabled) and I believe the code could more optimized and thus whole search faster in the (near) future.

Feel free to test the packages from BuildService (follow the instructions on wiki) and tell me what you think of it !

Hooray! The first public version of Scout was just released. It is a simple tool which allows user to look for (not yet installed) packages using simple queries (e.g. which autoconf macros does the package contain, which Java classes are present inside or what binaries does the package provide). Scout is available in openSUSE BuildService in home:prusnak:scout project. If you would like to install and test it, follow these three steps:

1. add the following repositories:

   zypper ar http://download.opensuse.org/repositories/home:/prusnak:/scout/openSUSE_11.0 scout
   zypper ar http://download.opensuse.org/repositories/home:/prusnak:/scout/data scout-data

   (change openSUSE_11.0 to your distribution if necessary)

2. install scout:

   zypper in scout

3. add any of the index data you find attractive (only example – see scout wiki page for the whole list)

   zypper in scout-bin-suse110
   zypper in scout-java-suse110

   Data package names are in format scout-module-repo. Indexes for autoconf macros are in autoconf packages, bin are indexed binaries and java are indexed java classes. Repository names are either distributions (sle10, suse101, suse102, suse103, suse110) or BuildService projects (jpackage17 for Java:jpackage1.7). Simple, isn’t it?

   (Warning: Java indexes are pretty large – 5MB RPMs and around 30MB when installed.)

When you are ready, run scout without parameters to see the help. I think that the usage is pretty straightforward (nevertheless, I will try to create demonstration video soon). You can also try running scoutcsv or scoutxml – they are only symlinks, but they produce CSV or XML output, instead of table output.

I hope you will like it!

The next thing to do is to add support for ZYPP repositories (sat-solver files) in module for binaries, so user could query packages (even in the BuildService repositories) without installing any external index data. When this is done, it would be a piece of cake to replace the earlier implementation of command-not-found with the new one using scout as its backend. Unfortunately, this is not going to happen before the ZYPP bindings for Python (python-zypp) are fixed. (API has changed and it seems that only Ruby bindings are maintained.) I tried to fiddle with it (bugzilla), but I’m not a SWIG expert.