The KDE Split Ebuilds HOWTO

1.  The Split KDE Ebuilds

What they are

Until January 2005, the only KDE ebuilds in Portage were 'monolithic' ones. That is to say, there were only 15 ebuilds (kdebase, kdenetwork, ...), and each one installed many applications that did not, in fact, depend on one another. This was clearly a suboptimal situation, and not very Gentoo-ish, but it was tolerated for a long time.

The new 'split' ebuilds (for konqueror, kmail, ...) rectified the situation by providing separate ebuilds for all the separate KDE applications. This gave us a grand total of about 330 new ebuilds in the kde-base category.

We still provide monolithic ebuilds for 3.5 (up till 3.5.9) and they are cleanly interoperable with the split ones. However, the split ebuilds are the new default, and there will be no monolithic ebuilds after KDE 3.5.9.

Finally, it should be mentioned that there are split ebuilds for Koffice as well. These provide kword, kugar, etc. as separate packages.

How to install the split ebuilds

The latest stable KDE release, as of this writing, is 3.5.9. The latest unstable (~arch) is 3.5.10. Split and monolithic ebuilds for both releases are present in Portage. The latest 4.1.x release is also in the tree.

• To emerge a particular package, such as kmail, simply emerge kmail.
• To emerge the basic KDE environment allowing you to login into a minimalistic KDE session, emerge kdebase-startkde.
• Finally, for the exact equivalent of one of the monolithic packages - for instance, to get all the applications included in kdebase using split ebuilds - you can emerge kdebase-meta (or kdepim-meta, etc.) To get absolutely all KDE split ebuilds, emerge kde-meta.

How to upgrade from the monolithic to the split ebuilds

If you have the KDE 3.4.x or 3.5.x monolithic ebuilds installed, you must unmerge them before emerging the split ebuilds. However, this process can be done for each monolithic ebuild in turn; you don't have to unmerge all of KDE at once.

If you're in doubt, remember there are blocking dependencies in place between each monolithic ebuild and the split ebuilds derived from it. Portage won't allow an illegal state to be created, so any emerge or unmerge it allows is OK.

Advantages of the split ebuilds

Here's a brief list of what we gain from switching to the split ebuilds:

• Most KDE packages aren't changed at all between minor KDE releases. For example, the update from 3.3.1 to 3.3.2 changed fewer than 100 packages out of 320. Split packages allow us to create new ebuilds only for the packages that are actually changed, saving (in this example) more than two-thirds of the compilation time on an upgrade.
• Patches usually affect a specific package. With split ebuilds, they can be tested, approved and committed faster, and the developers have less to do; and, as above, the user will spend less time upgrading. This is especially important for security updates.
• Users of other desktops and leaner WMs can emerge a few KDE apps they like without the (quite big) overhead of the rest of, say, kdebase or kdepim.
• Users can fine-tune the packages they have installed. Reasons you might want this include:
  • You care about compilation time. emerge kdebase kdepim kdenetwork takes far too long when what you really need is konqueror, kmail and kopete. Besides, CPU time is money... somewhere.
  • You care about disk usage. Every unused package is that many megabytes blocking the pores between your disk's sectors. A disk with more free space breathes freely; it's a fast, happy disk.
  • You care about system security. All installed software is a potential source of vulnerabilities, and there's no excuse for unused software left lying around.
  • You faithfully adhere to the Gentoo Way, and can't stand packages being bundled together and forced on the user. (Neither could we.)
• Finally, the split ebuilds also allow more compile-time flexibility with USE flags.

Split and monolithic ebuild interoperability

Split and monolithic ebuilds can be mixed freely. The only restriction is that a monolithic ebuild can't be installed at the same time as a split ebuild deriving from it. There are blocking dependencies in the ebuilds that enforce this, so you can do anything emerge allows you to do.

Ordinarily, however, there's no reason to use such a mixed configuration. In fact, except for special cases like very slow-compiling boxes (mips), you should use the split ebuilds for all your needs.

The split ebuilds are also the default ebuilds. This means that when some other ebuild depends on a KDE application, it will want to install a split ebuild. However, the matching monolithic ebuild will also satisfy that dependency, so you can emerge the monolithic ebuild manually and then emerge the ebuild that depended on it.

2.  Performance issues

Why split ebuilds are slow

It's been said before that split ebuilds would take much more time to emerge than the monolithic ones, due to the overhead of unpacking and running configure for every package. A complete emerge kde-meta could take 20-30% longer than a classic emerge kde, unacceptable in an already long compile.

Moreover, at present the split ebuilds always run make -f admin/Makefile.cvs (this means running autoconf, automake, etc. and several related kde-specific scripts). This adds an additional slowdown of approximately the same order as a configure run.

Finally, a split ebuild needs to extract specific files out of a large tarball. This is slower than extracting a dedicated, small tarball. However, creating such small tarballs for the autotools-based build system of KDE 3.x is difficult.

It is worth reiterating here that with the split ebuilds a KDE upgrade's compilation time can be greatly reduced by only upgrading the packages that actually changed. The benefit from a single such update often overshadows the overhead incurred during the original installation.

Finally, installing all of KDE makes sense if you want to explore the available packages or are setting up a multi-user environment; however, most people use only some of the 300+ KDE apps available. Anyone who really cares about compilation time, such as owners of older boxes, can gain more time by selectively installing packages than they might lose by the overhead incurred.

How split ebuilds will be made faster

Most or even all of the split ebuilds' performance issues are tied to autotools - autoconf, automake and other tools which manage the ./configure;make;make install build system used in KDE 3.x.

KDE 4 has adopted a completely new build system, cmake, which among other things greatly reduces the time its equivalent of a make -f admin/Makefile.common; ./configure takes.

3.  Split ebuilds FAQ

Why are some split packages missing the newest ebuild versions?

As explained above, not all applications are really updated between minor KDE releases, and so not all applications receive new ebuild versions. For instance, libkdenetwork wasn't updated in 3.5.0_beta2, so the latest ebuild available with that release was 3.5_beta1.

This is done purely to reduce compilation time during an upgrade. If we had made a libkdenetwork-3.5.0_beta2 ebuild, it would have installed precisely the same files as the 3.5_beta1 ebuild. The various dependencies are updated to work correctly (i.e. no ebuild will depend on libkdenetwork-3.5.0_beta2).

Note that, because of this, if you want to install masked versions of KDE, you might also need to unmask packages from a previous version of KDE, if they are also masked. You might want to read this bug for more details.

Can't we do this already with DO_NOT_COMPILE?

DO_NOT_COMPILE is an environment variable internal to the KDE build system. It allows selectively disabling subdirectories from compilation. Some people used to use it to compile subsets of the monolithic KDE ebuilds. For instance, running DO_NOT_COMPILE=konqueror emerge kdebase would install a kdebase without the konqueror application.

However, DO_NOT_COMPILE was never intended to be used to interfere with the operation of a package manager's automated builds. It does not work, it can break your system, and it was never supported. We request everyone to refrain from using it.

Here is a partial list of the problems with DO_NOT_COMPILE:

• It completely breaks Portage's dependency tracking. Portage does not know about DO_NOT_COMPILE, and thinks the entire monolithic package has been installed and can satisfy other packages' deps. This can cause other packages not to emerge or not to run.
• It forces the user to know the names and meanings of all the different existing subdirs of the KDE modules. Very few users do know this, unless they're KDE developers, so they can't use DO_NOT_COMPILE properly.
• KDE module subdirs can have interdependencies between them, require a particular build order, require another dir to be present even if it does not have to be installed, and so forth. We put a lot of work into the split ebuilds to make them work properly in this regard. DO_NOT_COMPILE is not nearly a fine enough tool to achieve the same results, even given sufficient knowledge on the user's part. The only thing you can do with it is disable a few applications from compiling. It is practically impossible to use it to install only a few selected applications from modules like kdebase or kdepim.
• If I installed kmail yesterday and want to add korn today, using DO_NOT_COMPILE, it entails recompiling kmail as well. This means DO_NOT_COMPILE is always much slower than split ebuilds.
• DO_NOT_COMPILE can't be used to make precompiled packages (such as the GRP) containing individual KDE apps.

Aren't you putting too big a load on the Gentoo KDE maintainers?

Surprisingly, this question gets asked a lot. I'm glad users are so considerate of us maintainers. Let me take this opportunity to assure you that we're taking on the split ebuilds of our own free will; that we believe we'll be able to continue maintaining them well; and that there's no chance of talking us out of it :-)

For completeness' sake, I should mention that maintainers from other archs have in fact complained about the increased workload of testing and keywording so many separate ebuilds. We're working to resolve this and it's a major reason why monolithic ebuilds are in fact still available for KDE 3.5.

Are you going to remove the old-style (monolithic) ebuilds?

We intend to do so eventually. However, there are both monolithic and split ebuilds for all the KDE 3 releases up to 3.5.9. For KDE 3.5.10 and later, and KDE4, we don't provide monolithic ebuilds anymore.

If you prefer the monolithic ebuilds over the split ones, please tell us your reasons.

There are too many ebuilds! How am I going to find the one I need?

Well, first of all, if you know the package you're looking for came with kdebase, you can still emerge kdebase-meta, with much the same results as if you emerged the monolithic kdebase. So, things haven't actually become any worse due to the split ebuilds.

Of course, all the usual ways of locating a package also apply. How would you find your ebuild if it was a Gnome application? As a minimum, you do have to know the name of the application you're looking for.

The situation could, perhaps, be improved by introducing some more -meta ebuilds. They are merely lists of dependencies, so they don't cost us anything. This hasn't been decided yet. Also, it would be nice to have Portage sets functionality before we do this extensively.

How can I list/unmerge all split ebuilds derived from a given package?

The objective here is to list all split kde ebuilds derived from, say, the kdebase monolithic ebuild. Once again, the proper implementation (such as GLEP 21) would make this trivial. Today, however, you must become involved in the KDE eclasses' implementation details to some degree. So, if you use any of these approaches in a script that's not for private use, tell us about it.

kde-functions.eclass defines functions called get-parent-package() and get-child-packages() which do the translation for you. These two functions are the correct way to accomplish this task from an ebuild or an external bash script. Here is an example:

$ function die() { echo $@; } # called to report errors
$ source /usr/portage/eclass/kde-functions.eclass
$ get-parent-package konqueror # won't work, you must specify full name
Package konqueror not found in KDE_DERIVATION_MAP, please report bug # error printed
$ get-parent-package kde-base/konqueror # fully qualified package name
kde-base/kdebase # result printed
$ get-child-packages kde-base/kdebase
(Long list of packages printed here)

If your script isn't in bash, you can grep kde-functions.eclass to extract the (multiline) definition of the variable KDE_DERIVATION_MAP, which the aforementioned functions use. This variable contains a whitespace-separated list of words, and each two consecutive words map a parent package to a child split ebuild.

The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license.