Dependency Hell

Poul-Henning Kamp writes about A Generation Lost In The Bazaar:

Here is one example of an ironic piece of waste: Sam Leffler's graphics/libtiff is one of the 122 packages on the road to www/firefox, yet the resulting Firefox browser does not render TIFF images.

Which garners this (part of a) reply from "iain":

So libtiff is an implicit dependency for Firefox even though Firefox cannot display tiff images, and this is an example of how badly designed the whole system is. Is it not more likely that someone somewhere (possibly the author, in his haste to show us how hardcore he was because he compiles everything from source) had forgotten to pass a --disable-tiff to one of the other 121 packages?

...which is in itself a symptom of the problem. Not only is the author expected to deal with all the fucking around required to build 121 pieces of interdependent software so that he can then turn his effort to building the web browser he initially wanted, he is expected to keep track of what dependencies in each individual dependency might or might not really be required.

And. And! Be able to think ahead so that he can remember in six months when he requires a different piece of software that has a requirement for whatever-it-is-that-depends-on-libtiff -- and itself does require the tiff functionallity -- he can go back and rebuild the partially built library (and libtiff, of course).

This is the contradiction between "easy" and "minimal". The "easy" way to build things is to require every possible sub-library you might ever call, even though the software which has required you won't ever call all of the features your library offers. But that leads to things like "Firefox requiring libtiff" which offends so many sensibilities. So the "minimal" way to do things is to laboriously keep track of both upstream and downstream dependencies, turning off features as required, so that things that are not needed are not built until they are. If, of course, you can remember later that you did turn them off.

Tools that make this "easy" -- such as rpm and yum -- merely hide the problem, and give you lots of opportunity to step into messes of your own making.

The "super-easy" way to deal with this would be for applications to ship with all of their dependencies, which would then be stored in their own directory structure. This would remove the dependency problem, since you'd get the whole bundle. But it would mean you would be then using up a ton more disk space since Firefox would have to ship with basically its own operating system. You would also use up more memory, since Firefox's loading of the libtiff stuff would be different from Chrome's -- even if the code that produced the library was identical in both places.

It would also mean you could uninstall a application, totally and completely, including all dependencies, with a simple directory removal.

On the one hand: disk is cheap. On the other hand: so is RAM. On the realistic hand, most of us are running computers that may not be expanded to use this cheap disk and RAM. (My home system is maxed out at 3.2GB, which is tons for a single user, but now I have three people logged into it constantly, so it isn't sufficient any more. But short of buying a new computer (or the ridiculous prospect of installing an SSD into this dinosaur) I'm stuck.) So while in the long run shipping your own collection of libraries may be the way to go, it won't fly in the short run.

Not that it will ever happen because it is, frankly, an ugly and wasteful solution.

Overall though I can't think that we can totally blame the bazaar for the hole we've dug, since for the most part we've only improved on shovels that were already in existence. And until the rate of change in computer applications programming slows down, we will never turn it into either a "profession" (a la engineers) or a "trade".

I mean seriously, how consistant would plumbing be if the principles guiding the design of the pipes fitting and water management changed totally every six years?