Ian Bicking: site-packages Considered Harmful

by jesse in ,

Ian Bicking has an interesting post: site-packages Considered Harmful to which I made a comment: My comment: "Totally agreed.

One of the big things I have to deal with are black-box "appliance" installations of hosts. In this case, my application should not have to modify site-packages to install things it needs (like sqlobject, postgresql packages, etc) on the remote host. Once I modify anything outside of my $CWD (i.e: the installation path) I have altered the system in such a way that in all future installations I have to walk down into that site-packages directory and check for versions/updates/etc.

I typically work around this by dropping everything into a /lib directory of my app - and then hacking sys.path to suck in the directory relative to my applications path. This makes execution outside of that directory difficult as I have to assume that /lib is relative to the binary location. I realize there are other ways around this - but have something built into the interpreter that says "I am only going to check dir X for modules (of a set revision)" then otherwise forcing users/applications to hack with sys.path and the $PYTHONPATH is sort of cludgy.

I don't have any good alternatives - but I do know that in addition to having to work with an appliance where I shouldn't be modifying anything outside of $MYDIR and then potentially working with multiple versions of Python on the same machine (2.3/2.4) with the current site packages methodology is really frustrating."

Expanding on this a little bit - I really don't like the expectation of libraries installed into a core system path, especially for things that I write (i.e: single use installation systems, portable tools, etc). I've had to frequently rekit the appliance OS I work with to add missing python modules that are just too heavy to drop into a seperate /lib directory for a tool that's only 150 lines of code.

I know there are other (better) ways of handling it then what I have done. For example, I'll typically have a directory structure like this:

foo.py /lib module.py Or: /bin foo.py /lib module.py

And in foo.py I do some simple "if os.path.isdir()" checks and hack sys.path to append the lib directory. It's hackish, and I always end up having to rely off of a potentially relative path. I could of course, in foo.py look at the path of foo.py and pull in the lib directory based off of the full path, minus foo.py (and add /lib) which would allow you to run the scripts outside of the foo.py directory (bypassing relative path issues) but in all, it's just annoying.

Like I said in the comment - I don't have any good suggestions. Except maybe to have python's interpreter be intelligent enough to look for path's relative to the path of the script in question for a directory named python-lib (I know... a hack) and for tools people to be able to drop things in that directory.

All in all though - I still wouldn't give up Python