Ian Bicking: site-packages Considered Harmful

February 8th, 2006 § 1 comment

Ian Bick­ing has an inter­est­ing post: site-packages Con­sid­ered Harm­ful to which I made a comment:

My com­ment:
“Totally agreed.

One of the big things I have to deal with are black-box “appli­ance” instal­la­tions of hosts. In this case, my appli­ca­tion should not have to mod­ify site-packages to install things it needs (like sqlob­ject, post­gresql pack­ages, etc) on the remote host. Once I mod­ify any­thing out­side of my $CWD (i.e: the instal­la­tion path) I have altered the sys­tem in such a way that in all future instal­la­tions I have to walk down into that site-packages direc­tory and check for versions/updates/etc.

I typ­i­cally work around this by drop­ping every­thing into a /lib direc­tory of my app — and then hack­ing sys.path to suck in the direc­tory rel­a­tive to my appli­ca­tions path. This makes exe­cu­tion out­side of that direc­tory dif­fi­cult as I have to assume that /lib is rel­a­tive to the binary loca­tion. I real­ize there are other ways around this — but have some­thing built into the inter­preter that says “I am only going to check dir X for mod­ules (of a set revi­sion)” then oth­er­wise forc­ing users/applications to hack with sys.path and the $PYTHONPATH is sort of cludgy.

I don’t have any good alter­na­tives — but I do know that in addi­tion to hav­ing to work with an appli­ance where I shouldn’t be mod­i­fy­ing any­thing out­side of $MYDIR and then poten­tially work­ing with mul­ti­ple ver­sions of Python on the same machine (2.3/2.4) with the cur­rent site pack­ages method­ol­ogy is really frustrating.”

Expand­ing on this a lit­tle bit — I really don’t like the expec­ta­tion of libraries installed into a core sys­tem path, espe­cially for things that I write (i.e: sin­gle use instal­la­tion sys­tems, portable tools, etc). I’ve had to fre­quently rekit the appli­ance OS I work with to add miss­ing python mod­ules that are just too heavy to drop into a seper­ate /lib direc­tory for a tool that’s only 150 lines of code.

I know there are other (bet­ter) ways of han­dling it then what I have done. For exam­ple, I’ll typ­i­cally have a direc­tory struc­ture like this:

foo.py
/lib
module.py
Or:
/bin
foo.py
/lib
module.py

And in foo.py I do some sim­ple “if os.path.isdir()” checks and hack sys.path to append the lib direc­tory. It’s hack­ish, and I always end up hav­ing to rely off of a poten­tially rel­a­tive path. I could of course, in foo.py look at the path of foo.py and pull in the lib direc­tory based off of the full path, minus foo.py (and add /lib) which would allow you to run the scripts out­side of the foo.py direc­tory (bypass­ing rel­a­tive path issues) but in all, it’s just annoying.

Like I said in the com­ment — I don’t have any good sug­ges­tions. Except maybe to have python’s inter­preter be intel­li­gent enough to look for path’s rel­a­tive to the path of the script in ques­tion for a direc­tory named python-lib (I know… a hack) and for tools peo­ple to be able to drop things in that directory.

All in all though — I still wouldn’t give up Python

  • cav71

    you could try some­thing along the lines:

    import os
    import sys

    if __name__ == “__main__”:
    bdir=os.path.dirname(__file__)
    sys.path.insert(0,bdir)

What's this?

You are currently reading Ian Bicking: site-packages Considered Harmful at jessenoller.com.

meta