So you want to use python on the mac?

March 16th, 2009 § 37 comments

4655664E-CBA5-4AF4-B813-87854FC67289.jpgIn a com­plete tan­gent from my numer­ous other projects, I’ve had a few peo­ple ask me recently about python on the mac, how to get started/etc.

I’m going to solely focus on python in Leop­ard (10.5.x) and not any­thing before that. Any­thing before that is dead to me! DEAD!

If you open a shiny new mac­book, and want to get started hack­ing, the first place you need to stop is the Apple Dev site, and down­load the lat­est ver­sion of XCode. XCode installs gcc and other tools which you are going to want and need sooner or later. It’s a mon­ster down­load, but bite the bullet.

By default, Leop­ard comes with Python 2.5.1 — the bina­ries and stan­dard library (and any Leop­ard addi­tions) are installed into /System/Library/Frameworks/Python.framework/Versions/2.5/ /Library/Python//, the site-packages direc­tory is in /Library/Python/…/site-packages. This is known as a frame­work build of Python, and alter­ing that instal­la­tion is, well, a bad idea. I don’t rec­om­mend it.

Now, some peo­ple may rec­om­mend you install mac­ports or fink: these are both “sorta” pack­age man­agers for OS/X, and while I do have mac­ports installed, I do not use it for python work. I pre­fer com­pi­la­tion and self man­age­ment, that and I’ve had issues with mac­ports in the past. So, go ahead and install it, but don’t use it for python work.

So, now you have xcode — fire up terminal.app, from now on, this is your best friend. When you fire it up, you’re in your /Users/username direc­tory — your home direc­tory. You want to look for one of the fol­low­ing files, .bashrc or .bash_profile. If the lat­ter doesn’t exist, do a “touch .bash_profile” — this will cre­ate an empty one for you, which you will need later on.

If you were to type “python” in right now, the default leop­ard install (2.5.1) would pop up. The site-packages direc­tory lives in /Library/Python/2.5/site-packages. If you were to start installing python mod­ules, that’s where they would go. But we’re not going to do that.

Head over to python.org — go to the down­load page and down­load the lat­est 2.6 release of python (2.6.1 right now). Unpack and install it — the installer will install this into Library/Frameworks/Python.framework/ — this is out­side of the default sys­tem install. It may install sym­bolic links to the bina­ries in /usr/local/bin, oth­er­wise scripts that come with installed pack­ages and the python binary itself go into /Library/Frameworks/Python.framework/Versions/2.6/bin.

The installer will also mod­ify your ~/.bash_profile we cre­ated to prepend this onto your PATH vari­able. If you view your ~/.bash_profile you should see a mod­i­fied ver­sion which adds the /usr/local/bin or the /Library/Frameworks/Python.framework/Versions/2.6/bin direc­tory to your path (it should be labeled with a comment).

Note, that if the installer doesn’t update your pro­file, sim­ple add this line:

export PATH=/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/local/bin:$PATH

I totally boned this ear­lier: Just to clar­ify, the down­loaded ver­sion of python installs sym­bolic links in /usr/local/bin which point to bina­ries in the /Library/Frameworks/Python.framework/Versions/2.6/bin — any other tools you install will place their bina­ries into that /Library/Frameworks/Python.framework/Versions/2.6/bin direc­tory as well (some might also sym­link into /usr/local/bin).

Do the fol­low­ing “. ~/.bash_profile” — this sources the pro­file and changes your envi­ron­ment. You only need to do this once for this win­dow, any time you open a new ter­mi­nal win­dow, this is done for you.

Now you have the lat­est and great­est python 2.6 installed. You still have the sys­tem install in the default loca­tion, and you can swap between either one by using an explicit path, for exam­ple “/usr/bin/python” loads the sys­tem version.

IDLE is installed into the /Applications/Python… direc­tory, if you need it.

So you have a new shiny install, and you’re ready to get started…

No!

If you started down­load­ing libraries and stuff to this new install you’d be taint­ing an oth­er­wise pris­tine instal­la­tion of python. No, you wouldn’t want to do that, would you?

Rather than taint it, we’re going to install a few depen­den­cies you’ll need an a handy tool called vir­tualenv. Trust me, you’ll thank me.

Most of the known python world uses easy_install (with setup­tools) to install pack­ages from the web. I pre­fer pip. To be effec­tive in the world, you’re going to want to install both, and only fall back to easy_install if pip fails you.

So, down­load this python script — save it to a tem­po­rary direc­tory, your down­loads folder, it doesn’t mat­ter. This is ez_setup.py, and will boot­strap setup­tools and easy_install for you — just run:

sudo python ez_setup.py

Once that’s com­plete, head over to pypi and down­load the pip tar­ball. Unpack it and exe­cute “sudo python setup.py install”.

Next up, well — let’s cheat — in ter­mi­nal, type:

sudo pip install virtualenv

This installs easy_setup and pip and vir­tualenv into the 2.6.1 install we installed pre­vi­ously. The bina­ries are installed to /Library/Frameworks/Python.framework/Versions/2.6/bin. Your egre­gious mod­i­fi­ca­tion of this pris­tine install is almost complete.

Next install virtualenvwrapper:

1
sudo pip install virtualenvwrapper

The envi­ron­ment file is installed to /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc

Now edit your ~/.bash_profile and add two lines at the bottom:

1
2
export WORKON_HOME=$HOME/virtualenvs
source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc

Save it and then run “. ~/.bash_profile” — this will add the vir­tualenv com­mands to your envi­ron­ment. Hooray! This also sets the default direc­tory that they’re stored in to “~/virtualenvs”. Much vic­tory to be had. You should run “mkdir ~/virtualenvs” at this point.

So, we have the basics. Now, you’re going to want to make your default vir­tualenv, in ter­mi­nal type:

mkvirtualenv default

This will cre­ate a ~/virtualenvs/default vir­tual envi­ron­ment where you can poke around and begin to install juicy things into. Note your prompt changes to reflect the envi­ron­ment loaded. I have my .bash_profile set to load the default ver­sion when­ever I fire up terminal.app, for example:

1
2
3
export WORKON_HOME=$HOME/virtualenvs
source $HOME/env_ext/virtualenvwrapper_bashrc
workon default

You can point editors/etc to point at this envi­ron­ment as well, bina­ries, libraries/etc are loaded into ~/virtualenvs/default/bin and ~/virtualenvs/default/lib for example.

You can install all sorts of good­ies, for exam­ple, if I wanted to cre­ate a pygame envi­ron­ment, I would do this:

1
2
3
mkvirtualenv pygame
cd ~/virtualenvs/pygame
easy_install pygame

And then at any time, I could swap over to the pygame one with the “workon” com­mand pro­vided by vir­tualenv wrapper.

You can also cre­ate a vir­tual env pointed at the old sys­tem instal­la­tion — some­thing like this should do the trick:

1
2
cd ~/virtualenvs
virtualenv -p /usr/bin/python python2.5.1

And then “workon python2.5.1″ will load the vir­tualenv point­ing at that inter­preter. Viola! You’ve man­aged to not taint the built in ver­sion, add a more recent ver­sion (mod­i­fy­ing it only a lit­tle) and add the abil­ity to rapidly swap envi­ron­ments. Hooray!

Astute read­ers might point out that adding the “to the side” install of python would remove access to the pyobjc bridge that ships with the built in ver­sion, you can pip/easy_install the objc libraries now: check out pypi. Also, adding the 2.5.1 vir­tualenv means you can hack on the shipped ver­sion of pyobjc as well.

I’d rec­om­mend ready the docs for vir­tualenv, vir­tualenv wrap­per, pip and easy_install so you under­stand how things actu­ally work, I’ve only cov­ered the basics here.

The down­loaded ver­sion of python doesn’t taint your sys­tem, it installs cleanly to the side of the built in ver­sion of python, any­thing past that point is shell hackery.

Side note
If you want a quick and dirty way of installing a more fea­ture rich ver­sion of read­line, you can also just run:

pip install readline

I’d also like to finally point out that this makes my life (I’m is run­ning the default + lat­est shipped 2.6 + lat­est shipped 3.0 + 2 ver­sions for python svn) life much eas­ier to deal with.

See also this write up by Kumar McMillan

  • http://spookypony.com Peter Hern­don

    Bril­liant! Thanks very much for this. I’ve been using my default installed envi­ron­ment, and have had a series of nig­gling doubts about how I’ve been (ab)using things. You’ve clearly spelled out the process for set­ting things up with proper sep­a­ra­tion. I’ll be upgrad­ing to a new note­book in a cou­ple of months, and will be using your recipe.

  • http://jessenoller.com jnoller

    Let me know if you have any issues!

  • http://blog.roberthahn.ca/ roberthahn

    thanks for this, Jesse. Is your ver­sion of Python 2.6 a 64-bit exe­cutable? I’ve been hav­ing trou­ble get­ting it to com­pile as a 64-bit app (fol­lowed what­ever direc­tions I could find, looked straight­for­ward enough but no 64-bit app at the other end).

  • http://jessenoller.com jnoller

    Nope, by default the installed ver­sion of python *and* the one from the web­site are in 32 bit mode.

  • Nat

    small point, but isn’t the default Leop­ard Python frame­work actu­ally installed in /System/Library, rather than /Library?

  • http://jessenoller.com jnoller

    thumper:~ jesse$ python –c “from distutils.sysconfig import get_python_lib; print get_python_lib()”
    /Library/Python/2.5/site-packages

    Not any­more it seems

  • Nat

    ok, I’m not totally crazy. The bina­ries and stan­dard mod­ules and such are in /System/Library/Frameworks/Python.framework/Versions/2.5/

  • http://jessenoller.com jnoller

    True, I’ll amend it

  • Doug Napoleone

    NOTE: if you are build­ing libraries for mul­ti­ple ver­sions of python on your Mac (mod_wsgi, psycopg2, your own 3rd party exten­sion libraries) then you need to install the dif­fer­ent ver­sions of python into inde­pen­dent frame­works. This is done via con­fig­ure when you build python, you spec­ify an alter­na­tive name.

    This is because of the mess the frame­work sys­tem is with the cus­tom gcc which comes with xcode. You can spec­ify the frame­work (-Wlframework,Python) but you can not spec­ify which VERSION f the frame­work to use as it will always used ‘Cur­rent’ with is a sym­link to 2.5 or 2.6 depend­ing on what was LAST installed.

    As a result I set the frame­work name to include the ver­sion num­ber Python25 or Python26 respec­tively. Then I can spec­ify which frame­work to use ‘Python25’ or ‘Python26’.

    Fur­ther NOTE: this does NOT work for your binary releases as they then expect that spe­cific Frame­work name :-(

    For that I end up hav­ing to have mul­ti­ple Library direc­to­ries, which is an even big­ger hassle.

    If any­one has any infor­ma­tion on mak­ing this work bet­ter for build­ing binary mod­ule for python for mul­ti­ple ver­sions of python on a sin­gle mac, I would greatly appre­ci­ate it.

  • http://jessenoller.com jnoller

    Inter­est­ing; this I did not know (although, it’s out of my par­tic­u­lar prob­lem domain) — if I find the time I might dig into it a bit more and see if there’s any­thing to be done.

    Another thought is to out­line it in detail and add it to the python bug­tracker, see if an enhance­ment can be made on our end.

  • Doug Napoleone

    To be hon­est this is all just a workaround to a prob­lem with the apple sup­plied GCC. I have already filed a bug there, but not sure what will hap­pen. There is no rea­son why you should not be able to tell the com­piler which ver­sion of the frame­work to com­pile against.

  • http://blog.roberthahn.ca/ roberthahn

    yeah, ok. I was ask­ing because I was try­ing to set up a python web stack (apache 2 + mod_wsgi + python) and was hav­ing trou­ble get­ting a 64-bit apache (the stock install in Leop­ard) to play nice with a 32-bit python — i did get it sorted out, but the expe­ri­ence has left me won­der­ing if there was a bet­ter way. :)

  • http://blog.dscpl.com.au Gra­ham Dumpleton

    For ref­er­ence, the 32 bit/64 bit issues that come up with Apache+mod_wsgi are doc­u­mented in:

    http://code.google.com/p/modwsgi/wiki/Installat

    Equally rel­e­vant if still using mod_python.

  • http://machine-envy.com/blog James

    What build options are you using for Python?

    enable-framework?

  • Nat

    In the arti­cle, I think he’s just talk­ing about using the .dmg installer, rather than build­ing it from source. If you want to install a ver­sion for which there is not an installer out yet, Kumar has some nice instruc­tions for build­ing the frame­work here: http://farmdev.com/thoughts/66/python-3–0-on-ma

  • http://www.doughellmann.com/ dhell­mann

    vir­tualen­vwrap­per is also easy_install-able (I haven’t tried pip but it should work, too).

  • http://parand.com/say/ Parand

    Talk about timely, I’m just set­ting up a new OS X envi­ron­ment. Thanks for this.

  • http://pieceofpy.com Wayne Witzel III

    Great write up. Get­ting my new Mac­book Pro next week so excel­lent timing.

  • Simon Hibbs

    I must be doing some­thing wrong. I’ve installed Python, but when­ever I install another pack­age (easy_install, pip, vir­tualenv) none of the scripts that should go into /usr/local/bin are there. The mod­ules are installed because I can import them in Python, but I can’t run the scripts from the com­mand line because they aren’t in the path (the path is cor­rect). I’ve tried unin­stalling and rein­stalling ez_setup.py for exam­ple, to no avail.

    I’m guess­ing it’s a per­mis­sions thing, but I’m using sudo and the per­mis­sions on /usr/local/bin are cor­rect. Any sug­ges­tions? How does python know where to put these files? I must have got an incor­rect path in there some­where but I can’t find it.

    PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

    Simon Hibbs

  • http://jessenoller.com jnoller

    Did you install python 2.6.1 from the web­site? If so, did you source your bashrc or restart terminal.app to ensure the $PATH was set prop­erly, and that you’re using /usr/local/bin/python instead of /usr/bin/python? I see you have set $PATH, but did you *export it*?

    PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/X11/bin
    export $PATH

    Type “which python” to show where the python binary is that is being loaded when you exe­cute “python” on the com­mand line.

    You may want to fol­lowup with me via email — jnoller at the google mail place.

  • Ned Deily

    Unlike with the Apple-supplied Python, easy_install using the python.org pythons does not install scripts to /usr/local/bin by default. If you are not using a vir­tualenv, the default loca­tion for scripts is in the python frame­work bin direc­tory, for example

    /Library/Frameworks/Python.framework/Versions/2.6/bin

    and you would need to add that direc­tory to your path. If you use a vir­tualenv, there will be a sep­a­rate bin direc­tory within the vir­tual env and that direc­tory will be added to your path when you acti­vate it.

  • http://jessenoller.com jnoller

    Yup, I fol­lowed up with simon via email, I totally boned this, and I’m alter­ing the post to reflect that.

  • http://jessenoller.com jnoller

    Post amended.

  • http://jessenoller.com jnoller

    Post amended to reflect cor­rect path

  • http://jessenoller.com jnoller

    I am using the python.org ver­sions. If I am com­pil­ing my own, it’s gen­er­ally for python-core work, and I don’t use –enable-framework

  • http://jessenoller.com jnoller

    Amended to reflect this fact

  • Pat

    OK, I’m doing things the “right” way and haven’t budged past the point where I last under­stood what I was doing. There­fore: I ask this very sim­ple Q what does, “Do the fol­low­ing “. ~/.bash_profile” — this sources the pro­file and changes your envi­ron­ment.”, mean exactly?

    Am I sup­posed to be in Ter­mi­nal? Do I run a com­mand? Do means many things, but here evi­dently for me it’s ambigu­ous. What I gist from it is that when I run a new win­dow in Term, I’ll get ref­er­ences to the later ver­sion of Python, correct?

    Other than that, I am on board and look for­ward to work­ing with Python 2.61(2)

    Thanks for your contributions!

    Pat

  • http://jessenoller.com jnoller

    Yes, you need to be in ter­mi­nal, and type that com­mand. Every­thing I
    out­lines here is in terminal.app

  • Pat

    Hi Jesse!

    Hey I am almost home. I can switch between Py 2.5 and 2.6. It seems to work well. How­ever, there seems to be a bit of a snag with the “ille­gal option”. The .bash_profile has been mod­i­fied as you instructed to reflect the path as outlined:

    »export PATH=/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/local/bin:$PATH
    source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc
    export WORKON_HOME=$HOME/virtualenvs«

    How­ever, I get this mes­sage from my term win­dow on open.
    Last login: Tue Apr 7 09:11:58 on ttys002
    dirname: ille­gal option — b
    usage: dirname path
    p-ws-mac-mini:~ pw$

    I had another error mes­sage before I added the (.) cre­at­ing an invis­i­ble dir. I can­not glean where this is com­ing from.

    All else appears to func­tion as you have outlined.

    Thanks for your help.

    Pat

  • http://jessenoller.com jnoller

    Com­ment out the “source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc” line and reload every­thing to exclude the pos­si­bil­ity it’s a bug in vir­tualenv wrap­per first

  • Doug Lator­nell

    I’m curi­ous how you han­dle your “almost always used” tools in this vir­tualenv setup? In my case those are things like nose, cov­er­age, ipython, and Sphinx. I saw the add2virtualenv com­mand in vir­tualen­vwrap­per and thought, “Cool! I’ll just install nose, et al in my default vir­tualenv and add them to other vir­tualenvs as nec­es­sary.” Alas, that doesn’t work very well for pack­ages like nose that have entry points in bin/ — the she-bang path has to be edited so that it points to the cor­rect place, or I get import errors.

    So, do you install nose in every vir­tualenv you cre­ate, or do you “egre­giously mod­ify” your pris­tine site-packages/, or is there another option I’m missing?

  • http://jessenoller.com jnoller

    An addi­tional boot­strap script I have adds in the “com­mon things” — installing them into the cen­tral loca­tion (e.g. /bin) means that when I call nose, I am call­ing nose that’s installed against an inter­preter I may not be using (e.g., 2.5 vs. 2.6). See also the postmkvir­tualenv hook for vir­tualen­vwrap­per, which could be used to exec com­mands, rather than just change set­tings. I have a script which I run by hand though

  • http://jasonmbaker.wordpress.com Jason Baker

    Minor cor­rec­tion: OS X comes with Python 2.5.1 by default, not Python 2.5.2.

  • http://blog.captnswing.net captnswing

    I think I man­aged to get a full 64bit python web stack on OS X 10.5.

    see here http://blog.captnswing.net/2009/04/19/python-mo

  • http://jessenoller.com jnoller

    Fixed

  • Niklas

    Hello, thanks for a nice and clean way to run python. How­ever, WORKON_HOME needs to be set before virtualenv_wrapper is sourced, oth­er­wise it won’t have any effect.

  • http://jessenoller.com jnoller

    Fixed

What's this?

You are currently reading So you want to use python on the mac? at jessenoller.com.

meta