Skip to content

So you want to use python on the mac?

4655664E-CBA5-4AF4-B813-87854FC67289.jpgIn a complete tangent from my numerous other projects, I’ve had a few people ask me recently about python on the mac, how to get started/etc.

I’m going to solely focus on python in Leopard (10.5.x) and not anything before that. Anything before that is dead to me! DEAD!

If you open a shiny new macbook, and want to get started hacking, the first place you need to stop is the Apple Dev site, and download the latest version of XCode. XCode installs gcc and other tools which you are going to want and need sooner or later. It’s a monster download, but bite the bullet.

By default, Leopard comes with Python 2.5.1 – the binaries and standard library (and any Leopard additions) are installed into /System/Library/Frameworks/Python.framework/Versions/2.5/ /Library/Python//, the site-packages directory is in /Library/Python/…/site-packages. This is known as a framework build of Python, and altering that installation is, well, a bad idea. I don’t recommend it.

Now, some people may recommend you install macports or fink: these are both “sorta” package managers for OS/X, and while I do have macports installed, I do not use it for python work. I prefer compilation and self management, that and I’ve had issues with macports in the past. So, go ahead and install it, but don’t use it for python work.

So, now you have xcode – fire up terminal.app, from now on, this is your best friend. When you fire it up, you’re in your /Users/username directory – your home directory. You want to look for one of the following files, .bashrc or .bash_profile. If the latter doesn’t exist, do a “touch .bash_profile” – this will create an empty one for you, which you will need later on.

If you were to type “python” in right now, the default leopard install (2.5.1) would pop up. The site-packages directory lives in /Library/Python/2.5/site-packages. If you were to start installing python modules, that’s where they would go. But we’re not going to do that.

Head over to python.org – go to the download page and download the latest 2.6 release of python (2.6.1 right now). Unpack and install it – the installer will install this into Library/Frameworks/Python.framework/ – this is outside of the default system install. It may install symbolic links to the binaries in /usr/local/bin, otherwise scripts that come with installed packages and the python binary itself go into /Library/Frameworks/Python.framework/Versions/2.6/bin.

The installer will also modify your ~/.bash_profile we created to prepend this onto your PATH variable. If you view your ~/.bash_profile you should see a modified version which adds the /usr/local/bin or the /Library/Frameworks/Python.framework/Versions/2.6/bin directory to your path (it should be labeled with a comment).

Note, that if the installer doesn’t update your profile, simple add this line:

export PATH=/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/local/bin:$PATH

I totally boned this earlier: Just to clarify, the downloaded version of python installs symbolic links in /usr/local/bin which point to binaries in the /Library/Frameworks/Python.framework/Versions/2.6/bin – any other tools you install will place their binaries into that /Library/Frameworks/Python.framework/Versions/2.6/bin directory as well (some might also symlink into /usr/local/bin).

Do the following “. ~/.bash_profile” – this sources the profile and changes your environment. You only need to do this once for this window, any time you open a new terminal window, this is done for you.

Now you have the latest and greatest python 2.6 installed. You still have the system install in the default location, and you can swap between either one by using an explicit path, for example “/usr/bin/python” loads the system version.

IDLE is installed into the /Applications/Python… directory, if you need it.

So you have a new shiny install, and you’re ready to get started…

No!

If you started downloading libraries and stuff to this new install you’d be tainting an otherwise pristine installation of python. No, you wouldn’t want to do that, would you?

Rather than taint it, we’re going to install a few dependencies you’ll need an a handy tool called virtualenv. Trust me, you’ll thank me.

Most of the known python world uses easy_install (with setuptools) to install packages from the web. I prefer pip. To be effective in the world, you’re going to want to install both, and only fall back to easy_install if pip fails you.

So, download this python script – save it to a temporary directory, your downloads folder, it doesn’t matter. This is ez_setup.py, and will bootstrap setuptools and easy_install for you – just run:

sudo python ez_setup.py

Once that’s complete, head over to pypi and download the pip tarball. Unpack it and execute “sudo python setup.py install”.

Next up, well – let’s cheat – in terminal, type:

sudo pip install virtualenv

This installs easy_setup and pip and virtualenv into the 2.6.1 install we installed previously. The binaries are installed to /Library/Frameworks/Python.framework/Versions/2.6/bin. Your egregious modification of this pristine install is almost complete.

Next install virtualenvwrapper:

?View Code BASH
1

sudo pip install virtualenvwrapper

The environment file is installed to /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc

Now edit your ~/.bash_profile and add two lines at the bottom:

?View Code BASH
1
2

export WORKON_HOME=$HOME/virtualenvs
source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc

Save it and then run “. ~/.bash_profile” – this will add the virtualenv commands to your environment. Hooray! This also sets the default directory that they’re stored in to “~/virtualenvs”. Much victory to be had. You should run “mkdir ~/virtualenvs” at this point.

So, we have the basics. Now, you’re going to want to make your default virtualenv, in terminal type:

mkvirtualenv default

This will create a ~/virtualenvs/default virtual environment where you can poke around and begin to install juicy things into. Note your prompt changes to reflect the environment loaded. I have my .bash_profile set to load the default version whenever I fire up terminal.app, for example:

?View Code BASH
1
2
3

export WORKON_HOME=$HOME/virtualenvs
source $HOME/env_ext/virtualenvwrapper_bashrc
workon default

You can point editors/etc to point at this environment as well, binaries, libraries/etc are loaded into ~/virtualenvs/default/bin and ~/virtualenvs/default/lib for example.

You can install all sorts of goodies, for example, if I wanted to create a pygame environment, I would do this:

?View Code BASH
1
2
3

mkvirtualenv pygame
cd ~/virtualenvs/pygame
easy_install pygame

And then at any time, I could swap over to the pygame one with the “workon” command provided by virtualenv wrapper.

You can also create a virtual env pointed at the old system installation – something like this should do the trick:

?View Code BASH
1
2

cd ~/virtualenvs
virtualenv -p /usr/bin/python python2.5.1

And then “workon python2.5.1″ will load the virtualenv pointing at that interpreter. Viola! You’ve managed to not taint the built in version, add a more recent version (modifying it only a little) and add the ability to rapidly swap environments. Hooray!

Astute readers might point out that adding the “to the side” install of python would remove access to the pyobjc bridge that ships with the built in version, you can pip/easy_install the objc libraries now: check out pypi. Also, adding the 2.5.1 virtualenv means you can hack on the shipped version of pyobjc as well.

I’d recommend ready the docs for virtualenv, virtualenv wrapper, pip and easy_install so you understand how things actually work, I’ve only covered the basics here.

The downloaded version of python doesn’t taint your system, it installs cleanly to the side of the built in version of python, anything past that point is shell hackery.

Side note
If you want a quick and dirty way of installing a more feature rich version of readline, you can also just run:

pip install readline

I’d also like to finally point out that this makes my life (I’m is running the default + latest shipped 2.6 + latest shipped 3.0 + 2 versions for python svn) life much easier to deal with.

See also this write up by Kumar McMillan

  • Niklas
    Hello, thanks for a nice and clean way to run python. However, WORKON_HOME needs to be set before virtualenv_wrapper is sourced, otherwise it won't have any effect.
  • Minor correction: OS X comes with Python 2.5.1 by default, not Python 2.5.2.
  • Fixed
  • Doug Latornell
    I'm curious how you handle your "almost always used" tools in this virtualenv setup? In my case those are things like nose, coverage, ipython, and Sphinx. I saw the add2virtualenv command in virtualenvwrapper and thought, "Cool! I'll just install nose, et al in my default virtualenv and add them to other virtualenvs as necessary." Alas, that doesn't work very well for packages like nose that have entry points in bin/ - the she-bang path has to be edited so that it points to the correct place, or I get import errors.

    So, do you install nose in every virtualenv you create, or do you "egregiously modify" your pristine site-packages/, or is there another option I'm missing?
  • An additional bootstrap script I have adds in the "common things" - installing them into the central location (e.g. /bin) means that when I call nose, I am calling nose that's installed against an interpreter I may not be using (e.g., 2.5 vs. 2.6). See also the postmkvirtualenv hook for virtualenvwrapper, which could be used to exec commands, rather than just change settings. I have a script which I run by hand though
  • Pat
    Hi Jesse!

    Hey I am almost home. I can switch between Py 2.5 and 2.6. It seems to work well. However, there seems to be a bit of a snag with the "illegal option". The .bash_profile has been modified as you instructed to reflect the path as outlined:

    >>export PATH=/Library/Frameworks/Python.framework/Versions/2.6/bin:/usr/local/bin:$PATH
    source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc
    export WORKON_HOME=$HOME/virtualenvs<<

    However, I get this message from my term window on open.
    Last login: Tue Apr 7 09:11:58 on ttys002
    dirname: illegal option -- b
    usage: dirname path
    p-ws-mac-mini:~ pw$

    I had another error message before I added the (.) creating an invisible dir. I cannot glean where this is coming from.

    All else appears to function as you have outlined.

    Thanks for your help.

    Pat
  • Comment out the "source /Library/Frameworks/Python.framework/Versions/2.6/bin/virtualenvwrapper_bashrc" line and reload everything to exclude the possibility it's a bug in virtualenv wrapper first
  • Pat
    OK, I'm doing things the "right" way and haven't budged past the point where I last understood what I was doing. Therefore: I ask this very simple Q what does, "Do the following “. ~/.bash_profile” - this sources the profile and changes your environment.", mean exactly?

    Am I supposed to be in Terminal? Do I run a command? Do means many things, but here evidently for me it's ambiguous. What I gist from it is that when I run a new window in Term, I'll get references to the later version of Python, correct?

    Other than that, I am on board and look forward to working with Python 2.61(2)

    Thanks for your contributions!

    Pat
  • Yes, you need to be in terminal, and type that command. Everything I
    outlines here is in terminal.app
  • Simon Hibbs
    I must be doing something wrong. I've installed Python, but whenever I install another package (easy_install, pip, virtualenv) none of the scripts that should go into /usr/local/bin are there. The modules are installed because I can import them in Python, but I can't run the scripts from the command line because they aren't in the path (the path is correct). I've tried uninstalling and reinstalling ez_setup.py for example, to no avail.

    I'm guessing it's a permissions thing, but I'm using sudo and the permissions on /usr/local/bin are correct. Any suggestions? How does python know where to put these files? I must have got an incorrect path in there somewhere but I can't find it.

    PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

    Simon Hibbs
  • Post amended to reflect correct path
  • Ned Deily
    Unlike with the Apple-supplied Python, easy_install using the python.org pythons does not install scripts to /usr/local/bin by default. If you are not using a virtualenv, the default location for scripts is in the python framework bin directory, for example

    /Library/Frameworks/Python.framework/Versions/2.6/bin

    and you would need to add that directory to your path. If you use a virtualenv, there will be a separate bin directory within the virtual env and that directory will be added to your path when you activate it.
  • Post amended.
  • Yup, I followed up with simon via email, I totally boned this, and I'm altering the post to reflect that.
  • Did you install python 2.6.1 from the website? If so, did you source your bashrc or restart terminal.app to ensure the $PATH was set properly, and that you're using /usr/local/bin/python instead of /usr/bin/python? I see you have set $PATH, but did you *export it*?

    PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/X11/bin
    export $PATH

    Type "which python" to show where the python binary is that is being loaded when you execute "python" on the command line.

    You may want to followup with me via email - jnoller at the google mail place.
  • Great write up. Getting my new Macbook Pro next week so excellent timing.
  • Talk about timely, I'm just setting up a new OS X environment. Thanks for this.
  • virtualenvwrapper is also easy_install-able (I haven't tried pip but it should work, too).
  • Amended to reflect this fact
  • What build options are you using for Python?

    enable-framework?
  • Nat
    In the article, I think he's just talking about using the .dmg installer, rather than building it from source. If you want to install a version for which there is not an installer out yet, Kumar has some nice instructions for building the framework here: http://farmdev.com/thoughts/66/python-3-0-on-ma...
  • I am using the python.org versions. If I am compiling my own, it's generally for python-core work, and I don't use --enable-framework
  • For reference, the 32 bit/64 bit issues that come up with Apache+mod_wsgi are documented in:

    http://code.google.com/p/modwsgi/wiki/Installat...

    Equally relevant if still using mod_python.
  • I think I managed to get a full 64bit python web stack on OS X 10.5.

    see here http://blog.captnswing.net/2009/04/19/python-mo...
  • Doug Napoleone
    NOTE: if you are building libraries for multiple versions of python on your Mac (mod_wsgi, psycopg2, your own 3rd party extension libraries) then you need to install the different versions of python into independent frameworks. This is done via configure when you build python, you specify an alternative name.

    This is because of the mess the framework system is with the custom gcc which comes with xcode. You can specify the framework (-Wlframework,Python) but you can not specify which VERSION f the framework to use as it will always used 'Current' with is a symlink to 2.5 or 2.6 depending on what was LAST installed.

    As a result I set the framework name to include the version number Python25 or Python26 respectively. Then I can specify which framework to use 'Python25' or 'Python26'.

    Further NOTE: this does NOT work for your binary releases as they then expect that specific Framework name :-(

    For that I end up having to have multiple Library directories, which is an even bigger hassle.

    If anyone has any information on making this work better for building binary module for python for multiple versions of python on a single mac, I would greatly appreciate it.
  • Interesting; this I did not know (although, it's out of my particular problem domain) - if I find the time I might dig into it a bit more and see if there's anything to be done.

    Another thought is to outline it in detail and add it to the python bugtracker, see if an enhancement can be made on our end.
  • Doug Napoleone
    To be honest this is all just a workaround to a problem with the apple supplied GCC. I have already filed a bug there, but not sure what will happen. There is no reason why you should not be able to tell the compiler which version of the framework to compile against.
  • Nat
    small point, but isn't the default Leopard Python framework actually installed in /System/Library, rather than /Library?
  • thumper:~ jesse$ python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"
    /Library/Python/2.5/site-packages

    Not anymore it seems
  • Nat
    ok, I'm not totally crazy. The binaries and standard modules and such are in /System/Library/Frameworks/Python.framework/Versions/2.5/
  • Amended
  • thanks for this, Jesse. Is your version of Python 2.6 a 64-bit executable? I've been having trouble getting it to compile as a 64-bit app (followed whatever directions I could find, looked straightforward enough but no 64-bit app at the other end).
  • Nope, by default the installed version of python *and* the one from the website are in 32 bit mode.
  • yeah, ok. I was asking because I was trying to set up a python web stack (apache 2 + mod_wsgi + python) and was having trouble getting a 64-bit apache (the stock install in Leopard) to play nice with a 32-bit python -- i did get it sorted out, but the experience has left me wondering if there was a better way. :)
  • Brilliant! Thanks very much for this. I've been using my default installed environment, and have had a series of niggling doubts about how I've been (ab)using things. You've clearly spelled out the process for setting things up with proper separation. I'll be upgrading to a new notebook in a couple of months, and will be using your recipe.
  • Let me know if you have any issues!
blog comments powered by Disqus