Skip to content

Django, mod_wsgi, Apache and OS X – do it.

whut_2.jpgSo, I’m one of those people where I don’t like running things “too far” from what a production setup might look like (I code on OS/X, deploy to Linux). This is why I jump(ed) through various hoops on my OS X system to get Apache/Django/mod_wsgi/etc all up and running and happy (not for serving the site; just developing).

Since I like simple/succinct guides, I thought I’d post what I did so others can follow in my stead.

Note: These instructions work with the python 2.5 version which ships with Leopard, or a self-compiled version of 2.6 (which is what I prefer) – see the InstallationOnMacOSX mod_wsgi page. Additionally, see the “Missing Code For Architecture” section for possible work-arounds if you find yourself needing 32 bit execution of Apache; I think the “Forcing 32 Bit Execution” are preferred over the “thinning” of the Apache binary.

First, download and install mod_wsgi on leopard, this is as easy as (on Leopard):

curl -o mod_wsgi.tgz http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar -xzf mod_wsgi.tgz
cd mod_wsgi-2.5
./configure
make
sudo make install

Now, edit (via sudo) /etc/apache2/httpd.conf and add the line:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

After the rest of the LoadModule lines. Cool.

Invariably all of my directions play with virtualenv/virtualenvwrapper and pip:

mkvirtualenv django
cdvirtualenv
easy_install pip
pip install http://media.djangoproject.com/releases/1.1/Django-1.1-rc-1.tar.gz
django-admin.py startproject mysite
django-admin.py startapp myapp
cd mysite
mkdir apache
mkdir media

Now, that just sets up the skeleton – the meat of the wsgi configuration goes in apache/ in the mysite/apache directory. The first file is named mysite.wsgi:

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This does the needed wsgi project magic for the Django application – don’t worry about the interpreter path; we’ll do that next.

Next up is a file named apache_django_wsgi.conf, this looks like this:

# mod_wsgi configuration directives - I like having stdout access, the other two
# options run mod_wsgi in daemon mode - more on this in a minute.
WSGIPythonHome /<path to virtualenv>
WSGIRestrictStdout Off
WSGIDaemonProcess django
WSGIProcessGroup django

#
# This should be the path of the /mysite/media directory
# for example "/Users/jesse/mysite/media/"
#
Alias /site_media/ "<PATH TO>/mysite/media/"
<Directory "<PATH TO>/mysite/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Directory path to the admin media, for example:
#

Alias /media/ "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media/"
<Directory "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Path to the mysite.wsgi file, for example:
# "/Users/jesse/mysite/apache/mysite.wsgi"
#

WSGIScriptAlias / "<PATH TO>/mysite/apache/mysite.wsgi"

<Directory "<PATH TO>/mysite/apache">
Allow from all
</Directory>

The apache_django_wsgi.conf file is the meat-and-potatoes here. This sets up all the paths/permissions, and is in Apache httpd.conf format. You can pretty much logjam any apache configuration directive here that you like.

Your final step is to once again edit (via sudo) /etc/apache2/httpd.conf and add a line like this at the verrrrry bottom:

Include "/path to/mysite/apache/apache_django_wsgi.conf"

And then run “sudo apachectl restart”

You should now be able to hit http://127.0.0.1/ and see the friendly and inviting django welcome page. Note, that if you are using sqlite as your database, you should chmod a+rw the file, so that processes which are not you can mess with it.

There’s a final piece to this though. Normally, if you run mod_wsgi in embedded mode, you’re going to need to restart apache every single time you make a change to your django app.

Ah! But we’re running in daemon mode. This means all you need to do when you change a file is:

touch mysite/apache/mysite.wsgi

This will trigger a reload and magic happens. Me being as lazy as I am (ask my wife) ended up snagging Bruno Bord’s tdaemon script, and hacking it up a bit. The tdaemon script will watch a directory and run tests. Well, I wanted it to watch a directory (and let me filter sub directories) and then run that touch command. So I reused my watcher.py (here) – I used this to monitor my sphinx tree and run builds as well (and other stuff). Here’s how I’d use this:

workon django
cdvirtualenv
cd mysite
python ~/.slash/bin/watcher.py --command "touch apache/mysite.wsgi" -f media

This will auto-fire the touch command whenever it detects a file change (including svn updates).

You can also do this another way
In my rush to reuse a tool I use a bit (watcher) I skipped past the mod_wsgi document section on code reloading that shows how to setup a monitor which will watch .py file changes and kill the wsgi daemon, here. If you scroll down a bit, you’ll see the “Monitoring For Code Changes” section. All you need to do here is copy the code from the wiki into a module on your PYTHONPATH – in my case, I wrote it to mysite/apache/wsgi_monitor.py (just for this example! you should put it someplace else!) and then changed the mysite.wsgi file to import it, and set it up:

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
sys.path.append(apache_configuration) # you probably shouldn't do this.
import wsgi_monitor
wsgi_monitor.start(interval=1.0)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'ui.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This method – once you reload apache – will watch the project for changes and then kill the wsgi daemon (forces a reload). So there you go – two ways of doing it.

The nice thing about this setup is that I can make production version of the wsgi scripts and check them in, but keep local “my copies” (ala local_settings.py) additionally, I don’t have to jump through hoops to get static media and content served up via the django development server.

Additional reading:

  • The mod_wsgi wiki page http://code.google.com/p/modwsgi/wiki/Reloading... has a section on setting up monitoring in daemon mode. No messy touching a file needed, It Just Works (tm).
  • That example does much the same thing, it watches the timestamps (instead of the hashes) and then kills the wsgi daemon process. 6 of one way, half a dozen of the other. In my case, I prefer my method above given I already had watcher.py done (for my sphinx stuff).

    I added a note about it in my post, thank for reminding me
  • Unfortunately, you will quickly realize that this setup is not viable on Mac OS X. The issue lies in the way the Python sub-processes are created and the fact a forked process can no longer use most Mac OS X frameworks because its Mach ports have all been invalidated.

    Many Python modules on Mac OS X make use of frameworks. For example, urllib uses Carbon to read system proxy settings. The moment would use urllib, the Python process will abort().

    The only reliable method I have found to run Python applications on Mac OS X is via FastCGI, with the a single threaded (for what its worth....) Python server process launched independently (preferably via launchd).
  • Hmm? This is working for me just fine - so far my app has not hit any issues with this, but then again, I'm not using a framework build of python, nor am I using urllib.

    Edit: I just chucked 10 urllib.urlopen() calls into the views.py of my app and swapped to python 2.5.2 which ships on leopard; and it doesn't crash. Also, if you're right, then multiprocessing (which is a fork()ed process) on OS/X can not use anything which could call a framework API.
  • You might be getting lucky on it, or actually found a configuration that works (in which case, awesome!). But I've definitely had issues on Leopard with urllib[2] with mod_wsgi which forced me to switch to FCGI.

    This merits more investigation I suppose.
  • I'm interested in what your configuration was that it broke - I imagine Graham Dumpleton (the mod_wsgi maintainer) would be as well.
  • grahamd
    The issue with __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ is not a mod_wsgi problem. It is caused by running mod_php in the same Apache and using a PHP module which links in Carbon libraries. One such example is the phpcups.so module which links to CoreFoundation framework. See 'http://groups.google.com/group/modwsgi/browse_thread/thread/fa1ce8c9a9e45db1'.
  • grahamd
    Sorry if this is a dup. Looks like the system ate my comment because I didn't have an account. Lost the full message I had posted. Rather than type in again what I said, see:

    http://groups.google.com/group/modwsgi/browse_t...

    The __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ message is not a mod_wsgi problem but is caused by PHP modules loaded in mod_php into the same Apache.
  • Nope it didn't eat it - I just moderate all comments
  • grahamd
    Cool. I don't recollect the account sign up stuff that intruded into the work flow saying that the comment had still been accepted and was sent for moderation.
  • Chris F
    According to this (http://code.google.com/p/modwsgi/wiki/Installat...) there are issues with 32 vs 64-bit executables. It would certainly be nice if it just worked. "Thinning" does not look very appealing.

    I'm anxious for more info.
  • Chris, I had to do no such thinning of the executables. I ran the
    directions above, and it works. The line right above that states "Note: This issue is fixed in mod_wsgi vesion 2.0. It will backported to version 1.4 if such a version needs to be created."
  • using the Apache shipped with Leopard and a Framework install of 2.6, I did have to thin httpd.
  • Bleargh. I just installed the python.org version of 2.6 and slammed into the "Non Universal MacPython" issue. Back to my self-compiled version. I'll add notes to the post about the various issues.

    Did you try forcing apache into 32bit mode with the "Forcing 32 Bit Execution" instructions on the site?
  • Nice find with the wsgi_monitor. Otherwise, I believe I already had pretty much the same setup on my Ubuntu box. Nice to see setting up Apache / Django is becoming a little more standard. :)
  • Trying to get this working has gotten me to pretty much hate mod_wsgi. I followed these directions, then had to deal with "Permission denied Errno 13" errors referring to the location of the python egg cache. Once I figured out a workaround for that, the issues immediately shifted to architecture related nonsense. I'm using the same python installation that I use for all of my non-web-related python development, so why are these issues only popping up when I introduce mod_wsgi into the equation? I thought we were all supposed to accept mod_wsgi as our saviour for implementing a "simple to use Apache module" for Python web apps? This is not simple.

    One other question: what are the major differences between using mod_wsgi for the dev work and using the django dev server? Is that documented somewhere?

    Sorry man. Don't mean to rant on your parade. Good post, as usual.
  • Hey Brian-

    I have no idea what you're talking about with regards to the python egg cache. I also didn't run into architecture related issues; so I'd be interested to hear what those are. Are you hand compiling it? Is it the built in version? I know Graham has outlined some of the issues here: http://code.google.com/p/modwsgi/wiki/Installat... - the biggest one is the fact the python.org versions of python don't ship 64 bit compatible.

    Frankly, most of the issues I've run into are problems with the multi-architecture mess which is leopard+python builds - not mod_wsgi, mod_wsgi just shows the problems. Having a 64 bit apache, and a 32 bit version of Python (which python.org ships) doesn't exactly make things easy for mod_wsgi which has to work with both.

    As for the dev vs. production issue - I run the same config on the server (+ or - minus some tweaks) as I do on the desktop.

    I can understand your pain, which is why I tried to be explicit as possible. Have you talked to Graham (the mod_wsgi maintainer) about any of this?
  • re: egg cache -- after firing things up the first time, I got an error that ended like this:

    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The following error occurred while trying to extract file(s) to the Python egg
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] cache:
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] [Errno 13] Permission denied: '"'
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The Python egg cache directory is currently set to:
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] "/.python-eggs/"
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] Perhaps your account does not have write access to this directory? You can
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] change the cache directory by setting the PYTHON_EGG_CACHE environment
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] variable to point to an accessible directory.

    I fixed that by adding a "user=myuser" arg to the WSGIDaemonProcess directive (this after also creating 'chmod 777' dirs owned by the web user, which didn't work, and which I still don't understand). I also tried fixing this using os.environ('PYTHON_EGG_CACHE') in my myapp.wsgi script, and also using the WSGIPythonEggs directive mentioned here: http://code.google.com/p/modwsgi/wiki/Applicati... Nothing worked until I found "user=". BTW, I'm using mod_wsgi 2.5, installed (and dl'd for that matter) using your exact commands, because I was pretty sure going into it that we had near-identical macbook pros.

    I'm using the Apple-supplied python installation. I initially created my virtualenv using your instructions exactly, but went back to add '--no-site-packages' when I started having issues loading mysqldb. Then I actually just uninstalled mysqldb from the system, did the "export ARCHFLAGS" like the modwsgi site says to do to get around arch issues, built a new version of the module from source, and now I'm getting:
    "[Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
    [Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] ImproperlyConfigured: Error loading MySQLdb module: dynamic module does not define init function (init_mysql)
    "
    I guess it's progress, but again, hardly simple. I haven't finished googling the issue to see what the fix is. It's v. 1.2.3c1 of the module. I guess it's true that this is perhaps mostly an arch-level issue outside the control of modwsgi, but why is it that I can write CLI scripts with Python all day long and never run into any of these issues? I'm not blaming modwsgi or questioning what you've said, I really just want to understand this more clearly, because it's driving me insane, and that's not enjoyable.

    I haven't given much thought to arch up to now. Perhaps it'd be easier to just build *everything* from source: apache, python, etc. Have a bare-bones base python install, and just easy_install modules in virtualenv as needed. Advice/opinion hereby solicited. I'm not sure how to reach Graham - I'm sure there's a mod_wsgi google group, so I'll see what I find.
  • grahamd
    WSGIPythonEggs directive only applies to embedded mode and you are using daemon mode. For daemon mode you use python-eggs option to WSGIDaemonProcess. This is explicitly mentioned in the available documentation for the configuration directives. You can also set os.environ, but either way, Apache normally runs as a special user and so is still going to need write access to the directory you specify. As you find, just easier to use daemon mode and set the user as yourself.

    The whole reason the 32/64 bit issues comes up is because 'python' command line program is only installed as 32 bit and so everything runs as 32 bit and when people are using it initially don't see any problems. The MacOS X version of Apache however is a 64 bit capable application and so will default to running 64 bit if you have 64 bit cpu. The distutils package in Python though only builds 32 bit extensions, so unless third party packages have been made MacOS X aware in some way to get around that problem, they will fail to build for all architectures. Some times the simplest thing to do is to install your own version of Apache from source code, as it will only be 32 bit and thus avoid all the problems.

    Pretty well all the issues are documented some where as is a reference to mailing list for mod_wsgi. Of course, getting people to read documentation and asking questions on the correct forum is the challenge. :-)
  • By the way Graham; setting apache/python/mod_wsgi up with the default python2.6 interpreter was dirt simple.
blog comments powered by Disqus