Django, mod_wsgi, Apache and OS X — do it.

July 24th, 2009 § 25 comments

whut_2.jpgSo, I’m one of those peo­ple where I don’t like run­ning things “too far” from what a pro­duc­tion setup might look like (I code on OS/X, deploy to Linux). This is why I jump(ed) through var­i­ous hoops on my OS X sys­tem to get Apache/Django/mod_wsgi/etc all up and run­ning and happy (not for serv­ing the site; just developing).

Since I like simple/succinct guides, I thought I’d post what I did so oth­ers can fol­low in my stead.

Note: These instruc­tions work with the python 2.5 ver­sion which ships with Leop­ard, or a self-compiled ver­sion of 2.6 (which is what I pre­fer) — see the Instal­la­tionOn­Ma­cOSX mod_wsgi page. Addi­tion­ally, see the “Miss­ing Code For Archi­tec­ture” sec­tion for pos­si­ble work-arounds if you find your­self need­ing 32 bit exe­cu­tion of Apache; I think the “Forc­ing 32 Bit Exe­cu­tion” are pre­ferred over the “thin­ning” of the Apache binary.

First, down­load and install mod_wsgi on leop­ard, this is as easy as (on Leopard):

curl -o mod_wsgi.tgz http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar -xzf mod_wsgi.tgz
cd mod_wsgi-2.5
./configure
make
sudo make install

Now, edit (via sudo) /etc/apache2/httpd.conf and add the line:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

After the rest of the Load­Mod­ule lines. Cool.

Invari­ably all of my direc­tions play with virtualenv/virtualenvwrapper and pip:

mkvirtualenv django
cdvirtualenv
easy_install pip
pip install http://media.djangoproject.com/releases/1.1/Django-1.1-rc-1.tar.gz
django-admin.py startproject mysite
django-admin.py startapp myapp
cd mysite
mkdir apache
mkdir media

Now, that just sets up the skele­ton — the meat of the wsgi con­fig­u­ra­tion goes in apache/ in the mysite/apache direc­tory. The first file is named mysite.wsgi:

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This does the needed wsgi project magic for the Django appli­ca­tion — don’t worry about the inter­preter path; we’ll do that next.

Next up is a file named apache_django_wsgi.conf, this looks like this:

# mod_wsgi configuration directives - I like having stdout access, the other two
# options run mod_wsgi in daemon mode - more on this in a minute.
WSGIPythonHome /<path to virtualenv>
WSGIRestrictStdout Off
WSGIDaemonProcess django
WSGIProcessGroup django

#
# This should be the path of the /mysite/media directory
# for example "/Users/jesse/mysite/media/"
#
Alias /site_media/ "<PATH TO>/mysite/media/"
<Directory "<PATH TO>/mysite/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Directory path to the admin media, for example:
#

Alias /media/ "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media/"
<Directory "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Path to the mysite.wsgi file, for example:
# "/Users/jesse/mysite/apache/mysite.wsgi"
#

WSGIScriptAlias / "<PATH TO>/mysite/apache/mysite.wsgi"

<Directory "<PATH TO>/mysite/apache">
Allow from all
</Directory>

The apache_django_wsgi.conf file is the meat-and-potatoes here. This sets up all the paths/permissions, and is in Apache httpd.conf for­mat. You can pretty much log­jam any apache con­fig­u­ra­tion direc­tive here that you like.

Your final step is to once again edit (via sudo) /etc/apache2/httpd.conf and add a line like this at the ver­rrrry bottom:

Include "/path to/mysite/apache/apache_django_wsgi.conf"

And then run “sudo apachectl restart”

You should now be able to hit http://127.0.0.1/ and see the friendly and invit­ing django wel­come page. Note, that if you are using sqlite as your data­base, you should chmod a+rw the file, so that processes which are not you can mess with it.

There’s a final piece to this though. Nor­mally, if you run mod_wsgi in embed­ded mode, you’re going to need to restart apache every sin­gle time you make a change to your django app.

Ah! But we’re run­ning in dae­mon mode. This means all you need to do when you change a file is:

touch mysite/apache/mysite.wsgi

This will trig­ger a reload and magic hap­pens. Me being as lazy as I am (ask my wife) ended up snag­ging Bruno Bord’s tdae­mon script, and hack­ing it up a bit. The tdae­mon script will watch a direc­tory and run tests. Well, I wanted it to watch a direc­tory (and let me fil­ter sub direc­to­ries) and then run that touch com­mand. So I reused my watcher.py (here) — I used this to mon­i­tor my sphinx tree and run builds as well (and other stuff). Here’s how I’d use this:

workon django
cdvirtualenv
cd mysite
python ~/.slash/bin/watcher.py --command "touch apache/mysite.wsgi" -f media

This will auto-fire the touch com­mand when­ever it detects a file change (includ­ing svn updates).

You can also do this another way
In my rush to reuse a tool I use a bit (watcher) I skipped past the mod_wsgi doc­u­ment sec­tion on code reload­ing that shows how to setup a mon­i­tor which will watch .py file changes and kill the wsgi dae­mon, here. If you scroll down a bit, you’ll see the “Mon­i­tor­ing For Code Changes” sec­tion. All you need to do here is copy the code from the wiki into a mod­ule on your PYTHONPATH — in my case, I wrote it to mysite/apache/wsgi_monitor.py (just for this exam­ple! you should put it some­place else!) and then changed the mysite.wsgi file to import it, and set it up:

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
sys.path.append(apache_configuration) # you probably shouldn't do this.
import wsgi_monitor
wsgi_monitor.start(interval=1.0)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'ui.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This method — once you reload apache — will watch the project for changes and then kill the wsgi dae­mon (forces a reload). So there you go — two ways of doing it.

The nice thing about this setup is that I can make pro­duc­tion ver­sion of the wsgi scripts and check them in, but keep local “my copies” (ala local_settings.py) addi­tion­ally, I don’t have to jump through hoops to get sta­tic media and con­tent served up via the django devel­op­ment server.

Addi­tional reading:

  • http://spookypony.com/ Peter Hern­don

    The mod_wsgi wiki page http://code.google.com/p/modwsgi/wiki/Reloading… has a sec­tion on set­ting up mon­i­tor­ing in dae­mon mode. No messy touch­ing a file needed, It Just Works ™.

  • http://spookypony.com Peter Hern­don

    Gra­ham addresses dynamic code reload­ing on the mod_wsgi wiki, http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode — look for the sec­tion Mon­i­tor­ing For Code Changes. No need to touch files.

  • http://openid.devklog.net/jeanfrancois Jean-Francois Roy

    Unfor­tu­nately, you will quickly real­ize that this setup is not viable on Mac OS X. The issue lies in the way the Python sub-processes are cre­ated and the fact a forked process can no longer use most Mac OS X frame­works because its Mach ports have all been invalidated.

    Many Python mod­ules on Mac OS X make use of frame­works. For exam­ple, url­lib uses Car­bon to read sys­tem proxy set­tings. The moment would use url­lib, the Python process will abort().

    The only reli­able method I have found to run Python appli­ca­tions on Mac OS X is via FastCGI, with the a sin­gle threaded (for what its worth.…) Python server process launched inde­pen­dently (prefer­ably via launchd).

  • http://jessenoller.com jnoller

    Hmm? This is work­ing for me just fine — so far my app has not hit any issues with this, but then again, I’m not using a frame­work build of python, nor am I using urllib.

  • http://jessenoller.com jnoller

    That exam­ple does much the same thing, it watches the time­stamps (instead of the hashes) and then kills the wsgi dae­mon process. 6 of one way, half a dozen of the other. In my case, I pre­fer my method above given I already had watcher.py done (for my sphinx stuff).

    I’ll look at this and prob­a­bly post an update to add in a note about that

  • http://openid.devklog.net/jeanfrancois Jean-Francois Roy

    You might be get­ting lucky on it, or actu­ally found a con­fig­u­ra­tion that works (in which case, awe­some!). But I’ve def­i­nitely had issues on Leop­ard with urllib[2] with mod_wsgi which forced me to switch to FCGI.

    This mer­its more inves­ti­ga­tion I suppose.

  • Chris F

    Accord­ing to this (http://code.google.com/p/modwsgi/wiki/Installat…) there are issues with 32 vs 64-bit exe­cuta­bles. It would cer­tainly be nice if it just worked. “Thin­ning” does not look very appealing.

    I’m anx­ious for more info.

  • http://jessenoller.com jnoller

    Chris, I had to do no such thin­ning of the exe­cutable. I ran the
    direc­tions above, and it works

  • http://jessenoller.com jnoller

    I’m inter­ested in what your con­fig­u­ra­tion was that it broke — I imag­ine Gra­ham Dumple­ton (the mod_wsgi main­tainer) would be as well.

  • http://helveticascenario.net/ Nat Williams

    using the Apache shipped with Leop­ard and a Frame­work install of 2.6, I did have to thin httpd.

  • http://jessenoller.com jnoller

    Was this with wsgi 2.0 or an ear­lier version?

  • gra­hamd

    The issue with __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ is not a mod_wsgi prob­lem. It is caused by run­ning mod_php in the same Apache and using a PHP mod­ule which links in Car­bon libraries. One such exam­ple is the phpcups.so mod­ule which links to Core­Foun­da­tion frame­work. See ‘http://groups.google.com/group/modwsgi/browse_thread/thread/fa1ce8c9a9e45db1&#39;.

  • gra­hamd

    Sorry if this is a dup. Looks like the sys­tem ate my com­ment because I didn’t have an account. Lost the full mes­sage I had posted. Rather than type in again what I said, see:

    http://groups.google.com/group/modwsgi/browse_t…

    The __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__ mes­sage is not a mod_wsgi prob­lem but is caused by PHP mod­ules loaded in mod_php into the same Apache.

  • http://jessenoller.com jnoller

    Nope it didn’t eat it — I just mod­er­ate all comments

  • gra­hamd

    Cool. I don’t rec­ol­lect the account sign up stuff that intruded into the work flow say­ing that the com­ment had still been accepted and was sent for moderation.

  • http://twitter.com/PKKid Michael Shep­an­ski

    Nice find with the wsgi_monitor. Oth­er­wise, I believe I already had pretty much the same setup on my Ubuntu box. Nice to see set­ting up Apache / Django is becom­ing a lit­tle more standard. :)

  • Pingback: Josh Mather » Bookmarks for July 6th through August 11th

  • http://www.protocolostomy.com/ Brian

    Try­ing to get this work­ing has got­ten me to pretty much hate mod_wsgi. I fol­lowed these direc­tions, then had to deal with “Per­mis­sion denied Errno 13″ errors refer­ring to the loca­tion of the python egg cache. Once I fig­ured out a workaround for that, the issues imme­di­ately shifted to archi­tec­ture related non­sense. I’m using the same python instal­la­tion that I use for all of my non-web-related python devel­op­ment, so why are these issues only pop­ping up when I intro­duce mod_wsgi into the equa­tion? I thought we were all sup­posed to accept mod_wsgi as our sav­iour for imple­ment­ing a “sim­ple to use Apache mod­ule” for Python web apps? This is not simple.

    One other ques­tion: what are the major dif­fer­ences between using mod_wsgi for the dev work and using the django dev server? Is that doc­u­mented somewhere?

    Sorry man. Don’t mean to rant on your parade. Good post, as usual.

  • http://jessenoller.com jnoller

    Hey Brian–

    I have no idea what you’re talk­ing about with regards to the python egg cache. I also didn’t run into archi­tec­ture related issues; so I’d be inter­ested to hear what those are. Are you hand com­pil­ing it? Is it the built in ver­sion? I know Gra­ham has out­lined some of the issues here: http://code.google.com/p/modwsgi/wiki/Installat… — the biggest one is the fact the python.org ver­sions of python don’t ship 64 bit compatible.

    Frankly, most of the issues I’ve run into are prob­lems with the multi-architecture mess which is leopard+python builds — not mod_wsgi, mod_wsgi just shows the prob­lems. Hav­ing a 64 bit apache, and a 32 bit ver­sion of Python (which python.org ships) doesn’t exactly make things easy for mod_wsgi which has to work with both.

    As for the dev vs. pro­duc­tion issue — I run the same con­fig on the server (+ or — minus some tweaks) as I do on the desktop.

    I can under­stand your pain, which is why I tried to be explicit as pos­si­ble. Have you talked to Gra­ham (the mod_wsgi main­tainer) about any of this?

  • http://www.protocolostomy.com/ Brian

    re: egg cache — after fir­ing things up the first time, I got an error that ended like this:

    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The fol­low­ing error occurred while try­ing to extract file(s) to the Python egg
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] cache:
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] [Errno 13] Per­mis­sion denied: ‘“‘
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] The Python egg cache direc­tory is cur­rently set to:
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] “/.python-eggs/“
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1]
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] Per­haps your account does not have write access to this direc­tory? You can
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] change the cache direc­tory by set­ting the PYTHON_EGG_CACHE envi­ron­ment
    [Sat Aug 15 23:53:29 2009] [error] [client 127.0.0.1] vari­able to point to an acces­si­ble directory.

    I fixed that by adding a “user=myuser” arg to the WSGI­Dae­mon­Process direc­tive (this after also cre­at­ing ‘chmod 777′ dirs owned by the web user, which didn’t work, and which I still don’t under­stand). I also tried fix­ing this using os.environ(‘PYTHON_EGG_CACHE’) in my myapp.wsgi script, and also using the WSGIPythonEggs direc­tive men­tioned here: http://code.google.com/p/modwsgi/wiki/Applicati… Noth­ing worked until I found “user=”. BTW, I’m using mod_wsgi 2.5, installed (and dl’d for that mat­ter) using your exact com­mands, because I was pretty sure going into it that we had near-identical mac­book pros.

    I’m using the Apple-supplied python instal­la­tion. I ini­tially cre­ated my vir­tualenv using your instruc­tions exactly, but went back to add ‘–no-site-packages’ when I started hav­ing issues load­ing mysqldb. Then I actu­ally just unin­stalled mysqldb from the sys­tem, did the “export ARCHFLAGS” like the mod­wsgi site says to do to get around arch issues, built a new ver­sion of the mod­ule from source, and now I’m get­ting:
    “[Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] raise ImproperlyConfigured(“Error load­ing MySQLdb mod­ule: %s” % e)
    [Sun Aug 16 13:23:49 2009] [error] [client 127.0.0.1] Improp­er­ly­Con­fig­ured: Error load­ing MySQLdb mod­ule: dynamic mod­ule does not define init func­tion (init_mysql)
    “
    I guess it’s progress, but again, hardly sim­ple. I haven’t fin­ished googling the issue to see what the fix is. It’s v. 1.2.3c1 of the mod­ule. I guess it’s true that this is per­haps mostly an arch-level issue out­side the con­trol of mod­wsgi, but why is it that I can write CLI scripts with Python all day long and never run into any of these issues? I’m not blam­ing mod­wsgi or ques­tion­ing what you’ve said, I really just want to under­stand this more clearly, because it’s dri­ving me insane, and that’s not enjoyable.

    I haven’t given much thought to arch up to now. Per­haps it’d be eas­ier to just build *every­thing* from source: apache, python, etc. Have a bare-bones base python install, and just easy_install mod­ules in vir­tualenv as needed. Advice/opinion hereby solicited. I’m not sure how to reach Gra­ham — I’m sure there’s a mod_wsgi google group, so I’ll see what I find.

  • gra­hamd

    WSGIPythonEggs direc­tive only applies to embed­ded mode and you are using dae­mon mode. For dae­mon mode you use python-eggs option to WSGI­Dae­mon­Process. This is explic­itly men­tioned in the avail­able doc­u­men­ta­tion for the con­fig­u­ra­tion direc­tives. You can also set os.environ, but either way, Apache nor­mally runs as a spe­cial user and so is still going to need write access to the direc­tory you spec­ify. As you find, just eas­ier to use dae­mon mode and set the user as yourself.

    The whole rea­son the 32/64 bit issues comes up is because ‘python’ com­mand line pro­gram is only installed as 32 bit and so every­thing runs as 32 bit and when peo­ple are using it ini­tially don’t see any prob­lems. The MacOS X ver­sion of Apache how­ever is a 64 bit capa­ble appli­ca­tion and so will default to run­ning 64 bit if you have 64 bit cpu. The dis­tu­tils pack­age in Python though only builds 32 bit exten­sions, so unless third party pack­ages have been made MacOS X aware in some way to get around that prob­lem, they will fail to build for all archi­tec­tures. Some times the sim­plest thing to do is to install your own ver­sion of Apache from source code, as it will only be 32 bit and thus avoid all the problems.

    Pretty well all the issues are doc­u­mented some where as is a ref­er­ence to mail­ing list for mod_wsgi. Of course, get­ting peo­ple to read doc­u­men­ta­tion and ask­ing ques­tions on the cor­rect forum is the challenge. :-)

  • http://jessenoller.com jnoller

    By the way Gra­ham; set­ting apache/python/mod_wsgi up with the default python2.6 inter­preter was dirt simple.

  • gra­hamd

    WSGIPythonEggs direc­tive only applies to embed­ded mode and you are using dae­mon mode. For dae­mon mode you use python-eggs option to WSGI­Dae­mon­Process. This is explic­itly men­tioned in the avail­able doc­u­men­ta­tion for the con­fig­u­ra­tion direc­tives. You can also set os.environ, but either way, Apache nor­mally runs as a spe­cial user and so is still going to need write access to the direc­tory you spec­ify. As you find, just eas­ier to use dae­mon mode and set the user as yourself.

    The whole rea­son the 32/64 bit issues comes up is because ‘python’ com­mand line pro­gram is only installed as 32 bit and so every­thing runs as 32 bit and when peo­ple are using it ini­tially don’t see any prob­lems. The MacOS X ver­sion of Apache how­ever is a 64 bit capa­ble appli­ca­tion and so will default to run­ning 64 bit if you have 64 bit cpu. The dis­tu­tils pack­age in Python though only builds 32 bit exten­sions, so unless third party pack­ages have been made MacOS X aware in some way to get around that prob­lem, they will fail to build for all archi­tec­tures. Some times the sim­plest thing to do is to install your own ver­sion of Apache from source code, as it will only be 32 bit and thus avoid all the problems.

    Pretty well all the issues are doc­u­mented some where as is a ref­er­ence to mail­ing list for mod_wsgi. Of course, get­ting peo­ple to read doc­u­men­ta­tion and ask­ing ques­tions on the cor­rect forum is the challenge. :-)

  • http://jessenoller.com jnoller

    By the way Gra­ham; set­ting apache/python/mod_wsgi up with the default python2.6 inter­preter was dirt simple.

  • Pingback: Django, mod_wsgi, Apache and OS X | Open Serendipity

What's this?

You are currently reading Django, mod_wsgi, Apache and OS X — do it. at jessenoller.com.

meta