jessenoller.com

(photo cour­tesy of Sean MacEn­tee via flickr)

Some inter­est­ing bits and pieces (left­overs) from the week­end — I have a ten­dency to queue up a pile of “read later” stuff or email­ing myself a pile of things to talk about/link to/etc. Some­times, I actu­ally get to go through it all. Today — I get to share it to!

First up — Michael Foord (fuzzy­man) did two excel­lent posts — the first is on pri­vacy in Python — not big-P pri­vacy, but rather the programming/object pri­vacy. It’s a good read because it reen­forces the point that even if you think you’re being clever and hid­ing some­thing in a clo­sure to make it pri­vate, you can still get access to it. Remem­ber too — dun­ders (__foo) are just name-mangling. In Python, noth­ing is pri­vate (insert bad tast­ing joke about Python being face­book here).

Michael’s sec­ond post is on the Named­Tu­ple ker­fluffle that was stirred up Krist­jan Valur’s post on the use of exec() and named­tu­ple (short ver­sion: named­tu­ple cre­ates a string defin­ing the class and then calls exec to cre­ate the object. I think Michael is spot on — I think exec() gets a bad rap frankly, sure — it’s some­thing of a hand-grenade, if you use it wrong, you’re going to get hurt, but in this case I have to agree with Ray­mond in his com­ment on bug 3974 — the cur­rent imple­men­ta­tion is clear and main­tain­able. I don’t like the stern­ness of the reply — but he is right. Kristjan’s use-case is an inter­est­ing one, but I don’t think it’s one that war­rants the removal of the exist­ing imple­men­ta­tion. I’m won­der­ing if a “fall­back if exec doesn’t exist” is worth inclusion.

Then again, I’ve spo­ken to peo­ple who refuse to use named­tu­ples because they now know how the sausage is made. I still think the sausage is deli­cious.

Next is a pretty inter­est­ing dis­cus­sion on a pre­sen­ta­tion that came out of Pixar on get­ting over embar­rass­ment in order to get things done — I don’t have much to add above the com­ments on hacker news, except to say I think the same men­tal blocks they talk about for ani­ma­tors apply to pro­gram­mers, writ­ers, etc. We hide from code reviews, we hide our writ­ing until we think it’s “Pitch Per­fect” — when in real­ity, we shouldn’t. We should be shar­ing, dis­cussing and col­lab­o­rat­ing ear­lier, more fre­quently and more often.

Share and Ship early and often — see also “Don’t Go Dark”.

Finally, I was happy to see the “Quick and Easy Debug­ging in Python” post from Jeet Suku­maran — it’s always nice to evolve past sprin­kling print fairy dust all over your code for debug­ging — even if we all still do it despite know­ing or using PDB. Just to add to his post, if you want to increase your PDB-Fu, I rec­om­mend this “Debug­ging in Python” post by Steve Ferg, and the offi­cial doc­u­men­ta­tion for PDB. We should really add a HOWTO for this.

p.s. For addi­tional good-reading, check out “Pri­or­i­ties Don’t Exist in a Vac­uum” and “First Care”- while not ger­mane to what’s I’ve already writ­ten about, they’re a good essays on pri­or­i­ties. Thanks to the lat­est “Back to Work” pod­cast.

(photo cour­tesy of Sean MacEn­tee via flickr) Some inter­est­ing bits and pieces (left­overs) from the week­end — I have a ten­dency to queue up a pile of “read later” stuff...

On Fri­day of last week, a new post I wrote for my employer (Nasuni) went up — “Encryp­tion Keys, User Data and Sub­poe­nas”. In that post, I got to out­line, in clear “non slip­pery” lan­guage how my employer man­ages encryp­tion keys, what data they have access to, etc. One of my favorite quotes:

If a cus­tomer has pro­vided their own encryp­tion key(s) — Nasuni, or the cloud provider, do not have those keys, and can not pro­vide them as part of a sub­poena or other legal process. We can not decrypt or access your data. We can not sup­ply a key which we do not have. This is not pol­icy or trust level pro­tec­tion: It’s impossible.

We offer auto-generated and escrowed keys as a con­ve­nience to the user — the ben­e­fits of hav­ing this fea­ture out­weigh the cost. A user or com­pany who knows noth­ing about encryp­tion keys and key escrow can still have strong data secu­rity and instan­ta­neous dis­as­ter recov­ery, they can install a Filer in min­utes and imme­di­ately be up and running.

» Read the rest of this entry «

On Fri­day of last week, a new post I wrote for my employer (Nasuni) went up — “Encryp­tion Keys, User Data and Sub­poe­nas”. In that post, I got to out­line,...

The offi­cial announce­ment (well, the addi­tion of a web­site for it) of Google’s goo.gl URL short­en­ing service’s new web­site and fea­tures on the 11th got me think­ing really hard about com­pe­ti­tion, and Google. Specif­i­cally — how do you com­pete against the biggest tech­no­log­i­cal behe­moth ever seen by man? Some­thing I’m sure is on a lot of peo­ples’ minds at an increas­ing rate.

To be hon­est, many of these thoughts can prob­a­bly be applied to many incum­bents in the tech indus­try (includ­ing “enter­prise” software/hardware giants), Google is an easy tar­get for these thoughts though, because they are sim­ply so bad at some of this. This is part rant, part thought exper­i­ment – it’s entirely pos­si­ble I am entirely wrong.

What drove me to think­ing about this (for well over a week) is a base ter­ror I felt about the vague pos­si­bil­ity of being in a mar­ket Google might whim­si­cally enter at one point. Like, say I was bit.ly — and hap­pily the most pop­u­lar URL short­en­ing and ana­lyt­ics firm with thou­sands of cus­tomers, mil­lions of short­ened links, etc, etc. How would I feel if Google coughed and sud­denly entered an already tight (some would say arti­fi­cial) mar­ket with all salvos aimed right at my busi­ness (bit.ly seems game)? Can an ecosys­tem of star­tups sur­vive if Google pops into the room – can they still get VCs or Angel investors to lis­ten and invest in them? (See also: whatifgoogledoesit.com).

It’s not that Google sud­denly came out with a “bet­ter” thing then bit.ly — Google sim­ply came out with some­thing which “does the job” to the tech­ni­cal spec­i­fi­ca­tions they think are supe­rior, sit­ting on Google’s nearly unbeat­able infra­struc­ture and then threw the weight of their brand behind it.

Does it have all the pretty ana­lyt­ics Bit.ly has? No. Does it have cus­tom URLs? No. Does it need all of that? No, because it’s made by Google. The UI is per­fectly func­tional, but noth­ing to write home to mom about. Mil­lions of peo­ple will flock to the new ser­vice and hap­pily use it because it is Google. Bit.ly could very well now be on life sup­port, and will quickly run out of oxy­gen when/if Google ever decided to give pref­er­ence to goo.gl within their sites and appli­ca­tions (see the secu­rity argu­ment in the announce­ments – how long until the other short­en­ers are deemed “too insecure”?).

The very thought of this pos­si­bil­ity hap­pen­ing on some­thing I work on ter­ri­fies me. I’m pretty con­fi­dent on the tech­ni­cal prowess of the teams I work with and of the prod­ucts we make, but I’ll be damned if Google couldn’t wipe us out with a “prod­uct” with 25% of the fea­tures we have, sim­ply because of who they are. Maybe we could scratch by – maybe we already have an estab­lished user base. Maybe Google would kill their imple­men­ta­tion in a few months – who knows.

But Google has a flaw, sev­eral, in fact.

Com­pet­ing with Google on a tech­no­log­i­cal level is incred­i­bly hard — it’s not impos­si­ble, just hard. They have more PHDs and engi­neers per square foot than just about any­one. I think that breath­ing the Google­plex air alone prob­a­bly increases your IQ. I don’t know – I have some of the air on order. It’s easy for Google to build some­thing fifty per­cent of the way and release it, there­fore suck­ing the air out of the room. They don’t even need to “fin­ish” it — the very fact they’ve made it and put it every­where is enough to make a mar­ket dry up and users to flock to it. It will have enough func­tion­al­ity — and just enough — to get the job done (“per­fectly func­tional, albeit Spartan”).

Google is good at raw func­tion­al­ity and util­ity. They solve a prob­lem in nor­mally the most effi­cient way pos­si­ble, and Google is going to prob­a­bly go down as the most suc­cess­ful tech­nol­ogy com­pany in history.

Where Google fails — time and again — is being human.

No one invites Vul­cans from Star Trek to come and dec­o­rate cakes or enter­tain them at a party. No one accuses Vul­cans of hav­ing “really good empa­thy and cus­tomer ser­vice skills”. No, peo­ple call Vul­cans when they need to fig­ure out a hard prob­lem, or need some objec­tive analy­sis. They don’t expect bal­loon ani­mals and a Dora cake from them.

Google is a utility/commodity tech­nol­ogy com­pany (an exceed­ingly shrewd and pow­er­ful one) — but Android wins mar­ket share because it’s on more phones, not because the expe­ri­ence is bet­ter but sim­ply because it’s every­where – it’s on more and more phones every day. Plenty of the man­u­fac­tur­ers who have adopted it spent mil­lions design­ing UIs that sit on top of the default Android UI and make it “more friendly”. Every mar­ket they touch they fun­da­men­tally change the eco­nom­ics and expec­ta­tions of.

Google has become top dog for a rea­son — their tech­nol­ogy. It is really top notch and their search engine and adwords sys­tem changed the mar­ket (for the bet­ter), but it all shares the cold robotic embrace of the other Google prod­ucts. Their tech­ni­cal skills are beyond reproach, but they still lose in many cases against smaller, “richer” appli­ca­tions and sites because they fail at being human.

Expe­ri­ence Matters.

To Google; you are a sta­tis­ti­cal note — some­thing to be tracked, cat­e­go­rized and pro­filed. Why? Not though mal­ice or ill intent — not even slightly — rather, it is how they aim their real busi­ness at you: Adver­tis­ing. Google is not mali­cious, nor is it evil. Google is the log­i­cal robot who will tell you you’ve got can­cer while ask­ing for the time and not even blink. They con­tinue inter­est­ing projects which could change human­ity – but with the bed­side man­ner of a toaster (note though — the cold, cal­cu­lat­ing nature of the projects doesn’t dimin­ish the value).

When a com­pet­ing company’s users are sta­tis­tics: show those sta­tis­tics love and a human face and they will fol­low you to the ends of the earth. Incite pas­sion – give them a rela­tion­ship. A wise man once told me “the only way you can suc­ceed against an entrenched player is by lov­ing your users to death”.

Love your cus­tomers — say you make a code host­ing ser­vice — it’s hard to beat free (as is Google Code) and it’s hard to beat the fact that, yeah, they have all the basic fea­tures a code host should have — but you com­pete where Google can’t. You beat them in the User Inter­face depart­ment — you beat them with warm, invit­ing doc­u­men­ta­tion and a well designed, invit­ing web­site. You beat them by hir­ing a sup­port staff that actu­ally answers emails and picks up the phone. (See also: “Google Gets a C– from the Bet­ter Busi­ness Bureau”)

You com­pete against them by not being a cold, Spar­tan fea­ture robot. You make your thing usable, you make it pleas­ant. You make it so that users want to come back to you again and again because each time they do they don’t feel like they just got a hug from a Crafts­man work­bench. You make them feel like Mom just gave them a warm hug on a Christ­mas day every time they use your prod­uct. Not like mak­ing out with a socket set.

But, you say, Google can make a UI, right? Not quite — Google Wave may have been the best thing in the his­tory of earth: but no one except a few peo­ple could fig­ure out how to really use it. Tech­no­log­i­cally, it was awe­some, usabil­ity? Not so much. It was a bag of tech­ni­cally accu­rate fea­tures – but not a human inter­face. It was a “social net­work” put together by Vulcans.

The biggest thing, in my opin­ion, that Google has brought to the human side of the tech­no­log­i­cal table is that it has helped in recent years by bring­ing back a wave of min­i­mal­ism, sim­plic­ity of inter­face and speed to web appli­ca­tion design as a whole. In the right hands, min­i­mal­ism and sim­plic­ity are pow­er­ful tools. When they’re not in the right hands — well, hugs from a Crafts­man bench.

So — in order to com­pete with Google, you attack them on design – on engage­ment. You make your social fea­tures and good cus­tomer ser­vice into the barbs of loy­alty. You pick up that phone and let them know there’s some­one else at the end of the line will­ing to hear them out at 3am when every­thing goes to hell and they’re all alone. Even if that cus­tomer is crazy, you show them the respect they deserve as people.

Get vocal, pas­sion­ate users and build a loyal com­mu­nity — that alone will help you suc­ceed against Google. Make sure your cus­tomers know you love them, know that you sup­port them and want them to suc­ceed. Don’t just enable them to do some­thing, enable them to connect.

Build a brand against Google. Don’t be con­tent with doing some­thing — make sure you’re not just “the guys that did that thing” — or “those guys who came out with that thing”. Make your name syn­ony­mous with that thing. Make it so that the first thing peo­ple think of when con­sid­er­ing that thing is you.

Make it so that Google could come out with — say a video shar­ing site — tomor­row, and while it could be the best, most dis­trib­uted video thing ever (the bet­ter tech­no­log­i­cal choice) make it so that your users are so fiercely loyal that Google has to buy you and extin­guish the flames of the pas­sion you’ve incited just to get the announce­ment for their new thing two min­utes of air time.

You can only do these things — build­ing a brand — and build­ing a “cult” by doing the things Google — given its robotic fail­ings — can­not do. Love your users, infect them with your pas­sion — not just your tech­ni­cal prowess or abil­ity to scale or release new web codecs, or give them the right search results, or giv­ing away source – infect them with your pas­sion for what you do. Sup­port them, respond to them — even if you’re giv­ing it away for free — after all, noth­ing is free.

Pas­sion, com­pas­sion — con­nect­ing with other humans, peo­ple are always look­ing for a place that accepts them and makes them feel wel­come. They want to get real sup­port instead of emails that get sent to unknown voids and are never answered. Mak­ing things warm, invit­ing both in lan­guage and in the feel.

Just remem­ber — Google is a fan­tas­tic, nearly unbeat­able tech­ni­cal pow­er­house. You’ve got to be fast, high qual­ity and bet­ter where it counts the most.

“What about Don’t Be Evil?”, you say. Again, this is not an accu­sa­tion of Google being evil – they’re not. They’re being coldly log­i­cal in the way humans deal­ing with other humans aren’t. When Eric Schmidt, the CEO, stands up on stage and talks about pri­vacy being dead in the age we live in (the age of Face­book, and Twit­ter), or the refuge for crim­i­nals – he’s not “being evil”. He’s rep­re­sent­ing the coldly log­i­cal, algo­rithm based view of a search engine, and adver­tise­ment com­pany. (Check out duckduckgo.com)

In the age of blogs, Face­book, Twit­ter, MySpace and online med­ical records and a mil­lion other things, the log­i­cal exten­sion is that, yes – pri­vacy will be dead in a mat­ter of years. Look at the train wreck the buzz roll­out was – they shipped with the log­i­cal, auto-following and auto-public set­tings and features.

How­ever, nuk­ing years of email or delist­ing someone’s web­site with no human recourse is evil, and there­fore, can be used as a com­pet­i­tive advan­tage against them. Be more pri­vate, be avail­able to your cus­tomers. I know it’s expen­sive – but it’s how you can win. First mover advan­tage counts for a lot, but it doesn’t count for any­thing if you fail your com­mu­nity and users.

I use lots — and I do mean lots — of Google projects. I live in the lap of Google lux­ury as they give me free things that have “enough” fea­tures to sate my needs and require­ments. They’re pretty enough — sort of like my code edi­tor. I’m not pas­sion­ate about them, they’re func­tional util­i­ties (albeit incred­i­bly use­ful ones) — and at this point I’d prob­a­bly been inop­er­a­ble with­out a few of them. Google is a verb — JGI (Just Google it) leaves my mouth an innu­mer­able num­ber of times through the day. I have lots of friends who work at Google. Google has released an amaz­ing amount of open source soft­ware, and con­tinue to work on chang­ing the face of the Inter­net, and soci­ety as a whole.

But would I say their UIs are beau­ti­ful? No. Would I ever be con­vinced that send­ing an email about my account being bro­ken or dis­abled to Google’s sup­port line would be met with any­thing but metal­lic robot silence? Do I think pleas to relist my web­site in their index or rein­state an adwords account would be any more effec­tive then yelling at my garbage dis­posal? No.

No, none of these are true. Github (despite it being git) and bit­bucket are the bet­ter UIs for code host­ing — Word­Press and oth­ers are bet­ter hosted blog­ging sys­tems then Blog­ger, and so on, and so on. These ser­vices prob­a­bly don’t scale as well, or they can’t cal­cu­late the veloc­ity of an unladen swal­low if you hit control-m-x-y-*, but they com­pete with Google where it counts.

Com­pete with Google where it hurts the most: Being Human.

The offi­cial announce­ment (well, the addi­tion of a web­site for it) of Google’s goo.gl URL short­en­ing service’s new web­site and fea­tures on the 11th got me think­ing really hard about...

I’ve obvi­ously been quiet here on my per­sonal blog — as every­one who reads reg­u­larly knows I’m neck-deep in a pretty excit­ing startup call Nasuni as well as doing other projects, like the PSF Spon­sored sprints thing. That com­bined with twit­ter means my time for other addi­tional long-form con­tent is min­i­mal. So here’s a small roundup of inter­est­ing things:

Nasuni

Yup, still run­ning Python and Django! We’re actu­ally pretty proud to be a spon­sor for Djan­go­Con 2010 com­ing up in Sep­tem­ber — I’ll be attend­ing, so I hope to see all the famil­iar Django faces I know, and meet some new ones.

I’ve been blog­ging semi-regularly for the Nasuni blog itself — my posts are focused on product-things more than any­thing else. Here’s a small list of posts which I’ve done:

• The Road to Release — Fea­ture Pre­views — this is actu­ally my lat­est one, and the first in a series where I’ll be show­ing off some of the new fea­tures we’re adding in the lat­est release.
• Look­ing at Open­Stack, a Rack­space and NASA ini­tia­tive — For those of you who don’t know, Rack­space and NASA announced Open­Stack — the awe­some part? It’s all python — I had the swift com­po­nent (which pow­ers Rackspace’s cloud­files sys­tem) of Open­Stack run­ning pretty quickly. I’d rec­om­mend snag­ging the code from launch­pad and tak­ing a look. Swift (the stor­age com­po­nent) uses event­let — and Nova (the com­pute part) uses Tor­nado and Twisted.
• Stor­age Switzer­land Test Dri­ves the Filer — This is a response to an arti­cle writ­ten about the prod­uct — I actu­ally used it to pre­view some of the work going into the next release of the Filer.
• Thanks to Django — This piece goes into some detail about our use of Django, it’s one of our ways of say­ing thanks. I still need to rework it so we can send it over for the Django Suc­cess Sto­ries page.
• Thanks to the Sup­port­ing Cast — This is an ear­lier thank you post — but to the other peo­ple who have helped out a ton, includ­ing Greg New­man, Lin­coln Loop, and Revsys.
• The Donut Solu­tion — This was a fun one, mainly to show that yes — we’re lis­ten­ing hard to cus­tomer feed­back, and we’re improving/iterating quickly. Also, I get to show off UI improvements.
• Finally — The Nasuni Blog team — this is the rosetta stone for the authors of the blog, describ­ing who we are. I didn’t write this piece, but it’s good read­ing to fig­ure out who is who.

If you’re inter­ested in Nasuni — or cloud stor­age in gen­eral — I’d encour­age you to sign up for the RSS feed. We’re try­ing to keep the infor­ma­tion use­ful out­side of “just us” (despite my urge and predilec­tion to churn­ing out com­pletely product-related posts) — and if you ever have feed­back, drop us a line.

PSF Spon­sored Sprints

The project con­tin­ues on — we’ve funded two sprints so far, and have sev­eral more com­ing down the pike. We’re always in need of vol­un­teers to help us do things like the man­u­als and site maintenance/content author­ing. Here’s some highlights:

• The call for appli­ca­tions is open — The call for appli­ca­tions is open — and now I sus­pect we won’t be clos­ing it. Orig­i­nally, I thought we’d have to do things in waves of apply-approve. As time has pro­gressed, I no longer think this is the case.
• Mon­treal Python Pack­ag­ing sprint wrap up — the wrap up report for our first sprint!
• Europy­thon core sprint report - another wrap up report for the core sprint we pro­vided funds to.
• Just added the loca­tions page — we now have people/companies offer­ing up space for sprint­ers! Check it out!
• Finally - Sprints at PyOhio — PyOhio is going on this week­end, if you’re in the area you should really go check it out! Cather­ine has gone above and beyond with the entire “become a con­trib­u­tor” effort going on.

Please! If you’re think­ing about hold­ing a sprint - send us an appli­ca­tion! Heck, even if we’re not spon­sor­ing it, we’ll help pro­mote you via the blog, and the sprint cal­en­dar we have up. A lit­tle fact? The sprints we’ve funded so far, and that are on deck for fund­ing are all out­side of the US, which is both awe­some, and surprising!

PSF Board

Some of you prob­a­bly know that I’m cur­rently on the board of direc­tors for the PSF — things progress well here, but I mainly wanted to call out the excel­lent blog Doug Hell­mann has been author­ing for PSF news. You should really be watch­ing that because yes — we do do things, and hope­fully over the next year, we’ll be doing more awe­some things.

I’ve actu­ally got a big­ger post in the works for what I think the ulti­mate mis­sion of the PSF is/should be as well as “how do you get money from us” as well. Must find the time!

I’ve obvi­ously been quiet here on my per­sonal blog — as every­one who reads reg­u­larly knows I’m neck-deep in a pretty excit­ing startup call Nasuni as well as doing other...

The com­pany I’ve worked for since July of last year — Nasuni Cor­po­ra­tion (a startup in Mass­a­chu­setts) has gone live! This is the cul­mi­na­tion of a lot of hard, but exceed­ingly fun and excit­ing work over the past months.

The Nasuni team is an excel­lent one — and one I am very, very proud to be a part of. Our prod­uct is called the Nasuni Filer — a simple-to-use, ver­sioned, encrypted and cloud-storage backed vir­tual NAS (net­work attached stor­age) server (click here for more information).

With­out going into all of the fea­tures, our goal in mak­ing this was to make cloud stor­age sim­ple, acces­si­ble and secure — and I know we’ve accom­plished all three. All you do is down­load it, boot it and start using it — once you do so you have access to truly unlim­ited stor­age. It’s an unlim­ited filesys­tem for the cloud. Here’s the ele­va­tor pitch:

Nasuni has devel­oped a vir­tual file server, called the Nasuni Filer, that deliv­ers unlim­ited file stor­age and com­plete file pro­tec­tion for busi­nesses. Work­ing in part­ner­ship with lead­ing cloud stor­age ven­dors, the Nasuni Filer lever­ages the vast capac­ity of the cloud to store and pro­tect com­pany files off­site, while retain­ing the local func­tion­al­ity and per­for­mance of a tra­di­tional NAS.

This tech­nol­ogy allows busi­nesses to use the cloud provider of their choice as a replace­ment for tra­di­tional pri­mary stor­age. Snap­shots, file ver­sion­ing, and off­site stor­age are inte­grated into the file server itself — ensur­ing busi­ness file are safe and secure at all times. No need to man­age com­plex backup and DR schemes — if the file server is run­ning, files are protected.

We’ve launched the Beta of the prod­uct today — any­one can sign up, down­load and use it. Any­one can give us feed­back and sug­ges­tions — I encour­age all of you who might need some­thing like this to down­load and give it a try. If you want — go check out the videos we’ve put together show­cas­ing the Filer (and bet­ter yet — check out the awe­some ani­mated car­toon we have on the front page).

Most of you know that my blog is mainly Python ori­ented. Suf­fice it to say, Nasuni — and the Nasuni Filer make use of Python for a wide range of tasks. We use Python, Django and as much of the Python ecosys­tem as we can to drive every­thing from the web­site, to the GUI on the appli­ance itself — Python is part of the DNA of the com­pany, and it has served us well. With­out Open Source and Python — I don’t think it would have been pos­si­ble to build what we have built in as lit­tle time as we have.

We have a strong ded­i­ca­tion to not just Python, but open source in gen­eral (and a fair num­ber of us will be at PyCon this month). As time pro­gresses, now that we’re exit­ing stealth mode we plan on pos­si­bly open sourc­ing stuff we feel would ben­e­fit the com­mu­nity. Some of us already push patches back where and when we can, but as I said — as time pro­gresses this involve­ment will only increase.

So not only am I proud to announce the prod­uct, be part of this team and to see what we’ve made, I’m also happy to thank so many peo­ple in the Python and OSS com­mu­nity which have helped us reach this point.

So go — check it out, let us know what you think.

The com­pany I’ve worked for since July of last year — Nasuni Cor­po­ra­tion (a startup in Mass­a­chu­setts) has gone live! This is the cul­mi­na­tion of a lot of hard, but...

So, I’m one of those peo­ple where I don’t like run­ning things “too far” from what a pro­duc­tion setup might look like (I code on OS/X, deploy to Linux). This is why I jump(ed) through var­i­ous hoops on my OS X sys­tem to get Apache/Django/mod_wsgi/etc all up and run­ning and happy (not for serv­ing the site; just developing).

Since I like simple/succinct guides, I thought I’d post what I did so oth­ers can fol­low in my stead.

Note: These instruc­tions work with the python 2.5 ver­sion which ships with Leop­ard, or a self-compiled ver­sion of 2.6 (which is what I pre­fer) — see the Instal­la­tionOn­Ma­cOSX mod_wsgi page. Addi­tion­ally, see the “Miss­ing Code For Archi­tec­ture” sec­tion for pos­si­ble work-arounds if you find your­self need­ing 32 bit exe­cu­tion of Apache; I think the “Forc­ing 32 Bit Exe­cu­tion” are pre­ferred over the “thin­ning” of the Apache binary.

First, down­load and install mod_wsgi on leop­ard, this is as easy as (on Leopard):

curl -o mod_wsgi.tgz http://modwsgi.googlecode.com/files/mod_wsgi-2.5.tar.gz
tar -xzf mod_wsgi.tgz
cd mod_wsgi-2.5
./configure
make
sudo make install

Now, edit (via sudo) /etc/apache2/httpd.conf and add the line:

LoadModule wsgi_module libexec/apache2/mod_wsgi.so

After the rest of the Load­Mod­ule lines. Cool.

Invari­ably all of my direc­tions play with virtualenv/virtualenvwrapper and pip:

mkvirtualenv django
cdvirtualenv
easy_install pip
pip install http://media.djangoproject.com/releases/1.1/Django-1.1-rc-1.tar.gz
django-admin.py startproject mysite
django-admin.py startapp myapp
cd mysite
mkdir apache
mkdir media

Now, that just sets up the skele­ton — the meat of the wsgi con­fig­u­ra­tion goes in apache/ in the mysite/apache direc­tory. The first file is named mysite.wsgi:

1
2
3
4
5
6
7
8
9
10
11

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'mysite.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This does the needed wsgi project magic for the Django appli­ca­tion — don’t worry about the inter­preter path; we’ll do that next.

Next up is a file named apache_django_wsgi.conf, this looks like this:

# mod_wsgi configuration directives - I like having stdout access, the other two
# options run mod_wsgi in daemon mode - more on this in a minute.
WSGIPythonHome /<path to virtualenv>
WSGIRestrictStdout Off
WSGIDaemonProcess django
WSGIProcessGroup django

#
# This should be the path of the /mysite/media directory
# for example "/Users/jesse/mysite/media/"
#
Alias /site_media/ "<PATH TO>/mysite/media/"
<Directory "<PATH TO>/mysite/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Directory path to the admin media, for example:
#

Alias /media/ "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media/"
<Directory "<PATH TO>/virtualenv/site-packages/django/contrib/admin/media">
Order allow,deny
Options Indexes
Allow from all
IndexOptions FancyIndexing
</Directory>

#
# Path to the mysite.wsgi file, for example:
# "/Users/jesse/mysite/apache/mysite.wsgi"
#

WSGIScriptAlias / "<PATH TO>/mysite/apache/mysite.wsgi"

<Directory "<PATH TO>/mysite/apache">
Allow from all
</Directory>

The apache_django_wsgi.conf file is the meat-and-potatoes here. This sets up all the paths/permissions, and is in Apache httpd.conf for­mat. You can pretty much log­jam any apache con­fig­u­ra­tion direc­tive here that you like.

Your final step is to once again edit (via sudo) /etc/apache2/httpd.conf and add a line like this at the ver­rrrry bottom:

Include "/path to/mysite/apache/apache_django_wsgi.conf"

And then run “sudo apachectl restart”

You should now be able to hit http://127.0.0.1/ and see the friendly and invit­ing django wel­come page. Note, that if you are using sqlite as your data­base, you should chmod a+rw the file, so that processes which are not you can mess with it.

There’s a final piece to this though. Nor­mally, if you run mod_wsgi in embed­ded mode, you’re going to need to restart apache every sin­gle time you make a change to your django app.

Ah! But we’re run­ning in dae­mon mode. This means all you need to do when you change a file is:

touch mysite/apache/mysite.wsgi

This will trig­ger a reload and magic hap­pens. Me being as lazy as I am (ask my wife) ended up snag­ging Bruno Bord’s tdae­mon script, and hack­ing it up a bit. The tdae­mon script will watch a direc­tory and run tests. Well, I wanted it to watch a direc­tory (and let me fil­ter sub direc­to­ries) and then run that touch com­mand. So I reused my watcher.py (here) — I used this to mon­i­tor my sphinx tree and run builds as well (and other stuff). Here’s how I’d use this:

workon django
cdvirtualenv
cd mysite
python ~/.slash/bin/watcher.py --command "touch apache/mysite.wsgi" -f media

This will auto-fire the touch com­mand when­ever it detects a file change (includ­ing svn updates).

You can also do this another way
In my rush to reuse a tool I use a bit (watcher) I skipped past the mod_wsgi doc­u­ment sec­tion on code reload­ing that shows how to setup a mon­i­tor which will watch .py file changes and kill the wsgi dae­mon, here. If you scroll down a bit, you’ll see the “Mon­i­tor­ing For Code Changes” sec­tion. All you need to do here is copy the code from the wiki into a mod­ule on your PYTHONPATH — in my case, I wrote it to mysite/apache/wsgi_monitor.py (just for this exam­ple! you should put it some­place else!) and then changed the mysite.wsgi file to import it, and set it up:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import os, sys
 
#Calculate the path based on the location of the WSGI script.
apache_configuration = os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
workspace = os.path.dirname(project)
sys.path.append(workspace)
 
sys.path.append(apache_configuration) # you probably shouldn't do this.
import wsgi_monitor
wsgi_monitor.start(interval=1.0)
 
os.environ['DJANGO_SETTINGS_MODULE'] = 'ui.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This method — once you reload apache — will watch the project for changes and then kill the wsgi dae­mon (forces a reload). So there you go — two ways of doing it.

The nice thing about this setup is that I can make pro­duc­tion ver­sion of the wsgi scripts and check them in, but keep local “my copies” (ala local_settings.py) addi­tion­ally, I don’t have to jump through hoops to get sta­tic media and con­tent served up via the django devel­op­ment server.

Addi­tional reading:

• http://code.djangoproject.com/wiki/django_apache_and_mod_wsgi
• http://ericholscher.com/blog/2008/jul/8/setting-django-and-mod_wsgi/
• http://code.google.com/p/modwsgi/wiki/IntegrationWithDjango

So, I’m one of those peo­ple where I don’t like run­ning things “too far” from what a pro­duc­tion setup might look like (I code on OS/X, deploy to Linux). This...

Lately, I’ve been rumi­nat­ing on require­ments and require­ments man­age­ment (also known as dis­as­ter con­trol). I was actu­ally typ­ing some­thing up on this, but Steve Yegge hit the nail on the head — then he rammed it through the board and into the house next door:

Any­way, there you have it: the slightly expanded ver­sion of the email I sent that CEO guy. Gath­er­ing busi­ness require­ments is hokum. Hooey. Horse­shit. Call it what you want, but it’s a sign of orga­ni­za­tional (or indi­vid­ual) clue­less­ness. If you don’t already know exactly what to build, then you’re in the wrong busi­ness. At the very least, you should hire some­one who does know. Don’t gather busi­ness require­ments: hire domain experts.

Also, FWIW, here’s the hack­ernews dis­cus­sion. Here is a link from one of the com­ments point­ing to some­thing Linus once said about specs:

they’re dan­ger­ously wrong. Real­ity is dif­fer­ent, and any­body who thinks specs mat­ter over real­ity should get out of ker­nel pro­gram­ming NOW. When real­ity and specs clash, the spec has zero mean­ing. Zilch. Nada. None.

I think one of the com­ments also added some­thing spec­tac­u­lar — not­ing that “build­ing some­thing for your­self” is why so many open source projects flour­ish. If you’re build­ing some­thing use­ful for your­self — there’s a high chance that some­one else is going to want to A>Use it B>Buy it — “build­ing for your­self” is also in some ways, “keep­ing the vision clear”

One of the key con­cepts which seems to be the under­cur­rent to what he talk about is vision. You need some­one who can stand up and say “this is what the prod­uct is, does and where it is going”. You need that vision­ary who can clearly out­line what itch you are try­ing to scratch. In open source — that’s the project “core” — in busi­ness, it’s the CTO or founder. It’s always the per­son that had the itch, they’ve “walked a mile in the shoes” so to speak.

That vision has to be the core of both the prod­uct, and all of the require­ments — this “clar­ity of vision” (some might say “sim­plic­ity of vision”) is what makes so many projects and prod­ucts successful.

Sure — as you grow you’ll add fea­tures: You don’t want to stag­nate — but those fea­tures have to make sense — they have to mesh with the core vision of the prod­uct. You don’t add a source code man­age­ment ser­vice to say, twitter.

Why? Because even if 1 cus­tomer thinks “that it would be AWESOME” — you’re going to spend $X hours of engi­neer­ing time glu­ing a volvo on the side of your bat­tle­ship, and unless those $X hours are com­pen­sated by the amount of money the cus­tomer is will­ing to pay (it never is) you’ve wasted time, and mud­died the func­tion­al­ity and phi­los­o­phy of the prod­uct. It’s about as use­ful as a screen-door on a submarine.

When you’re think­ing about require­ments ask your­self this: If at the start, you can not describe exactly what your prod­uct does in under a minute — you’ve already got a prob­lem. If adding this fea­ture makes it even harder to describe/encapsulate the vision and capa­bil­i­ties of the prod­uct you’re rapidly run­ning towards wronger-than-wrong.

If you your­self would not use the fea­ture: Does it really make sense? When a cus­tomer requests a fea­ture — does it make sense for any­one out­side of them? Would you be bet­ter served by pro­vid­ing an API and an SDK?

This is the beauty of things like a clean API — you can keep the phi­los­o­phy and core of the product/project clean and empower your users to build any num­ber of things they want on top of your prod­uct. Keep it sim­ple, keep it clean. Empower your users to “mashup” as they need or want to.

Think of it terms of cook­ing: If you wouldn’t eat it your­self, in all like­li­hood your cus­tomers won’t like it, at best it will be mediocre. The best chefs taste and con­sume what they cook.

Right now I’m wish­ing Brian Fitzpatrick’s keynote from pycon: “You *can* Fool All of the Peo­ple All of the Time” was online in video form.

See also: THE TECHNOLOGY OBSESSION by S. Lott

Lately, I’ve been rumi­nat­ing on require­ments and require­ments man­age­ment (also known as dis­as­ter con­trol). I was actu­ally typ­ing some­thing up on this, but Steve Yegge hit the nail on the...

Yup. I just keep gen­er­at­ing foun­tains of use­ful things. I have 15 Jaiku invites, and 5 Ever­note invites. I rec­om­mend both, although right now I use more of twit­ter until I can con­vince peo­ple to move over to Jaiku cause it’s more stable.

And ever­note is awesome.

And, as before — leave me a com­ment or send me an email for an invite — I will need your email address to send it/them to you. Let me say that again: I need your email address. I promise I won’t send you wacky Japan­ese game show youtube links.

Yup. I just keep gen­er­at­ing foun­tains of use­ful things. I have 15 Jaiku invites, and 5 Ever­note invites. I rec­om­mend both, although right now I use more of twit­ter until...

Ivan Krstić just posted an entry named “Sic Tran­sit Glo­ria Lap­topi”.

With­out com­ment­ing on the prob­lems of the OLPC project, which — for some time — has seemed to be rapidly push­ing itself into obliv­ion — I com­pletely agree on Ivan’s points on open source and frankly, every­thing else he says.

It’s sad­den­ing that the project which thrilled me — due to the ideas out­line by Ivan — now dis­gusts me and so many others.

Ivan Krstić just posted an entry named “Sic Tran­sit Glo­ria Lap­topi”. With­out com­ment­ing on the prob­lems of the OLPC project, which — for some time — has seemed to be...

Just a quick note — I’ve got 10 Ever­note invites — if you don’t know what Ever­note is, check out the Ars review here. Post in the com­ments if you’d like to test drive it. I’m going to need an email address.

So far, I’m lov­ing it — but I’m also some­one with about 500+ book­marks in Safari, a ton of “to read” stuff saved on my hard drive, and over 100 RSS feeds in Net­NewsWire. So some­thing to help me archive/save/search every­thing is key.

Now I just want PDF index­ing support.

Ok, I have 10 more left!

Just a quick note — I’ve got 10 Ever­note invites — if you don’t know what Ever­note is, check out the Ars review here. Post in the com­ments if you’d...