2011 In Review: The Python Portion

by jesse in , , ,

As I said in my post this morning - "2011 in Review: The Personal Portion" - it's that time where we're all taking stock and reflecting back on 2011.

In this post's case, I'm taking stock of the things that changed for me - things that stick out in my mind and projects I've either started, floundered or run completely into ground.

Design and Experience Matter

Perhaps the biggest shift for me in Python-as-a-whole is a movement more towards the social / management aspects. I'm a Python Software Foundation board member, so obviously me needing to take a "bigger view" isn't that surprising. What has been surprising to me is that everywhere I turn, I see things we as a whole can do better.

Now, before you think I'm about to go off the deep end; let me assure you - I wouldn't trade the community I'm lucky to be part of for anything, as I've said more eloquently before. However, only a fool believes that anything is perfect, and only the insane only focus on the flaws.

Taking a step back, I've seen more and more things that I think we can do a better job at, and these realizations all revolve around my continued "transition" from more back-end to more front-end design and coding. As I've become more focused on the users/community and those who are new, I've grown to internalize the fact that design and experience matter not only in code, and in a GUI, but they matter to a community and language as a whole.

I've spent the better part of this past year focused on issues around this - encouraging people to get involved in the "softer" side of things - helping out with documentation, mentorship and education, trying to get people to think more about one another and those just getting started and introduced to things.

I think that we as a community - and I mean everyone - from Django to Plone, from Twisted to Tornado, from PyPy to cPython can take a look at the "more human" aspects and find things to improve. Sometimes it requires fresh eyes to show you what's broken - people who do code reviews regularly know this.

For an example, look at Kenneth Reitz' Requests module - billed as "HTTP for Humans" - this might be a perfect example of the point I'm trying to get across. Built on top of "less friendly" libraries, it's API is a joy to use. It's simple, it's clear - the documentation is well done and the entire project feels very welcoming. Perhaps "Welcoming" is the best word for what I'm looking for.

I get stuck in wanting to fix "all the things" - and I can't help but get mired down in the details of how we make everything more welcoming and the experience better, how do we lower the barrier and reduce friction. The result is that I've broken my promises to myself and taken on more things than I can possibly hope to do justice.

How do we make things more welcoming, how do we help the new people, how do we help those of us growing stuck in our ways to find and explore new things? How can we do this as a community to lift us all up? What I think we need is a series of small, positive changes. Little things like, say:

  • User friendly READMEs and Documentation. Yes - I said friendly - don't assume your users are magical super smart engineers and users. While the article is more web focused, I enjoyed "The Myth of the Sophisticated User" - please don't assume people are running bleeding edge version of everything, and please don't assume everyone knows 20 years of Python package development.
  • Mentorship! Set up something within your project or team that is focused on mentoring people to a point where that person is comfortable to be a contributor.
  • Stop the vitriol. If you find yourself angry when you're typing that reply to a mailing list; walk away. If you see others being hostile or just flat out rude, call them out on it (privately first, no reason to be a jerk). Aim to be polite and welcoming.
  • The next time you're putting something up on the web? Take a moment to think about or learn about making something - yes - pretty and usable. Even if it's something simple, take a moment to realize that you're building something that may be your future user's first experience with you. It may be as simple as picking up "Design for Hackers" (which I quite liked) or just going with something with sane defaults - like twitter bootstrap.
  • Speaking of sane defaults - please be opinionated. When a new user wants to install something, don't give them the complete history of packaging, just gently explain to them how to do it. Even if I don't agree with the way you do that, it's a far cry from 20 years of development history being dumped on someone when a simple pip install <blah> could work. The same goes for your software: Pick sane, rational defaults and abstract away as much as you can. Put examples of usage before the API in documentation.
  • APIs and syntax matter: your communications channels to your users are APIs and syntax just as much as your actual code and libraries.

Moving on - I hate to say it this way; but think of the Users and target audience. Remember, you - the person reading this - and I - are in a tiny minority of the population where software (for the most part) isn't magic, we understand history and we're very tolerant of unfriendly things and failures because that's how we "grew up".

Not everyone knows how to build an interpreter; or a web framework - it doesn't mean they still can't contribute.

The Python Software Foundation

As most of you know - I am one of the directors of the Python Software Foundation, and have been the past two years. 2011 was another year where the PSF got to do some pretty cool things. I've been stressing and pushing more and more that the PSF has to be focused not just on the "IP" of Python, or just on cPython development - we have to take a larger view of the entire community - this means encouraging projects such as PyPy, outreach workshops, conferences, etc via grants and support.

You should really take a look at the Python Software Foundation's blog - Doug Hellmann, Brian Curtin and others have done their best to document and showcase what the PSF has been up to, and where we're trying to help.

My primary focus has been encouraging things such as the Outreach and Education committee, and working behind the scenes with a lot of people to improve the Python.org infrastructure. More recently I've been working on a project which should hopefully become public soon - but is tied to my first point about Design and Experience and the PSF.

I want the PSF to grow in the good works it performs - more grants as we can afford it, getting better hosting for things as needed, helping out projects like Read The Docs or helping push forward Python 3. The PSF is the Python Software Foundation - we need and should be supporting and helping everything from PyPy to PyPI, cPython to Scipy.

I think the best way for me to help here is to pick up where I left off documenting the PSF. Once again - the design and interface matter.

The Sprints Committee

As part of my board work back in 2010 I helped start the Python Sprints project - and under Brian Curtin's guidance in 2011, it has continued to make small donations in places it matters. In 2012, I'd like to see if I can spin back around and help it grow more and flourish, perhaps even be able to provide more money where it's needed. It's growth has been slow - but that's also due to us seeing less sprints overall it seems.


Started as a side project (yes. another one. sigh.) Get Python 3 is meant to serve as a pile of information and resources about Python 3 - and as many of the aspects of Python 3 as possible. Where to get funding, how to port, what is ported. I've actually gotten some excellent help from others (see github) and I'm hoping to grow it more. I've gotten pretty good feedback on it - and I never turn down a patch!

Python (Core) Mentorship

Driven from my experience with the first point about being welcoming, I've done my best to spin up the Python Core Mentorship group, a team / list focused on mentoring new people into contributing to core Python. To quote the home page:

The mission of the Python Core Mentor Program is to provide an open and welcoming place to connect students, programmers – and anyone interested in contributing to the Python Core development. This project is based on the idea that the best way to welcome new people into any project is a venue which connects them to a variety of mentors who can assist in guiding them through the contribution process, including discussions on lists such as python-dev, and python-ideas, the bug tracker, mercurial questions, code reviews, etc.

While traffic is low, I think it has done it's job - as with everything else on my list, I'd like to see growth - as it is, due to everything else on my plate, others have stepped up to help lead and guide the group. As it is, I've run into a case where as I've found with many other projects like this - people are already "tapped out" - myself included. More on resource contention later - and I should really do a poll and gauge the list for the relative level of success they feel the group has engendered.

Python Speed Project

Another side-burner project is the Speed.python.org project - this one makes me sad(der) than my other time-starved projects. While we have finally been able to set it up as a PyPy build slave and have it feeding results to speed.pypy.org (see the speed-python results), it has not taken off as much as I hoped. We have a beast of a machine (see my initial announcement) - but we've hit the resource wall like everything else. Not enough people with enough time and the right skills.

The Elephant in the room: PyCon 2012

My single biggest project this year has been getting PyCon 2012 ready to fly - everything from getting the new website launched, the staff assembled, writing a code of conduct, and providing white-glove service and support (and getting) our amazing list of sponsors.

I can't really estimate how many hours I've "worked" on Python - but I can tell you every hour has been worth it. Even though it's sucked my time from other things and projects, it looks like it's going to be an amazing conference. We have robots, we have amazing talks, amazing keynote and plenary speakers (Paul Graham and Stormy Peters for starters). We have awesome tutorials and even more to come.

PyCon represents the single biggest "community act" that the Python Software Foundation performs - not only does the PSF fund PyCon, but it manages it, assumes the risk, etc. I wrote about it in detail in my post "Making the Case for Sponsorship" and in the "Everybody Pays" post. I'm hoping to continue to write up more and more of the details of the inner workings of PyCon, as I think it's an important series of data points and lessons. Remember - any funds "left" from PyCon go the PSF which allow the foundation to issue grants to other conferences, to developers, groups and workshops. It helps us help you.

PyCon 2012 is the thing I am most proud of; we have 80 sponsors and partners (Such as OpenHatch and PyLadies), we have a solid team of organizers working together to bring PyCon 2012 to fruition. We have a robust financial aid program as is tradition. I can only hope that I have the tenacity and will to see it come together and be able to look at a sea of 1500 Pythonistas - new and old in Santa Clara.

ps: You can register here. :)

Blood from a Stone

How do you get more time from people who are busy? Time and Time again, I've found myself asking that question. Each one of the projects I've listed has hit the same issue over and over again. How do you get the volunteers necessary to help? Heck, even my call for help with multiprocessing in August fell on a mostly flat note - probably due to me.

I no longer feel "ok" asking for help with new projects simply due to the fact that I know everyone is busy - it's insane of me to ask people to take their time away from their projects or families or jobs.

What that means however is that I have completely failed in the not-taking-on-new-things department - and I don't see this changing much without me flat out learning to tell myself "no". I believe in this community - I believe in the people, the friends I have, the language and everything involved. It's not just another tool for me; it never has been. I'm still learning, and mostly failing (or flailing, depends on where I'm standing).

Finishing this one off

Looking at the list I've typed out above, I suddenly have the feeling that I didn't actually do much last year, I know thats wrong (a nasty look from my family members would easily remind me of that). I have been able to help out where I can making things more friendly, more welcoming and to reach out when and where I can to offer help, and support.

I've watched the community change in some dramatic ways, I've looked on as PyPy has gained amazing momentum, more and more vendors and companies have come out with Python support and stating that they're using Python (and are hiring). I've gotten to work with PSF members, the board, and many, many others - all I can do is keep at it, and hope I do things justice.

2011 in review: The Personal Portion

by jesse in ,

Yup; it's that time - everyone and their brother is doing a post looking back at 2011 and taking stock of the good, the bad and the ugly. I'm no different - 2011 was a year that largely represented a massive shift in my life's tectonic plates.

I've decided to break this reflection into two related parts - the more personal stuff (this one) and the big-P Python stuff - both have seen shifts and changes worth noting, and both are inextricably tied for me. I've intentionally skipped all of the Python** stuff (including PyCon) that I've been working on - that's going to come next.

Personal Changes

In late 2010 I was playing paintball - something which everyone should try at least once - it truly is a blast. However, at the time I was grossly overweight (280/285 lbs heading to 300) and running around outdoors with 20+ lbs of equipment. It was a normal Sunday game when I pivoted in the perfectly wrong way - my foot had gotten stuck in some tree roots and when I pivoted, my right knee dislocated and I collapsed face-first into a pile of tree branches.

I did not realize that my knee has dislocated, just that my leg wasn't working. I slapped my knee, hard, bent it and got up and kept playing. The adrenaline kept me going for several more hours while I continued to play on a knee of questionable veracity. When I got to my car a few hours later, all I knew is that my knee felt funny, and my cargo pants where tight where my knee was.

When I got home and changed, the truth came out. My knee had swollen to the size of a cantaloupe and turned several ugly colors. I figured I has injured it, and largely ignored it. Then the pain set in the next day.

Fast forward through many doctor appointments, MRIs, and two more dislocations - once getting my daughter out of the bathtub which required my wife to come in and put my knee back into place because I was busy crying on the floor, and the second just getting out of bed. My knee, from that initial dislocation had become very weak. The doctor told me flat out that I needed physical therapy and rehab, otherwise surgery was going to be required.

He told me I needed to change things. Looking at myself in the mirror, I realized that something had to be done - I was stressed, overweight and my path was out of whack. I couldn't deal with surgery with three year old and a now pregnant wife. I got a cortisone shot and went up the street to the local Bikram Yoga studio - I had never done yoga before - I walked in, slapped down some money and went into a 120 degree studio.

This is a photo of my from June 2010:



I became a Bikram convert over night - the owner of the local studio Bob is an amazing man, friendly, kind - all of the instructors helped me through learning and growing and pushing through the pain, the heat and everything that comes with a grossly overweight ex-smoker who was drinking 2+ pots of coffee a day jumping in head first. I quickly ramped to doing classes 3 times a week.

Additionally, I completely altered my diet - I've long dabbled in low-carb/no-carb/ketogenic, but this time I jumped in no-holds barred. No sugar, I cut my coffee intake to one cup a day, no carbs/gluten, period. 2011 came quickly, and I kept it up. Yoga, diet - lather, rinse and repeat. I shed enough weight that people at PyCon 2011 didn't recognize me. Good. Not good enough. Throughout 2011 I kept this up - dropping from an easy 280 lbs to 165 at my lowest. Later in the year I added weight lifting with coworkers at lunch - even later I started the couch to 5k program to start running (even doing it the "barefoot" way).

Now, as the year turns, I weigh a healthy 175 lbs - I've put on muscle mass, kept my flexibility, kept on my diet which has shifted into a more Paleo form than what it had been (mainly adding fruit back in, but still skipping carbs/gluten/sugar - I still mostly only eat meat and vegetables). I can now run for 30 minutes without feeling like death and hit 4.2 miles. My knee still bothers me sometimes, but I've dodged surgery. I can now look at my daughters and wife and hope that I'll be around a lot longer than I would have been had I not done these things. I feel more alive than ever before.

Me, December 2011:



During 2011, I also switched to an all standing desk setup (yup, despite the knee):

I'm happy to say that this continues - thanks to an excellent gift from my wife, I even have a nice standing setup at home now. It's been over 7 months since I last sat down at work to work. Sure, I sit at lunch, and in the car - I'm not that weird, but I continue to reap the benefits I outlined in those posts.

I also started working on my mental health, and focus. Trying to learn how to meditate, working on minimizing distractions and building small improvements to my workflow. Focusing on being open to change and criticism. Focusing on things I had ignored for a long time.

You can't go and just fix your physical self - you have to take care of the mental aspects as well. I've had to learn this over and over the hard way, and it is still a daily fight between what I was, and what I want to me. I have to focus on small changes and improvements constantly - otherwise it's deadly simple to fall back on old ways.

I did a post some time ago - "On Family, Cranking and Changing" - I still read this once in awhile to remind myself where I need to go and what I need to accomplish. I can't lose sight.

Now for the hard part.


2011 also brought my family to the brink - and I mean that in the literal sense. There was a time where my wife and I would look at each other hopelessly, wondering what we would do and how we would pull through. In June, we had our second daughter Addison Joy. The pregnancy was really rough and my coworkers and boss supported me through the needed "disappearing". My wife spent a lot of time in the hospital, and there were many times where we were worried that things wouldn't work out.

Luckily, my wife - and Addison, pulled through. I don't know how they did it, and I suspect we've burnt a lifetime of karma and luck in just a few months, but they both came through. Addison was born, and I once again new the joys and pains of having a new born daughter. Throughout all of this, our oldest daughter Abigail trooped on through - it was a lot to ask for a 3/4 year old, but she continually amazed me. To look at her face and see how much she worships and loves her mother - to see how she loves Addison - that's to know something you'll never see anywhere else.

Not everything was well - and we didn't know it yet, but the worst storm was yet to come.

To quote my post - "Thank you - the impossibility of "It's going to be OK"":

But, so, AJ was born - and at first, everything seemed to be fine. 10 fingers, 10 toes and pooping - that's sort of what you hope for in a newborn. We took her home, she saw her pediatrician, and that was that.

Well, no. Around the time Addison was three weeks old (shortly before my first child's birthday) my wife Dusty started noticing that Addison was behaving erratically/oddly - and if you have any experience with infants, you'd know how hard it is to actually determine "odd" behavior. Almost everything they do is odd, down to timing exactly the worst moment when to spit up on you (point of fact - it is after you've showered, and are walking out the door).

In this case, the odd behavior my wife noticed was actually a pattern - and that's when you need to worry. You want consistency in certain areas, you want to see continual improvement, you want them to consistently eat, poop and sleep. However, a pattern of odd movements tipped my wife (who is a fantastic analyst) off that something was not quite right.

What my wife found was that Addison for periods of time anywhere from 1-2 minutes her eyes would slit and roll back and she would freeze up. The best way to describe it is it was almost as if she would just "check out" - as if someone hit a power switch.

It's still hard for me to read that post - it's difficult for me to communicate the emotions - the fear, the outright terror of not knowing what was wrong with our baby girl. More hospitals, more doctors. My new born daughter with a helmet of leads and electrodes coming off of her head. Sleeping in cots in hospital rooms. My wife eloquently wrote a series of posts:

Something I say in that thank you post is something that will stick in my mind forever. When my friends and people I barely knew in the Python community heard and saw what we were going through as a family, the support we got was flooring. It still makes me tear up thinking of all the cards, well wishes and other things - a little toy for Abigail, Doug Napoleone coming over to help me out with something, everything that the Python community did for our family. It is, and was amazing. I can never thank all of you enough for what you did for us, and how you helped us pull through.

The number of emails I got from other parents in the community who suffered through things like this, the well wishes - I, I can't even go into everything that happened. Words can not express it. All I can say is that many times, my wife and I found ourselves in tears, crying with one another because of some act or gift or email from someone in the community.

In October, I did a quick Google+ post, providing an update on how things had panned out - quoting that post:

Addison's diagnosis - if you want to call it that - is Cerebral Palsy - Hypertonia (http://en.wikipedia.org/wiki/Hypertonia). This means that she does have a disorder, but it's not one treated with drugs - just physical therapy and frequent checkups. We have a nurse and a physical therapist who come weekly and check on her thanks to early intervention. She's developing well - she's eating baby food, smiling and generally being a normal baby. All we have to do is keep up with the therapy and in theory her brain will "auto correct" as time goes on. She's 17lbs and counting at just about 5 months and just giving hints of crawling.

In addition to the hypertonia, she was diagnosed with non epileptic seizures - again, not something we can do much about other than to love her, keep up with checkups and wait.

So that's where we are - we have a happy, cooing, laughing, happy baby and just have to keep a close eye on her and work through things that come up. It's too early to tell if her problems will have long term consequences. The doctors all hope that she's "error correct" around these things and she'll be OK. But we won't know until we see her development at 6 months, 9 months and 1 year - we still have that "threat" that something could happen - her brain could stop developing, or conditions could get worse.

But its hard to think about that - because I don't see the problems - every day, I pick up and hold and play with a beautiful, cheerful baby who wants nothing more than to chew on my fingers (she's teething) and laugh. I don't think about the future much, because it's unknowable, and we'll cross that bridge when it comes. Sometimes it pops into my head - that worry, that doubt, and I push it to the side and think of what we've already gone through.

It's now December - almost January. Addison has continued to thrive - the fear and the worry aren't forgotten - we have regular visits from a physical therapist and nurse to continually check on her. She still has some issues we continue to work through, and we've got a series of appointments with neurology specialists, but its hard to think that anything is "wrong" with her at all.

She's almost 20lbs (huge baby!) - she's babbling, she's gotten her first tooth (on christmas eve to boot) - she loves her walker and worships her sister. She laughs more than any baby I've ever seen, and that laugh is angelic. I don't know what the future holds, and I don't know how long our luck will hold out, but what I do know is that I have two beautiful daughters who have changed my life forever.

I have found friends where I did not expect, compatriots and support. I have found that my coworkers, community and friends are more amazing than I could have ever expected. And Addison thanks you:

IMG 3672


Finally, Work

I love my job, what more is there to say? 2011 was a break out year for me personally - and a break out year for Nasuni - we've built something amazing, something that companies want. With any luck, we have begun to change how businesses will store their data and what they come to expect from an enterprise class product. I get to do what I love, with people that are awesome.

Of course, 2011 found me growing more into doing things I never really expected to be doing - I've continued a shift from the back end/glue and more into the front-end, spending most of my time working on user interfaces, beating my head against internet explorer. I've spent more time in JavaScript than I care to admit. Learning CSS, re-learning design, layout, thinking constantly about user experience, staring at color palettes for days.

And I - We - are far from done. I've mentally grown into a mindset that "UI" (user interface) doesn't just stand for the graphic design of a site - and that UX (user experience) isn't just about how things are laid out on a page. UI/UX has to be thought about from the part the user sees, feels and uses all the way down to the lowest level API of your system.

Good Design (notice the big D) means APIs matter. It means that everything from error messages, to documentation to customer support and care matter. You can't ignore any of it. You can't slap a CSS framework into place and think you're done with "Design". It means caring about the user completely, and without regard to your biases or skills.

Good Design also matters in communities - user experience, interfaces - thinking about others - of course, I'm getting ahead of myself and delving into the second post.

Wrapping this one up

On a personal level - 2011 was a year I doubt I'll forget any time soon. It's been a mixture of pain and pleasure and constant evolution and change. 2011 changed who I fundamentally am as a person, and I hope I'll never be the same.

Again, thank you all - you know who you are.

And to my family: Dusty, Addison, and Abigail (who is so smart it scares me) - I love you.


Quick example of extending UserCreationForm in Django

by jesse in ,

I just banged my head against this, and with no good answers floating around out there, I thought I'd share. In my case, I just wanted to extend the basic django.contrib.auth.forms.UserCreationForm in order to make it so when a user was added, an email address had to be supplied in addition to the username and password fields.

Here is a working example (forms.py) - just so I don't forget it:

from django import forms
from django.contrib.auth.models import User
from django.contrib.auth.forms import UserCreationForm

class UserCreateForm(UserCreationForm):
    email = forms.EmailField(required=True)

    class Meta:
        model = User
        fields = ("username", "email", "password1", "password2")

    def save(self, commit=True):
        user = super(UserCreateForm, self).save(commit=False)
        user.email = self.cleaned_data["email"]
        if commit:
        return user

You have to modify the save method on the form to add the email to user object returned by the super call. You can use this to expose other fields on the User object as needed.

Porting to Python 3: An offer for you.

by jesse in ,


Recent posts and discussions around porting of existing libraries and frameworks to Python 3 have been pretty interesting. I think that there have been a lot of good points brought up in the discussion (See: Armin's Post (and followup), Nick's entry on Python 3 and Nick's email to Python-Ideas).

On a personal level; I've felt frustrated that there's not much that I can do myself - I do believe that 2.7 is the proper end of the road of Python 2, and I do think that Python 3 is the future of the language. Does that mean Python 3 is perfect? Oh hell no. Does it mean that we can do work to make Python 3 the "Python 3" we all want and need?

Yes it does.

So; while there is nothing I can do directly other than continue to work on the site I've been slowly building - GetPython3.com with help from the community - there is an aspect I can help with from a Python Software Foundation / Grants level. That means money (well, not unlimited).

As some of you might know - the PSF has actually issued grants to developers who have applied to port important libraries to Python 3 - as I say on the GetPython3 page:

In short: yes - there's a bevy of information, videos and blog posts out there that can help you on your way. Python 3 is the future of the Python language, and entities such as the Python Software Foundation strongly believe in supporting the porting effort.

For example, the Python Software Foundation has issued developer grants to port projects such as the email package, PyOpenSSL, and WebOb. It has also provided developer grants for other general Python development work, such as to Brett Cannon that allowed him to completely revamp the Python developer's guide.

The Python Software Foundation is here for not just CPython, or python-core, or python-the-language. It is here for Python - the community, it's efforts, its developers, designers and people.

Certain projects - most notably PyPy - have already started donation programs to help fund large-scale development efforts to Python 3. Others may soon follow.

Additionally to the grants-to-developers aspect - the PSF Sprints project has been issuing grants for Python sprints in general, which means you can apply / ask for a grant for a port-to-python3 workshop or sprint any time!

But; back to where I was going...

My offer to you, the community is this - I can not guarantee you will get a grant, or funding - but what I can do, and what is within my power as a fellow member and PSF Director is offer to help write, and review applications to the PSF Board of directors for grant applications.

That's right - I will assist you in writing an application that will be submitted to the PSF Board for approval, for grants aimed at porting libraries or frameworks to Python 3; or doing specific documentation / core work for Python 3. I can help you write it; provide templates, discuss it with you (I may have some elves help me) and ultimately help you put it in front of the board for approval.

Obviously; the PSF does not have unlimited funds; nor can it spend funds irrationally. Python 3 is important however - critically so - and while we can not fund everything, we can do what we can. I am aiming at libraries/frameworks which are in widespread use (e.g. notable) and that other projects/libraries/frameworks depend on heavily (for example, see the Py3k poll).

Before getting started, you should read the basic PSF Grant guidelines and you should look through the information on http://getpython3.com/.

If you are interested in this; drop an email to jnoller@python.org - I don't promise immediate up-to-the-second turn around - I've obviously got a lot on my plate right now, but I will do my best to help.