jessenoller.com

Yeah, I haven’t posted in awhile — since pycon I’ve been sick off an on, work­ing my butt off at the place which allows me to pur­chase ice cream for my kid, and so on. Busy busy. Not to men­tion, I’ve been suf­fer­ing a slight case of burnout — long story.

That all being said, I think it was last week when I twit­tered a minor philo­soph­i­cal point which was picked up and ran with by pydanny. The lit­tle point I made was some­thing like:

I don’t think it’s unrea­son­able to be able to name at least 5 things you don’t like/would change about some­thing you love. Imple­men­ta­tion details are fair game too

Now, before I delve into my per­sonal list, I want to pro­vide some con­text to this com­ment. It actu­ally has some his­tory, and it’s not an orig­i­nal thought — I think Titus Brown started a meme around this last year. In his case, it was purely based around python.

• Titus’ Post
• Daniel’s Post
• Brett’s Post
• Jacob’s Post

In my case, I’ve long main­tained that if you can not name things you would change, irk you or gen­er­ally dis­like about some­thing (not just a lan­guage) you sup­pos­edly love, whether it be a tool, a lan­guage, an OS, etc — then it shows you have a cer­tain lack of self-awareness or prag­ma­tism (there is another word I’m grasp­ing for here, but it escapes me).

His­tor­i­cally, I’ll ask this in inter­view sit­u­a­tions whether I’m speak­ing with some­one who is a test engi­neer (name and explain 5 things you love/hate about auto­mated test­ing?) a pro­gram­mer (name and explain 5 things you love/hate about lan­guage $FOO) — gen­er­ally speak­ing, this is great dis­cus­sion fod­der, and allows you to probe the thought process of the candidate.

For exam­ple, if some­one says “I hate Python’s white­space” and they’re inter­view­ing for a Python cod­ing posi­tion, I think it fair game to dig into that a bit and see if its ratio­nal, and ulti­mately ask the ques­tion: If you hate some­thing so fun­da­men­tal, why do you use it/why do you want to pro­gram in it for the fore­see­able future?

In any case, I promised a few peo­ple I’d give them the short­list of nits (I don’t hate these things, I sim­ply dis­like them) I have with Python. It’s impor­tant to remem­ber that:

1. I con­tribute work to python-core (see the mul­ti­pro­cess­ing module)
2. I pro­gram in Python daily for work, and in my free time too.
3. I par­tic­i­pate (when I’m not on a self imposed exile) on the mail­ing lists and dis­cus­sions (see Python-dev, etc.
4. I too, am a strong believer in “put up or shut up“

Now, part of me is sad that I have to pref­ace me being crit­i­cal with a dis­claimer like the above; but alas — some peo­ple, espe­cially those on the inter­net, thrive on con­tro­versy and fail to read more than 5 or 6 words before post­ing some half-witted response, or worse yet, some­one skims to the gripes I have, finds one they want to take me to task about and says “SUBMIT A PATCH !!!11″.

I do con­tribute back, so you can avoid telling me to sub­mit a patch, ok?

That all being said, here’s my list:

1. Con­cur­rency: This is actu­ally a love/hate topic for me. Obvi­ously, I’m the main­tainer of the mul­ti­pro­cess­ing mod­ule, which side­steps the GIL, but the GIL is still an irri­tant for me (given I do write a lot of threaded code). A lot of peo­ple are very famil­iar with the fact I am a pro­po­nent of threads and processes/IPC, as both serve dif­fer­ent (yet over­lap­ping) pur­poses. There is room for both. Hope­fully unladen-swallow will be able to get rid of the GIL, and then we can all move on with our lives: So long as in killing it, we don’t hose the ecosys­tem of C extensions.
   • Addi­tion­ally, I would love to see a decent corou­tine imple­men­ta­tion included in the stan­dard library, once PEP 380 is done and in the bag, if you need jus­ti­fi­ca­tion, see David Beazley’s corou­tine talk. Again, while peo­ple might dis­agree with this, say­ing that coroutines/processes/threads all “do the same thing” and would vio­late “TIOWTDI” (There’s Only One Way To Do It) I would strongly dis­agree with them. In the case of con­cur­rency, dif­fer­ent solu­tions fit dif­fer­ent prob­lems. We do not have a grand uni­fied the­ory of con­cur­rency within python.
   • Also in the con­cur­rency vein, I would like to see a cross lan­guage messaging/serialization system/format even­tu­ally come in. Right now, we have pickle; most recently, JSON — and JSON might be the final answer in this regard, but some­thing akin to pro­to­col buffers has also piqued my inter­est. Given we have JSON, I’m not ter­ri­bly hot on this one.
   • Finally, I’d like to see more of the java.util.concurrent abstrac­tions migrated in. I mean, using python threads isn’t hard, seri­ously, but more/better abstrac­tions make things nicer for everyone.

2. The Stan­dard Library: This, again, is a love/hate thing — I love the stan­dard library, and I will gladly argue with any­one who sug­gests get­ting rid of it. How­ever, that said — I would like to see the entire thing get a much bet­ter doc­u­men­ta­tion treat­ment, the docs while good, could be 1000x bet­ter, more clear/etc. I would also make every sin­gle mod­ule in there PEP 8 com­pli­ant. I know that sounds like a style-nazi thing, but if that’s the style we’re to use, I think the first thing to adhere to that is the stan­dard lib.
   • It’s also dis­or­ga­nized. While flat is bet­ter than nested, I’m sorry — but I think mak­ing it deeper and putting all the things like one another into the same name­spaces does make sense.
   • I would also break out the stdlib from core. This idea was dis­cussed at the python lan­guage sum­mit, and I think almost every­one there was in agree­ment. The idea would be to sep­a­rate out the stdlib into it’s own path inside the repo, and other python imple­men­ta­tions (such as Jython/etc) could use that copy as their copy of the stdlib mod­ules. Any­thing which was CPython spe­cific (such as mul­ti­pro­cess­ing) would stay with core/be marked as CPython only.
   • Tak­ing this con­cept of break­ing out the stan­dard library a lit­tle fur­ther: I would begin to evolve it a lit­tle more quickly. There’s a strong dif­fer­ence between changes to the lan­guage, and changes to the stan­dard library. In the case of the for­mer; it should evolve slowly, and care­fully. In the case of the lat­ter (the stdlib) I think it could — and pos­si­bly should, evolve more quickly. By evolve, I mean “get cleaned up, have things removed/added” more quickly. I do not, how­ever, mean with less thought. There’s obvi­ously a lot of “buts” and other con­cerns with this idea, but it’s just a thought. I think com­part­men­tal­iz­ing this into python-core and python-stdlib meshes with how a lot of peo­ple think about things.

3. The Docs: I touched on this in the stdlib one, but the stan­dard library doc­u­men­ta­tion, as well as rich exam­ples for a lot of the core fea­tures are lack­ing. Many of them focus on syn­tax and not nec­es­sar­ily on use. For exam­ple, I would gladly inte­grate all of Doug Hellmann’s Python Mod­ule of the Week posts into the stan­dard library doc­u­men­ta­tion tomor­row, and whole­sale if I could — his exam­ples are much more rich than those we find in the cur­rent docs.
   Many peo­ple, includ­ing myself, have been work­ing on mak­ing these bet­ter — in my case, I need to over­haul the mul­ti­pro­cess­ing docs when I have a chance.
   • Don’t get me wrong — I actu­ally appre­ci­ate the docs we have, they keep me sane, but they can be bet­ter, more clear and in some cases, more prac­ti­cal. One or two exam­ples for usage just doesn’t cut it.

   • update see: http://tosh.pl/gminick/gsoc/sphinx/

   

4. Pack­ag­ing: Ahh­hhh! I’m not going to go too deep into this rab­bit hole, espe­cially given I know Tarek is hack­ing away at mak­ing python pack­ag­ing a much bet­ter ani­mal, but the entire setuptools/eggs/distutils/etc pile is well, frus­trat­ing. I just want a clean, stan­dard way of pack­ag­ing my pack­ages, built into core, that doesn’t force me into install into the global site-packages direc­tory. Also, unin­stall, dammit. I know setup­tools and easy_install and eggs were designed to scratch an itch: and I do use easy_install, but the entire pile of things need to be made into a stan­dard, imple­mented in core and we need to move on.
   • How­ever, as I pointed out dur­ing the lan­guage sum­mit — I don’t think some­thing like easy_install belongs in core, instead I think core should make what easy_install does (to a cer­tain extent) eas­ier and stan­dard, so peo­ple can use what­ever tools/scripts/etc they want. One ring to bind them!

5. Lint­ing: Ok, face it, if you’re on a big enough team, you need to have a pre com­mit hook for your VCS that lints the code, and yacks if it doesn’t con­form. I would love for one to be built into the stdlib, but some­thing like pylint is too big, pychecker is too sim­ple, and I haven’t used pyflakes recently enough to com­ment. There was a thread on python-ideas about this recently — and maybe Jeremy Hyl­ton is right, and it doesn’t belong in core, but if that’s the case, we need to pick one to “endorse” on the python doc web­site. Maybe in a “get­ting started with devel­op­ing python” doc­u­ment, which is linked in size 30 font, and links to a lin­ter, maybe the pep8.py and reindent.py scripts, etc. It should be painfully obvi­ous where to get and how to use these tools. Yeah, I know “waaaaah why didn’t they link to mine” — well, because we liked this one over here more. QQ.
   • As it stands, I can not count the num­ber of times I’ve been asked about lin­ters and style check­ers for python code. Maybe we make three pack­ages: python-core, python-stdlib and python-tools.

6. Optional Sta­tic Typ­ing: This one doesn’t make me feel like I’ll make any friends, but I would love to have pre-runtime, sta­tic typ­ing as an option to python — maybe as a –anal-types flag. Guido has dis­cussed (part 2) the dif­fi­cul­ties of this before, so I don’t think this will ever come (the clos­est we get to “type safety” is func­tion anno­ta­tions, which make me feel funny in sen­si­tive places). The biggest rea­son I have for sta­tic typ­ing of any fla­vor, is that I would pre­fer to have the abil­ity to catch some errors prior to run­time. That’s all. On a big enough team, all hack­ing on the same (mas­sive) python code base, I’ve found you do want the abil­ity to turn some­thing like this on — it helps you with a (small) class of very annoy­ing bugs.
   • I do love me the dynamic/late typ­ing sys­tem of python, and I use it to my advan­tage as much as pos­si­ble. So, I wouldn’t trade the dynamism of Python for sta­tic types, it’s just an nit I have. Of course, maybe some­thing like inter­faces (as Jacob points out in his list) might solve some of the issues I have (mainly bad peo­ple doing silly things). The rest of the stuff is why I write unit tests and actu­ally run the damned code.
   • Yes, I know the draw­backs of some­thing like this; I also don’t have some sort of magic solu­tion to be able to wave a wand and do this. Nor do I have a con­crete pro­posal, oth­er­wise you’d see an email on python-dev. Other peo­ple much smarter than me have pointed out the sheer enor­mity and numer­ous draw­backs to some­thing like this. No, I don’t expect magic fairy dusty to sud­denly appear and just “make this work”.

7. Stan­dard Library Part II: Yeah, you might notice a lot of my gripes are around the stdlib — but in par­tic­u­lar, I want to point out the state of XML han­dling in the stan­dard library is about as clear as wear­ing glasses made of meat. Addi­tion­ally, the httplib/urllib/urllib2 thing? Yeah. No.
   • While I’m harp­ing on this stuff, get rid of the com­mands mod­ule, any­thing that is not in sub­process should be put there. Since I men­tioned sub­process, needs more doc­u­men­ta­tion also, non block­ing asyn­chro­nous input/output/handling of sub­process data should be easy, and built in. There’s a GSoC project around this spin­ning up, so we’ll see.

That all being said, would I trade python for some­thing else? Not right now. Most of my nits are exactly that: nits, and most of all, they’re not impos­si­ble to change or resolve (given enough time, and resources).

I can make a sim­i­lar list for OS/X, Linux and other things I use day in and day out — hell, I can make one for myself (ask my wife about me grip­ing about me some­time). I can prob­a­bly make a list like this for every sin­gle thing I’ve writ­ten, tools, scripts, apps, etc.

Like I said, being aware of, and try­ing to over­come your own short­com­ings is how we all improve. In the case of a lan­guage, you can’t just keep adding things into a stan­dard library and call it “bet­ter” — you have to take a con­stant look at what you’ve done to date with a crit­i­cal eye, and ask your­self “what can we do better”.

Yeah, I haven't posted in awhile - since pycon I've been sick off an on, working my butt off at the place which allows me to purchase ice cream for my kid, and so on. Busy busy. Not to mention, I've been suffering a slight case of burnout - long story. That all being said, ...