Python Debugging; Embarrassment, Contracts and Nothing is private

May 31st, 2011 § 1 comment § permalink

Random links

(photo cour­tesy of Sean MacEn­tee via flickr)

Some inter­est­ing bits and pieces (left­overs) from the week­end — I have a ten­dency to queue up a pile of “read later” stuff or email­ing myself a pile of things to talk about/link to/etc. Some­times, I actu­ally get to go through it all. Today — I get to share it to!

First up — Michael Foord (fuzzy­man) did two excel­lent posts — the first is on pri­vacy in Python — not big-P pri­vacy, but rather the programming/object pri­vacy. It’s a good read because it reen­forces the point that even if you think you’re being clever and hid­ing some­thing in a clo­sure to make it pri­vate, you can still get access to it. Remem­ber too — dun­ders (__foo) are just name-mangling. In Python, noth­ing is pri­vate (insert bad tast­ing joke about Python being face­book here).

Michael’s sec­ond post is on the Named­Tu­ple ker­fluffle that was stirred up Krist­jan Valur’s post on the use of exec() and named­tu­ple (short ver­sion: named­tu­ple cre­ates a string defin­ing the class and then calls exec to cre­ate the object. I think Michael is spot on — I think exec() gets a bad rap frankly, sure — it’s some­thing of a hand-grenade, if you use it wrong, you’re going to get hurt, but in this case I have to agree with Ray­mond in his com­ment on bug 3974 — the cur­rent imple­men­ta­tion is clear and main­tain­able. I don’t like the stern­ness of the reply — but he is right. Kristjan’s use-case is an inter­est­ing one, but I don’t think it’s one that war­rants the removal of the exist­ing imple­men­ta­tion. I’m won­der­ing if a “fall­back if exec doesn’t exist” is worth inclusion.

Then again, I’ve spo­ken to peo­ple who refuse to use named­tu­ples because they now know how the sausage is made. I still think the sausage is deli­cious.

Next is a pretty inter­est­ing dis­cus­sion on a pre­sen­ta­tion that came out of Pixar on get­ting over embar­rass­ment in order to get things done — I don’t have much to add above the com­ments on hacker news, except to say I think the same men­tal blocks they talk about for ani­ma­tors apply to pro­gram­mers, writ­ers, etc. We hide from code reviews, we hide our writ­ing until we think it’s “Pitch Per­fect” — when in real­ity, we shouldn’t. We should be shar­ing, dis­cussing and col­lab­o­rat­ing ear­lier, more fre­quently and more often.

Share and Ship early and often — see also “Don’t Go Dark”.

Finally, I was happy to see the “Quick and Easy Debug­ging in Python” post from Jeet Suku­maran — it’s always nice to evolve past sprin­kling print fairy dust all over your code for debug­ging — even if we all still do it despite know­ing or using PDB. Just to add to his post, if you want to increase your PDB-Fu, I rec­om­mend this “Debug­ging in Python” post by Steve Ferg, and the offi­cial doc­u­men­ta­tion for PDB. We should really add a HOWTO for this.

p.s. For addi­tional good-reading, check out “Pri­or­i­ties Don’t Exist in a Vac­uum” and “First Care”- while not ger­mane to what’s I’ve already writ­ten about, they’re a good essays on pri­or­i­ties. Thanks to the lat­est “Back to Work” pod­cast.

PyCon: Everybody Pays

May 25th, 2011 § 11 comments § permalink

Pre­am­ble

There’s been some recent dis­cus­sion about DjangoCon(.eu | .us) and whether or not speak­ers should have to pay for admis­sion as well — see Chris Wanstrath’s (of Github) tweets (here and here) and this Con­vore thread for exam­ples. Obvi­ously, as PyCon is the “big dog” so to speak for Python con­fer­ences, every­one looks to “us” for a model to work from, or how we man­age things. I’ve seen a lot of poop slung towards the Djan­go­Con orga­niz­ers, mainly due to a lack of know­ing “why” cer­tain poli­cies (such as “Every­one Pays”) exist for Djan­go­Con, PyCon, and other conferences.

As co-chair and pro­gram com­mit­tee chair last year, and pro­gram com­mit­tee chair the year before, and now chair for the next two years — I fig­ured it might be good to take a moment to explain the ratio­nale behind PyCon’s approach — as well as some sta­tis­tics about the bud­get. I’m not going to state that this pol­icy is per­fect; nor that it won’t be changed; I also will not release the bud­get pub­licly — I don’t think giv­ing every­one a spread­sheet with­out the con­text of the hun­dreds of man hours of work that go into it is use­ful, at all.

Impor­tant Note: PyCon is orga­nized and man­aged by the Python Soft­ware Foun­da­tion — this means that, as part of being a 501c3 char­ity, some of the finan­cials from past PyCon is avail­able as part of pub­licly acces­si­ble finan­cial doc­u­ments of the foun­da­tion. You can find those on the PSF’s site.

The same rea­son­ing may not apply to a con­fer­ence that is orga­nized by a com­mer­cial entity or is done for profit. OSCON is a com­mer­cial con­fer­ence, so hav­ing speak­ers get in free is gen­er­ally expected. DjangoCon.us is in the mid­dle — it is orga­nized for profit by a com­mer­cial entity, but it also con­tributes heav­ily back to the Django Soft­ware Foun­da­tion. DjangoCon.eu is man­aged dif­fer­ently as well.

» Read the rest of this entry «

Free Idea: Code equivalent to Morning Pages/750 Words

May 22nd, 2011 § 4 comments § permalink

I had a bit of an idea that almost turned into another crank based on yesterday’s blog post “On Fam­ily, Crank­ing and Chang­ing” — the “code equiv­a­lent” for the idea stem­ming from the Artists Way’s con­cept of “Morn­ing Pages”/750Words and the Writ­ing Down the Bones con­cept of “Keep­ing the Hands Moving”.

I made the mis­take how­ever, of ask­ing on twit­ter — never, ever ask a bunch of pro­gram­mers quote “What do you think the char count or line num­ber of a rea­son­ably sized Python mod­ule is?” — never. Just don’t. I’m not even ask­ing that here. Instead, I’m giv­ing you a sim­ple idea for a web appli­ca­tion. Take the imple­men­ta­tion of github’s gist’s/paste­bin, mash it up with a built in code edi­tor (such as Ace), and kick out a programmer’s ver­sion of 750 Words. The goal is not to force peo­ple to share code pub­licly — if they want, let them flip that bit (or just post/link to github’s gist sys­tem). The goal is to encour­age programmers/hackers to explore/write some new:

  • code
  • tests
  • docs

And I mean new — not some­thing bogged down by their every day projects. It could just be a clever one liner ala Raymond’s python tips. It could be a few-hundred-line piece of code that just tracks how many times they’ve watched their favorite show using some cool mod­ule. Just keep the hand mov­ing; just take 3, 10 or 30 min­utes a day and keep a pri­vate code jour­nal of new ideas, con­cepts, etc. Write some morn­ing pages.

Project Euler is algo­rithm prob­lem solv­ing, so it doesn’t ful­fill the goal. Code Kata’s come damn close (see also Cod­ing Horror’s take) — but again, those are focused on pro­vid­ing a prob­lem, and then hav­ing some­one come up with a solu­tion. I’d like some­thing that just sets forth the ideas exposed for writ­ers above, and sets it in a pro­gram­ming context.

If you want, allow users to seed it with sug­gested ideas. So that when a user logs in, they might have set a “pre­ferred pro­gram­ming language(s)” and get a a lit­tle javascript popup with a prob­lem sourced from another user, like say “use python threads” or “write an appli­ca­tion that forks”. This lat­ter part ties into an idea I had stem­ming from Zed’s orig­i­nal “Learn Python The Hard Way” post­ing — too many of these prac­tice sites focus on algo­rithms and math­e­mat­i­cal things that prob­a­bly won’t help you in the next 8 hours. Some­thing that had prob­lems like “send a receive UDP mes­sages from a socket” and the like — focus­ing on com­mon, prac­ti­cal things.

Any­ways so what are the require­ments for “XXXcode” or “xxxlinesaday”/etc:

  • Allow signup/signin
  • Allow pub­lic shar­ing of journals/non public
  • Allow sub­mit­ted “ideas”
  • Out­line the idea stem­ming from the Artists Way/Writing Down the Bones.
  • Make it look decent (I’m tired of ugly pro­gram­mer sites. Go get some­thing from the­me­for­est if you need it)

There ya go. Take it and run with it if you want. Just stop both­er­ing me about lines of code in a mod­ule. Man you guys can be pedantic.

 

Images

On Family, Cranking and Changing.

May 21st, 2011 § 6 comments § permalink

Intro­duc­tion

Some­time on May 6th — over two weeks ago now — I shot out a innocu­ous tweet ask­ing what might be a good blog topic. I think I said some­thing like “python, fam­ily, .…” — the over­whelm­ing major­ity of them responded with “write some­thing on fam­ily”, iron­i­cally, as I was read­ing those responses sit­ting at a stop light (yeah, I know — don’t yell at me) I got a phone call from my wife ask­ing me to come home imme­di­ately. You see, my wife is very, very preg­nant with our sec­ond child, and it’s not been an easy preg­nancy for her. We’ve had a lot of scares, and we’ve spent a fair amount of time in and out of our local hos­pi­tal — so when she said “come home right now” — you can expect that every­thing else pretty much evac­u­ated my brain except that.

I guess the twit­ter responses acted as sort of a cos­mic hint as to what was com­ing — the abbre­vi­ated ver­sion being that my wife, bless her soul, was deemed “high risk” and admit­ted to the hos­pi­tal on the 6th — with the expec­ta­tion that she would not leave until she gave birth. Fast for­ward two weeks in the hos­pi­tal, a myr­iad of tests, blow­ing out bud­gets on gas (what the hell) and being a “Sin­gle Dad” for most of that that time, and I can hap­pily report that mommy and baby are sta­ble, but still hang­ing out inside my wife.

These past two weeks taught me a lot about myself, about my fam­ily — some things that I thought I knew were brought to task and tested — heck, some of the things I was going to write about were put to the ulti­mate test. You may not really care about any of this — I’d go some­place else at this point if that’s the case. I do hope to out­line my thoughts on bal­anc­ing things (though I remain ter­ri­ble at it) and find­ing the time. You’re not going to find a cure all, or a hack that will “just make things work” — that doesn’t exist.

» Read the rest of this entry «

On Contribution

May 5th, 2011 § 16 comments § permalink

Pre­am­ble

A recent post on Hacker News enti­tled “Why I still don’t con­tribute to open source” raised a series of thoughts in my head — but then, before I could even take the time to gen­er­ate the small­est of replies, Brian Curtin swooped in and made prob­a­bly one of the sin­gle best com­ments in reply, and on the sub­ject of con­tri­bu­tion that I’ve ever read. The orig­i­nal post; and Brian’s excel­lent reply were enough to get me obsess­ing about this sub­ject again — eager to share my thoughts and feel­ings on it.

A while ago (almost exactly a year, oddly enough) I posted a ques­tion here to my blog — “Why aren’t You Con­tribut­ing (to Python)” — the replies to that post, both in the orig­i­nal com­ments, Hacker News, Red­dit and other thought­ful responses by Tim GoldenYaniv Aknin and Brian Curtin made me sit back and think for a long time what we — as a “python core” team could do to help reduce the fric­tion and over­all bar­rier to con­tribut­ing to core itself. It also made me take a step back and ask myself why I con­tribute to Python — not just via core, but also to PyCon and the PSF.

» Read the rest of this entry «

Getting to do what you love, with people that are awesome.

May 1st, 2011 § 3 comments § permalink

On Fri­day of last week, a new post I wrote for my employer (Nasuni) went up — “Encryp­tion Keys, User Data and Sub­poe­nas”. In that post, I got to out­line, in clear “non slip­pery” lan­guage how my employer man­ages encryp­tion keys, what data they have access to, etc. One of my favorite quotes:

If a cus­tomer has pro­vided their own encryp­tion key(s) — Nasuni, or the cloud provider, do not have those keys, and can not pro­vide them as part of a sub­poena or other legal process. We can not decrypt or access your data. We can not sup­ply a key which we do not have. This is not pol­icy or trust level pro­tec­tion: It’s impossible.

We offer auto-generated and escrowed keys as a con­ve­nience to the user — the ben­e­fits of hav­ing this fea­ture out­weigh the cost. A user or com­pany who knows noth­ing about encryp­tion keys and key escrow can still have strong data secu­rity and instan­ta­neous dis­as­ter recov­ery, they can install a Filer in min­utes and imme­di­ately be up and running.

» Read the rest of this entry «

Have you read your Python Docs Lately?

April 30th, 2011 § 8 comments § permalink

I — like most peo­ple com­fort­able with my rel­a­tively sane (yet lim­ited) knowl­edge of Python — rarely go and read the python docs except when I know what I’m look­ing for. For exam­ple, I have a win­dow open to iter­tools and col­lec­tions open a lot more than I care to admit (I love you named­tu­ple. I loves you so much), but I rarely go off look­ing for new things/stuff that’s changed.

For those few of you (and really, it should only be a few of you) who don’t know — Ray­mond Het­tinger is a python core devel­oper and author/maintainer of quite a few inter­est­ing mod­ules within the stan­dard library (and a con­stant stream of ASPN recipes). Ray­mond is also highly active on python-dev and other lists the sub­ject of this post being python-list (the one list I refuse to join just due to the time/attention tradeoff).

Oh, he’s also a cur­rent mem­ber of the Python Soft­ware Foun­da­tion board of direc­tors. To say he’s active would be an understatement.

Back to the sub­ject at hand: Docs.

Ray­mond posted some­thing I think should see a lit­tle bit more atten­tion — as I stated in my opener, many of us are guilty of not read­ing much in the way of Python docs unless we are look­ing for some­thing spe­cific. We miss out on gems/updates to older docs that really make them shine. We also miss out on new exam­ples, and other things that really help make Python’s docs some of the best out there.

As I said in my for­ward to Doug Hellmann’s “The Python Stan­dard Library By Exam­ple” -

The stan­dard library’s doc­u­men­ta­tion is good, and con­stantly improving/evolving. Given the size and breadth of the stan­dard library, it’s amaz­ing for what it is. It’s awe­some that we have hun­dreds of pages of doc­u­men­ta­tion con­tributed by hun­dreds of devel­op­ers and users that are used every sin­gle day by hun­dreds of thou­sands of peo­ple to cre­ate things – things as sim­ple as one-off scripts, and as com­plex as the soft­ware that con­trols giant robotic arms.

In Raymond’s post to python-list, he listed some new/newly updated docs that are really interesting:

The newly added/revised log­ging howto by Vinay (author/maintainer of the log­ging mod­ule) is really good to note (You have vis­ited the python docs how­tos, right? It’s here.). The addi­tional log­ging cook­book nicely com­pli­ments Vinay’s revised log­ging mod­ule documentation.

Oh, and if you were won­der­ing — yes, there is an easy to read list in the “What’s new” doc­u­ment for 2.7 (curated by Ray­mond) that lists out all the new Python 3 fea­tures back ported to python 2.7 for your read­ing plea­sure.

Last night, an addi­tional resource popped up on my radar — pyth0n.org — a site, while need­ing some CSS love (oh, the CSS love it needs) is an excel­lent way to jump to just about any­thing within Python (docs, classes, meth­ods, built-ins, etc). I think I killed it with load from tweet­ing about it last night, so be gen­tle. It really is a cool site, it’s like a uber-index of Python.

As an adden­dum — Ray­mond has also become very active on post­ing inter­est­ing bit of python knowledge/pro­tips on his twit­ter account, which have been a joy to see pop up in my tweet stream (Given how guilty I am of say­ing things that might be con­strued as “noise”). Some of highlights:

  • fac­toid: random.sample() auto­mat­i­cally chooses an algo­rithm, remember-previous-picks or remove-picks-from-the-population
  • pro-tip: deque’s optional maxlen argu­ment use­ful for n most recent items, mov­ing aver­ages, slid­ing win­dows, tail­ing files
  • pro tip: def mean(s): return math.fsum(s)/len(s) # don’t lose pre­ci­sion while adding large num­bers of small values.
  • Dirt sim­ple map/reduce util­ity for :

So, take a look around the lat­est docs and fol­low him, and other python devel­op­ers such (Barry War­saw, David Bea­z­ley, Nick Cogh­lan, R. David Mur­ray, Guido Van Rossum) on twit­ter (you can fol­low me if you want). Brett Can­non (another core com­mit­ter) has a twit­ter list of com­mit­ters.

There’s been a lot of work done across the board to improve the docs, add to them and clean them up. A lot of excel­lent peo­ple con­tribute changes almost daily to them. They are a fan­tas­tic resource, and are greatly com­pli­mented by work like the Python Mod­ule of the Week (Doug Hell­mann) and other tuto­ri­als spread across the web.

If you know of a par­tic­u­larly good tuto­r­ial, or expla­na­tion of a mod­ule, you might con­sider sub­mit­ting a ticket cit­ing it, and sug­gest­ing it be included into the stan­dard library docs as well. What are some of your favorite doc­u­men­ta­tion gems, or exam­ples for Python?

Switching to a standing Desk; thoughts

April 25th, 2011 § 19 comments § permalink

IMG 2155

For a 5 month update, check out: “The Stand­ing Desk Exper­i­ment 5 Months in

For about the past year or two, I’ve been work­ing on improv­ing my “state of being” for lack of a bet­ter term — focus­ing on more pro­duc­tiv­ity, health, etc — I think every­one goes though this at one point or another. Var­i­ous events of the past year — an incom­ing sec­ond kid, badly dislocating/injuring my knee and reha­bil­i­tat­ing that, real­iz­ing that being an over­weight, unhealthy geek prob­a­bly wasn’t in my, or my fam­i­lies’ long term inter­est and see­ing my lit­tle girl grow up faster than any­thing I’ve ever seen (see this excel­lent post by Mer­lin Mann — “Crank­ing”) has trig­gered me to make a series of changes, the most recent of which is directly inter­est­ing to a lot of you — I’ve switched over to a stand­ing desk setup at work.

» Read the rest of this entry «

Explaining the Python Software Foundation (PSF) Part 1

April 17th, 2011 § 1 comment § permalink

Intro­duc­tion

It’s been over a year since I was elected by the PSF mem­ber­ship to be a mem­ber of the board of direc­tors of the Python Soft­ware Foun­da­tion — this past week, I was lucky to be reelected. In that year, and even in the years prior to my elec­tion, I’ve long wanted to write down my thoughts and feel­ings about the foun­da­tion. This includes what it is, what it does and what it should do. You’d be stunned how many peo­ple — PSF mem­bers included — who don’t know the answer to these three things. Worse still, you’d prob­a­bly be even more sur­prised to find out many peo­ple who don’t think the PSF should exist — or even that it exists at all.

Before we start, I want to clar­ify a few things:

  • What fol­lows is my opin­ion; and may be shared by some — or none — of the cur­rent 13 PSF board members.
  • What fol­lows is not the opin­ion of my employer (not that I need to say that…).
  • What fol­lows has no rela­tion or bear­ing on my role as co-chair and pro­gram com­mit­tee chair of PyCon.
  • What fol­lows may be incom­plete, I reserve the right to amend or make clar­i­fy­ing statements.
  • None of this is meant as a crit­i­cism of any past direc­tors, mem­bers, etc.

With that pile of dis­claimers done, I will state this: what fol­lows is my offi­cial opin­ion of the PSF and it’s duties and role in the Python Com­mu­nity as a direc­tor of the PSF — this means that if you are a PSF mem­ber, and dis­agree with what fol­lows, con­sider vot­ing against me when the elec­tion comes up this year. I will state that it is my goal to not only express what I think the PSF is, does and should do but also to pro­vide some­thing that:

  • Helps explain the fun­da­men­tals of the PSF.
  • Helps explain what the Board of Direc­tors is.
  • Helps make the argu­ment why com­pa­nies should become sponsors.
  • Helps make the argu­ment as to why you should donate to the PSF.

Much of what I will be writ­ing may be a sim­pli­fied or updated ver­sion of the infor­ma­tion that is also on the main Python Soft­ware Foun­da­tion page. I’d like to note that this post has spanned sev­eral months of on-and-off writ­ing — and PyCon 2011 right in the mid­dle of it, so there is some infor­ma­tion shear as time passed.

» Read the rest of this entry «

Python Mentors site is up, hooray!

April 10th, 2011 § 0 comments § permalink

I posted the Python Men­tors — Python Core Men­tor­ship site last night. It’s pretty bare/minimalist (inten­tion­ally) right now — if there’s infor­ma­tion you think is miss­ing, or some­thing you would like to see, drop me a com­ment or an email. Feel like hack­ing on the source? Check out the bit­bucket repo — it’s a sim­ple blo­gofile site right now, and I don’t think we need much more for the forsee­able future.

Many thanks goes to OSUOSL for the free hosting.

 

Where Am I?

You are currently browsing the Programming category at jessenoller.com.