jessenoller.com

Welcome to Parenthood, here’s your germs

For the past year I've had the exciting opportunity to be a father - it really is fantastic, especially now that new parts of her personality is coming through and she's in the "crazy aboriginal where are your pants" stage.

However, on the flip side, it's a lot of work - tiring work. It also comes with a side-effect. Disease. I've been sicker the past year than I ever have been in my life. I think I end up with a cold or something else every month. I've had sinus infections, colds, Hand-Foot-and-Mouth and lack-of-sleep induced insanity.

Good times. At least I got to see the new batman movie, which was amazing. I'm going to go blow my nose again.

Python 2.6 and 3.0 Beta 2 released!

The second beta release of Python 2.6 and the second beta release of Python 3.0 are good to go.

From here on out, we're working on stability and bug fixes for final - this means I've got to close out the remaining multiprocessing package bugs and clean up the docs.

I really encourage people to download and test - it's important that these releases are as high-quality as possible. There's a lot of changes in both 2.6 and 3.0 and there's a lot of ground to cover.

Fuzzing Python

Well, now that I have a moment to catch my breath - here's an interesting utility that was released late last week. It's a Fuzz testing framework (see wikipedia) written in Python called fusil.

The reason I know about it, is that the author sent an announcement (and a handful of bugs) to python-dev. He (he being Victor) wrote a plugin capable of fuzzing the CPython code/library. Most of the bugs he found were segmentation faults (you can see the full list here).

In any case, fuzzing is an area I've been meaning to get "more into" - I think it's awesome that A: the tool is in python and B: there's a plugin to test python-c-code. You can view the list of python modules tested here.

This is pretty awesome!

Does anyone know what this error is? (Segfault) … nevermind

Ok, I'm at the end of my sanity. I'm finally over the hell-death which is hand, foot and mouth (courtesy of my daughter) which means I can type again, and now I'm stuck trying to debug some dastardly segfault. Here is it (courtesy of /var/log/messages):


Jun 30 12:49:13 NON kernel: python[29965]: segfault at 000000010000000f rip
000000000044068e rsp 00002aabea0294b0 error 4


This is on a 64 bit machine. The application is heavily, heavily threaded python using queues for inter-thread communication, and the threads are using libcurl via pycurl under that.

I get no exceptions, no errors: Nothing. The interpreter simply dies. If I attach an strace to it, I see this:


futex(0x2e4a6300, FUTEX_WAKE, 1) = 0
futex(0x2e4a6300, FUTEX_WAKE, 1) = 0
futex(0x2aaaac10cab0, FUTEX_WAIT, 0, NULL) = -1 EINTR (Interrupted system call)
--- SIGALRM (Alarm clock) @ 0 (0) ---
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---


Any ideas?

Edit: Never mind, I finally found the answer... I removed:
curl.setopt(pycurl.CONNECTTIMEOUT, 120)
After reading this mail thread, and the code now flies. I love life.

Python Magazine for June is Live!

This issue covers Pybots, PyObjc, and all sorts of other goodness. I got to be a reviewer and also get an article in titled "An Interview With Adam Olsen Author of Safe Threading".

Get it here.

Also, the cover is awesome.

Bug day/weekend June 21-22

See the Bug Day page on the wiki - this weekend is a python "bug day" weekend where everyone is encouraged to test, fix, find, and hack away at getting the releases closer to the "finish line".

I will be stealing some time when I can this weekend to also pitch in (and fix the bugs assigned to me, but lord knows how I am going to fix the solaris ones and test them!)

Minor problem with ‘make install/altinstall’ and multiprocessing.

Looks like I missed a spot when adding in the package, and if you do a 'make install' or make altinstall (which I never do - sorry about that) the package is missing.

See the bug report for more information and a small patch. Makefiles for the win.

As a side note - I do not recommend people do a make install into the the system path on their machines. It's generally poor form to do that with alphas/betas/trunks. If anything, just do a make, and the add a symbolic link in /usr/bin or wherever pointing to the python binary in the source directory.

Of course, I'm so anal about system paths I use virtualenv.py a *lot*. Of course, if I was 'make installing' a lot I would have found this. /grumble

Python 2.6 and 3.0 Beta 1 Released.

I can finally crawl out from under the rock I've been under to happily pass on the news that Python 2.6 and 3.0 Beta 1 is officially released. For those of you living on the moon - this release is especially exciting for me for a few reasons:

1. PEP 371, the addition of the pyprocessing module (as the multiprocessing module) has been implemented.
2. This was my first serious foray into core-development, so I learned a lot (some rather painfully/embarrassingly).
3. I ended up with commit privileges so I can help maintain the new package/module. My first checkin was unfortunately a patch to disable some tests that came in with the package.
4. I killed the buildbots and delayed the beta.

Mad props/thanks need to be passed on to a few people - Benjamin Peterson, Adam Olsen, Richard Oudkerk and many others helped get this done and helped me debug various problems that were exposed after the new package went in.

The docs for the package are here (2.6 dev docs). It's in both py3k and py2.6

I'll be working over the next few weeks on cleaning up the tests, the build and the docs for the new package. I welcome suggestions and urge people to file bugs they find in the bug tracker. Of course, if you include a patch - that's even better.

If you're looking for some cooler-stuff with the package: Look at the examples. The final one is an example of how to use the package to spread work amongst a cluster of machines.

You're going to notice something "special" about the package included in python if you have worked with it in the past: The methods have changed names. There was a discussion on Python-Dev in the context of the package's inclusion about whether or not to stick with "strict" threading module API naming, or to take this as a chance to move towards PEP 8-style naming. We chose the latter, and in fact the threading module itself is getting re-worked over time to move to the same PEP-8 style naming (tbd: I still need to pep this up). I think dropping the old Java-Style naming and going with Pythonic naming and accessing of things will be a generally Good Thing.

I've had to reiterate this a few times - but since I'm cheerleading, I might as well do it again: I do not think that the addition of this package is the "silver bullet" for "concurrency in python". I also don't think it will solve 100% of all the problems out there for people. It's useful, and a best of breed: but it is only one step in a larger movement. Adoption of Adam Olsen's work, continuing to refine the multiprocessing package, and thinking about distributed message protocols within the stdlib are just a few of the things we can do.

I'm seriously thinking about doing a introduction talk about the new package/concurrency stuff at pycon 09, although I don't know how many people would be interested.

Doing all of this work also got me even more excited for 2.6 and 3.0 - I really do recommend people download the new builds and really hammer them, it's important the final versions be as bug-free as possible.

The Abstract Cheetos Attack.

I need to write a CERT paper for this -

• Name of Attack:The Abstract Cheetos Attack
• Type of Attack: Password Vulnerability / Local Exploit / Brute Force
• Known fix: Lack of Cheetos
• Attack Vector: Food[1]/Social Engineering
• Exploit Details: It is possible to determine the most-frequently used letters on a given keyboard on the target's computer by providing the target with a "friendly" package of Cheetos at some regular interval, and then examining, over time, the build up of dangerously cheesey residue on the target's keyboard. Armed with the most frequent keystrokes, it is possible to perform a reduced brute-force attack on the target's account password. Due to the reduction in keys, it is possible to grossly reduce the time and resources required to identify the target's password. This attack also enables the attacker to determine other personal information including frequency of hand-washing, like/dislike of said "Food" and frequency at which the target cleans their clothes.

[1] Note that Cheetos may not be classified as "Food"

Programmer Insecurity and Mea Culpa.

Ben Sussman-Collins put up an excellent blog post on programmer insecurity - this rings particularly loud with me for a few reasons.

The first reason is that I used to be that guy - never checking anything in until I felt it was "perfect", then I swung to the other extreme - putting things in too quickly/aggressively.

It's a fine balance between the two, but one thing he says is especially pertinent in both open, and closed source worlds:

Be transparent. Share your work constantly. Solicit feedback. Appreciate critiques. Let other people point out your mistakes. You are not your code. Do not be afraid of day-to-day failures — learn from them. (As they say at Google, “don’t run from failure — fail often, fail quickly, and learn.”) Cherish your history, both the successes and mistakes. All of these behaviors are the way to get better at programming. If you don’t follow them, you’re cheating your own personal development.

I've grown to truly appreciate peer-review and discussion, I feel it makes all the parties involved that much better, and ultimately it improves the quality of the code. I've grown to miss peer-reviews when I don't have them - the chance to talk over the design of something and step through the code and debate various design points and possible improvements is very, very valuable.

Failure is not permanent with code: It is a transitional state which can be overcome.

That all being said: I must issue a mea culpa. Earlier this week I put together a patch for the multiprocessing package inclusion into python-core. Note that I've been using this package for some time, on multiple platforms - the tests have not failed me, and I felt that things were A-OK.

Once in though, the buildbots started doing something which reminds me of a particularly rowdy party at LinuxWorld way back - namely, all of them started churning away and promptly began puking on themselves. At least unlike me, they didn't misplace clothing or their hotel.

So, I broke the core.

I'm still chasing down the problems - we're suffering test lock-ups and a few compile errors on certain platforms (debian ppc for the loose). I feel awful because I did drop a code-bomb on Tuesday in my urgency to make the beta on wednesday. Dropping something that big right before a deadline is just poor form, and because of it - the betas didn't ship.

With that said: The work Ben Peterson, Adam Olsen, and many others have done to help (me/core) has been phenomenal and it simply reenforces why working in a community is so valuable for me. Now I just have to fix it. Anyone else want to help?