| Subcribe via RSS

Does anyone know what this error is? (Segfault) … nevermind

June 30th, 2008 | | Posted in Programming, Python

Ok, I'm at the end of my sanity. I'm finally over the hell-death which is hand, foot and mouth (courtesy of my daughter) which means I can type again, and now I'm stuck trying to debug some dastardly segfault. Here is it (courtesy of /var/log/messages):


Jun 30 12:49:13 NON kernel: python[29965]: segfault at 000000010000000f rip
000000000044068e rsp 00002aabea0294b0 error 4

This is on a 64 bit machine. The application is heavily, heavily threaded python using queues for inter-thread communication, and the threads are using libcurl via pycurl under that.

I get no exceptions, no errors: Nothing. The interpreter simply dies. If I attach an strace to it, I see this:


futex(0x2e4a6300, FUTEX_WAKE, 1) = 0
futex(0x2e4a6300, FUTEX_WAKE, 1) = 0
futex(0x2aaaac10cab0, FUTEX_WAIT, 0, NULL) = -1 EINTR (Interrupted system call)
--- SIGALRM (Alarm clock) @ 0 (0) ---
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

Any ideas?

Edit: Never mind, I finally found the answer... I removed:
curl.setopt(pycurl.CONNECTTIMEOUT, 120)
After reading this mail thread, and the code now flies. I love life.

Python Magazine for June is Live!

June 25th, 2008 | | Posted in Programming, Python

This issue covers Pybots, PyObjc, and all sorts of other goodness. I got to be a reviewer and also get an article in titled "An Interview With Adam Olsen Author of Safe Threading".

Get it here.

Also, the cover is awesome.

Bug day/weekend June 21-22

June 19th, 2008 | | Posted in Programming, Python

See the Bug Day page on the wiki - this weekend is a python "bug day" weekend where everyone is encouraged to test, fix, find, and hack away at getting the releases closer to the "finish line".

I will be stealing some time when I can this weekend to also pitch in (and fix the bugs assigned to me, but lord knows how I am going to fix the solaris ones and test them!)

Minor problem with ‘make install/altinstall’ and multiprocessing.

June 19th, 2008 | | Posted in Programming, Python

Looks like I missed a spot when adding in the package, and if you do a 'make install' or make altinstall (which I never do - sorry about that) the package is missing.

See the bug report for more information and a small patch. Makefiles for the win.

As a side note - I do not recommend people do a make install into the the system path on their machines. It's generally poor form to do that with alphas/betas/trunks. If anything, just do a make, and the add a symbolic link in /usr/bin or wherever pointing to the python binary in the source directory.

Of course, I'm so anal about system paths I use virtualenv.py a *lot*. Of course, if I was 'make installing' a lot I would have found this. /grumble

Python 2.6 and 3.0 Beta 1 Released.

June 19th, 2008 | | Posted in Programming, Python

I can finally crawl out from under the rock I've been under to happily pass on the news that Python 2.6 and 3.0 Beta 1 is officially released. For those of you living on the moon - this release is especially exciting for me for a few reasons:

  1. PEP 371, the addition of the pyprocessing module (as the multiprocessing module) has been implemented.
  2. This was my first serious foray into core-development, so I learned a lot (some rather painfully/embarrassingly).
  3. I ended up with commit privileges so I can help maintain the new package/module. My first checkin was unfortunately a patch to disable some tests that came in with the package.
  4. I killed the buildbots and delayed the beta.

Mad props/thanks need to be passed on to a few people - Benjamin Peterson, Adam Olsen, Richard Oudkerk and many others helped get this done and helped me debug various problems that were exposed after the new package went in.

The docs for the package are here (2.6 dev docs). It's in both py3k and py2.6

I'll be working over the next few weeks on cleaning up the tests, the build and the docs for the new package. I welcome suggestions and urge people to file bugs they find in the bug tracker. Of course, if you include a patch - that's even better.

If you're looking for some cooler-stuff with the package: Look at the examples. The final one is an example of how to use the package to spread work amongst a cluster of machines.

You're going to notice something "special" about the package included in python if you have worked with it in the past: The methods have changed names. There was a discussion on Python-Dev in the context of the package's inclusion about whether or not to stick with "strict" threading module API naming, or to take this as a chance to move towards PEP 8-style naming. We chose the latter, and in fact the threading module itself is getting re-worked over time to move to the same PEP-8 style naming (tbd: I still need to pep this up). I think dropping the old Java-Style naming and going with Pythonic naming and accessing of things will be a generally Good Thing.

I've had to reiterate this a few times - but since I'm cheerleading, I might as well do it again: I do not think that the addition of this package is the "silver bullet" for "concurrency in python". I also don't think it will solve 100% of all the problems out there for people. It's useful, and a best of breed: but it is only one step in a larger movement. Adoption of Adam Olsen's work, continuing to refine the multiprocessing package, and thinking about distributed message protocols within the stdlib are just a few of the things we can do.

I'm seriously thinking about doing a introduction talk about the new package/concurrency stuff at pycon 09, although I don't know how many people would be interested.

Doing all of this work also got me even more excited for 2.6 and 3.0 - I really do recommend people download the new builds and really hammer them, it's important the final versions be as bug-free as possible.

The Abstract Cheetos Attack.

June 18th, 2008 | | Posted in Comedy, Programming

I need to write a CERT paper for this -

  • Name of Attack:The Abstract Cheetos Attack
  • Type of Attack: Password Vulnerability / Local Exploit / Brute Force
  • Known fix: Lack of Cheetos
  • Attack Vector: Food[1]/Social Engineering
  • Exploit Details: It is possible to determine the most-frequently used letters on a given keyboard on the target's computer by providing the target with a "friendly" package of Cheetos at some regular interval, and then examining, over time, the build up of dangerously cheesey residue on the target's keyboard. Armed with the most frequent keystrokes, it is possible to perform a reduced brute-force attack on the target's account password. Due to the reduction in keys, it is possible to grossly reduce the time and resources required to identify the target's password. This attack also enables the attacker to determine other personal information including frequency of hand-washing, like/dislike of said "Food" and frequency at which the target cleans their clothes.

[1] Note that Cheetos may not be classified as "Food"

Programmer Insecurity and Mea Culpa.

June 13th, 2008 | | Posted in Programming, Python

sorry.jpgBen Sussman-Collins put up an excellent blog post on programmer insecurity - this rings particularly loud with me for a few reasons.

The first reason is that I used to be that guy - never checking anything in until I felt it was "perfect", then I swung to the other extreme - putting things in too quickly/aggressively.

It's a fine balance between the two, but one thing he says is especially pertinent in both open, and closed source worlds:

Be transparent. Share your work constantly. Solicit feedback. Appreciate critiques. Let other people point out your mistakes. You are not your code. Do not be afraid of day-to-day failures — learn from them. (As they say at Google, “don’t run from failure — fail often, fail quickly, and learn.”) Cherish your history, both the successes and mistakes. All of these behaviors are the way to get better at programming. If you don’t follow them, you’re cheating your own personal development.

I've grown to truly appreciate peer-review and discussion, I feel it makes all the parties involved that much better, and ultimately it improves the quality of the code. I've grown to miss peer-reviews when I don't have them - the chance to talk over the design of something and step through the code and debate various design points and possible improvements is very, very valuable.

Failure is not permanent with code: It is a transitional state which can be overcome.

That all being said: I must issue a mea culpa. Earlier this week I put together a patch for the multiprocessing package inclusion into python-core. Note that I've been using this package for some time, on multiple platforms - the tests have not failed me, and I felt that things were A-OK.

Once in though, the buildbots started doing something which reminds me of a particularly rowdy party at LinuxWorld way back - namely, all of them started churning away and promptly began puking on themselves. At least unlike me, they didn't misplace clothing or their hotel.

So, I broke the core.

I'm still chasing down the problems - we're suffering test lock-ups and a few compile errors on certain platforms (debian ppc for the loose). I feel awful because I did drop a code-bomb on Tuesday in my urgency to make the beta on wednesday. Dropping something that big right before a deadline is just poor form, and because of it - the betas didn't ship.

With that said: The work Ben Peterson, Adam Olsen, and many others have done to help (me/core) has been phenomenal and it simply reenforces why working in a community is so valuable for me. Now I just have to fix it. Anyone else want to help?

PEP 371: Addition of pyprocessing (as multiprocessing) accepted!

June 5th, 2008 | | Posted in Programming, Python

Per Guido:

I've accepted your PEP. I think it still needs some clean-up and
perhaps clarification of the agreement reached about API style, but
there is nothing now that keeps you from implementing it! Hopefully
you'll make the beta release early next week.

--Guido

Woop woop woop.

Invites for Evernote, Jaiku

June 2nd, 2008 | | Posted in Other, Technology

Yup. I just keep generating fountains of useful things. I have 15 Jaiku invites, and 5 Evernote invites. I recommend both, although right now I use more of twitter until I can convince people to move over to Jaiku cause it's more stable.

And evernote is awesome.

And, as before - leave me a comment or send me an email for an invite - I will need your email address to send it/them to you. Let me say that again: I need your email address. I promise I won't send you wacky Japanese game show youtube links.