Unladen Swallow: Python 3’s Best Feature.

January 6th, 2010 § 24 comments

holy-shit-awesome-2.jpgWe all know (well — unless you’ve under a rock) about Unladen-Swallow, the semi-Google-Sponsored optimization-focused branch of Python 2.x. Collin, Jef­frey and many oth­ers have been work­ing tire­lessly on port­ing the CPython inter­preter over to LLVM, apply­ing opti­miza­tion patches, writ­ing tests, etc all aimed at speed­ing up real-world oper­a­tions and code since before last year’s PyCon — at that time, the first release was already work­ing hard, ren­der­ing YouTube’s tem­plates. (See the Project Plan for more information)

You prob­a­bly also saw the 2009Q3 testing/performance results (here), show­ing a hefty speed increase for var­i­ous oper­a­tion. The 2009Q4 will be com­ing soon.

What you prob­a­bly don’t know — yet, is that a PEP is com­ing, propos­ing Unladen Swal­low be merged back to Python-core — specif­i­cally, the Python 3k branch. Yeah, that’s right — a PEP to merge Unladen Swal­low back into the moth­er­ship, but to Py3k only. Talk about a shot in the arm!

Quot­ing Collin Winter:

Python 3 is the nat­ural place for some­thing like Unladen Swal­low to land: Python 3 is clearly the future, and I think improved per­for­mance will be a strong sell­ing point in the language’s favor. What we and oth­ers have done for Python 2 applies directly to Python 3.

…

One of the things we’ve really focused on with Unladen Swal­low is cre­at­ing a solid foun­da­tion that the wider CPython devel­oper com­mu­nity can build on top of. We’ve fixed some nasty prob­lems in the x86-64 JIT, for exam­ple, so that the CPython com­mu­nity can focus on the fun stuff — more opti­miza­tions! Hav­ing the basic JIT infra­struc­ture in place and well-tested opens up a world of oppor­tu­ni­ties that didn’t exist before.

…

I think PyPy’s focus on research is incred­i­bly valu­able. Unladen Swal­low has always been about, “what can we achieve right now?” One of our guid­ing prin­ci­ples has been, Do noth­ing orig­i­nal, don’t be inno­v­a­tive. PyPy is look­ing 10 years down the road and think­ing, What can we do if we ques­tion every­thing? I think hav­ing those dif­fer­ent per­spec­tives is impor­tant to the com­mu­nity. So, no, I don’t see us in com­pe­ti­tion (with PyPy) at all.

There’s been a lot of talk about how slow adop­tion of 3k has been — a back­wards incom­pat­i­ble imple­men­ta­tion of a widely used lan­guage will always have slow(er) adop­tion then releases of the same lan­guage, back­wards com­pat­i­ble — but I’d argue that to an extent, part of the rea­son is that there “haven’t been enough car­rots” in the Py3k cart to pro­vide enough of an incen­tive for projects, libraries, etc to move to it at an accel­er­ated rate.

This, well — this changes things. A sig­nif­i­cantly faster inter­preter, a more ratio­nal GIL thanks to Antoine Pitrou — these are the improve­ments to Python 3 which make it the per­fect tar­get to aim for for new projects and libraries. It should dras­ti­cally help pro­vide the impe­tus for larger libraries to move over. Things like these are exactly why some­thing like the mora­to­rium make sense, efforts can be focused on the ecosys­tem of python out­side of the syn­tax, on the inter­preter, the stan­dard library, etc.

This is huge; and with a lit­tle luck, a lot of dis­cus­sion and debate — and a ton of work, by the end of 2010, we’ll have not only the best imple­men­ta­tion of Python, as a lan­guage to date in Python 3 — but the best inter­preter as well.

The PEP should go out for review soon, prob­a­bly within the next two weeks — and more details/information will be com­ing out at PyCon in Feb­ru­ary (you’re com­ing, right?!).

Also see Michael Foord’s post about this news too.

  • rhodium

    I com­pletely agree with you wrt this “can” be the defin­ing car­rot to push peo­ple to 3k. The mora­to­rium only gives this the needed push in order to show where it “should” go. Really good stuff — I can’t wait!

  • rhodium

    I com­pletely agree with you wrt this “can” be the defin­ing car­rot to push peo­ple to 3k. The mora­to­rium only gives this the needed push in order to show where it “should” go. Really good stuff — I can’t wait!

  • http://douglasandrade.com/ Dou­glas Soares de Andrade

    Hi !

    Hope this come to life !! Totally +1 from me. Collin went to pycon­brasil and showed us some great things about unlad­den and i was really impressed.

    I guess if they merge, Python 3 will be the best and new python devel­op­ers (libs,apps,frameworks) will use it more !

    Long life python, unlad­den, pypy and all these good things =)

  • http://douglasandrade.com/ Dou­glas Soares de Andrade

    Hi !

    Hope this come to life !! Totally +1 from me. Collin went to pycon­brasil and showed us some great things about unlad­den and i was really impressed.

    I guess if they merge, Python 3 will be the best and new python devel­op­ers (libs,apps,frameworks) will use it more !

    Long life python, unlad­den, pypy and all these good things =)

  • http://douglasandrade.com/ Dou­glas Soares de Andrade

    Hi !

    Hope this come to life !! Totally +1 from me. Collin went to pycon­brasil and showed us some great things about unlad­den and i was really impressed.

    I guess if they merge, Python 3 will be the best and new python devel­op­ers (libs,apps,frameworks) will use it more !

    Long life python, unlad­den, pypy and all these good things =)

  • http://glyph.twistedmatrix.com/ Glyph

    YESSSSSSSS!!!!!!

  • http://twitter.com/McJeff Jeff McNeil

    Typ­ing this a sec­ond time as my login seems to have trig­gered a pre-POST sub­mis­sion text box digestion.

    I agree with this from a techie per­spec­tive. I think it’s a great idea to merge the Unladen Swal­low code and the CPython proper. It can only mean good things for the com­mu­nity as a whole. At the very least, it ought to serve as a boon for Py3K development.

    Though, I keep going back to the fact that I don’t nec­es­sar­ily use Python for its speed. If I want speed, it’s C. Or, more often than that, I throw a box or three at the prob­lem as that’s cheaper than my time. I just can’t recall a prob­lem recently where I’ve said “Hrm, if this was only 35% faster…” I love the lan­guage because of the speed it gives me in trans­lat­ing ideas into func­tional source code.

    I would really like to help with the port­ing of some of these pack­ages. The prob­lem is that I don’t know where to begin! Sure, I can search the cheese shop, but even then I’ve got to deal with sub­mit­ting patches back to main­tain­ers that may or may not even be inter­ested. A while back I sug­gested via blog entry that I think it would be nice to have a 2-to-3 sta­tus page out there so that indi­vid­u­als (like myself) who are will­ing to help out can work down the check list.

    Over­all, I love it! Hope­fully it’s enough to push the num­ber of avail­able libraries and com­po­nents to crit­i­cal mass and spur on a more wide­spread adoption.

    Though, those of us stuck using RHEL 5 will be for­ever damned to Python 2.4!

  • cul­ture­anew

    “I would really like to help with the port­ing of some of these pack­ages. The prob­lem is that I don’t know where to begin!”

    FWIW, recently, in search of some­thing use­ful to do, I thought port­ing some­thing to Py3k might be fun, or maybe patch­ing some­thing. After pok­ing around python.org, and the bug data­base, I wan­dered off, discouraged.

    Man­age it and they will code…

  • Mag­nus Lyckå

    It’s about time to stop using RHEL 5, and it’s about time to let go of Python 2.5 sup­port. Once that’s done, once 2.6 is the old­est plat­form to sup­port for 3rd party libraries, sup­port­ing Python 3 become much less of a challenge.

  • http://jessenoller.com jnoller

    Why would 3rd party pack­ages, com­mu­nity libraries or projects be in
    the python-core bug tracker? If you want to port some­thing, look at
    http://pypi.python.org/pypi and con­tact the project owner!

  • http://twitter.com/hackerjack60 Jack Jansen

    Good plan!
    This is exactly the sort of push I need to get me to move to Python 3.

  • Peter

    Part of the prob­lem with Py3k is that many of the users don’t agree it is bet­ter. The new print syn­tax is bro­ken. We can no longer for­mat strings the old fash­ioned way. Both of these “improve­ments” are big down­grades. Peo­ple don’t know what makes good lan­guages. Some lan­guages get lucky, and end up very usable. Python did. Python 3k didn’t. Time to move on.

  • http://jessenoller.com jnoller

    The new print syn­tax is bet­ter (see what I did there?) — you can still
    for­mat strings the old way.

  • http://twitter.com/jasonsupdates Jason Baker

    Isn’t unladen swal­low devel­op­ing against Python 2.6? Are they plan­ning to make a Python 3 version?

  • http://jessenoller.com jnoller

    Yeah — RHEL is the bane of New Things, which I think is the point :)

    As for where to begin, I have an idea for an index like you sug­gest — I need to think about it more and find some­one will­ing to push it out, as I’ve got a pile of Other Really Impor­tant Things to Do. :(

  • http://jessenoller.com jnoller

    They are cur­rently devel­op­ing against trunk (2.7) how­ever; given their focus is on the byte­code layer, the tran­si­tion to Python 3 syn­tax is rel­a­tively simple(ish).

  • Masklinn

    > The new print syn­tax is broken.

    How? (and not only do I don’t see how it’s bro­ken, I do find it much bet­ter than the pre­vi­ous one espe­cially when doing any­thing beyond a vanilla print)

    > We can no longer for­mat strings the old fash­ioned way.

    You are wrong. The ‘%’ oper­a­tor is con­sid­ered obso­lete in Python 3, but it’s still here: http://docs.python.org/3.1/library/stdtypes.htm…

    IOW, you don’t give any jus­ti­fi­ca­tion to your first dec­la­ra­tion and the sec­ond one is plain wrong.

    > Python 3k didn’t.

    Python 3 is a bet­ter and more use­able lan­guage than Python 2 on pretty much every front. The only thing it’s miss­ing is libraries adop­tion, and I trust that this will right itself within a year or two at worst.

  • http://regebro.wordpress.com/ Lennart Rege­bro

    No, I’m not com­ing to PyCon, so you bet­ter keep me updated. ;-) This is fan­tas­tic news, I hope it will happen.

  • http://jessenoller.com jnoller

    I will try to! :)

  • Toot

    That seems like an ambi­tious under­tak­ing. But Python 3 really needs some­thing spe­cial to attract peo­ple. Right now it seems way to lit­tle ben­e­fit for a lot of pain of not being able to use older packages.

    But putting it into Python 3 for the “sole” pur­pose of being the car­rot — when in fact it cur­rently runs with python 2.X — that’s a lit­tle bit questionable.

  • http://twitter.com/McJeff Jeff McNeil

    I’ve the same (seem­ingly end­less) pile here, too. Though, what do you have in mind? If I can find a few cycles (and that’s a big *if*) I’d be will­ing to help a bit.

    Gen­er­ally, any­thing with the words “Enter­prise” or “Proven” in them tend to be tech­no­log­i­cal anchors.

  • http://regebro.wordpress.com/ Lennart Rege­bro

    “The prob­lem is that I don’t know where to begin! ”

    Well, start with port­ing some­thing that you your­self uses. If you don’t know how, it’s true that the doc­u­men­ta­tion is *very* lack­ing at the moment. I’m writ­ing an series of arti­cles in The Python Mag­a­zine start­ing in the Jan­u­ary issue, those might help.

  • http://stuartaxon.com/ Stu

    Hm, as far as I remem­ber the prob­lem was that unladen swal­low is based on LLVM which isn’t avail­able on as many plat­forms as CPython is.
    This would mean that the orig­i­nal imple­men­ta­tion would have to hang around until LLVM became avail­able on all the other architectures.

  • http://stuartaxon.com/ Stu

    Hm, as far as I remem­ber the prob­lem was that unladen swal­low is based on LLVM which isn’t avail­able on as many plat­forms as CPython is.
    This would mean that the orig­i­nal imple­men­ta­tion would have to hang around until LLVM became avail­able on all the other architectures.

What's this?

You are currently reading Unladen Swallow: Python 3’s Best Feature. at jessenoller.com.

meta