PEP 3148 Accepted: “futures — execute computations asynchronously”

July 11th, 2010 § 12 comments

Back in May, Guido assigned me “single-use” BDFL pow­ers when decid­ing on the state of PEP 3148. I had been work­ing on and off with Brian since his orig­i­nal pro­posal to the stdlib-sig mail­ing list.

Tonight, I “offi­cially accepted” the PEP Brian’s put a lot of work into. You can check out the PEP itself, or play with the code over here. The credit goes to Brian, and the small army of peo­ple who gave him feed­back includ­ing Jef­fery Yasskin who spent a fair amount of time iter­at­ing and expound­ing on things.

Below, is a snip­pet from one of my orig­i­nal emails to python-dev on my rea­son­ing for sup­port­ing the library, and also adding in the “con­cur­rent” namespace.

Baloney. A young library pro­vid­ing some syn­tac­tic sugar which uses
prim­i­tives in the stan­dard library to imple­ment a com­mon pat­tern is
fine for a PEP. We’ve hashed this out pretty heav­ily on the stdlib-sig
list prior to bring­ing it here. By the same argu­ment, we should shunt
all of the recent unittest changes and improve­ments into space, since
golly, other peo­ple did it, why should we.

This is some­thing rel­a­tively sim­ple, which I would gladly add in an
instant to the mul­ti­pro­cess­ing pack­age — but Brian’s one-upped me in
that regard and is pro­vid­ing some­thing which works with both threads
and processes hand­ily. Take a look at multiprocessing.Pool for exam­ple
- all that is some sugar on top of the prim­i­tives, but it’s good
sugar, and is used by a fair num­ber of people.

Let me also state — “my” vision of where futures would live would be
in a con­cur­rent pack­age — for example:

from con­cur­rent import futures

The rea­son *why* is that I would like to also move the abstrac­tions I
have in mul­ti­pro­cess­ing *out* of that mod­ule, make them work with both
threads and processes (if it makes sense) and reduce the
mul­ti­pro­cess­ing mod­ule to the base prim­i­tive Process object. A
con­cur­rent pack­age which imple­ments com­mon pat­terns built on top of
the prim­i­tives we sup­port is an objec­tively Good Thing.

For exam­ple, how many of us have sat down and imple­mented a thread
pool on top of thread­ing, I would haz­ard to say that most of us who
use thread­ing have done this, and prob­a­bly more than once. It stands
to rea­son that this is a com­mon enough pat­tern to include in the
stan­dard library.

In any case; con­sider me a strong +1 to adding it.

With that done, I am back to being only the benev­o­lent dic­ta­tor of my lawn, which is mostly dead.

Also, for added fun — check out the PyCon AU talk Brian did!

  • Anony­mous

    minor gripe — not a fan of the long, camel­case names in this mod­ule. Please don’t let java nam­ing atroc­i­ties infect python/modules.

  • rei­d­kleck­ner

    Hooray for new code! :)

  • http://www.boredomandlaziness.org Nick Cogh­lan

    I rec­om­mend that talk — it did a nice job of show­ing how easy the PEP makes it to take an iter­a­tive algo­rithm using map and turn it into a par­al­lel algo­rithm using futures.

  • http://jessenoller.com jnoller

    Those are class names, and classes in python are typ­i­cally camel case.

  • http://jessenoller.com jnoller

    Yay!

  • Baczek

    i hope this was done with at least some coop­er­a­tion with twisted guys…

  • http://jessenoller.com jnoller

    The twisted folk pro­vided feedback

  • Baczek

    glad to hear that.

  • tmals­burg

    Why “if future.exception() is not None”? (I’m refer­ring to this exam­ple:http://code.google.com/p/pythonfutures/ ) Is Python becom­ing the next C? Could the excep­tion not be thrown when the result is accessed, i.e. on future.result().
    I find it really ugly that this pro­posal bypasses Python’s excep­tion han­dling and comes up with a new mechanism.

  • http://jessenoller.com jnoller

    Ni

  • http://damjan.softver.org.mk/ gdam­jan

    (not fol­low­ing the orig­i­nal dis­cus­sions)
    I hope the new con­curent pack­age would also be use­full with gevent/greenlets too

  • http://jessenoller.com jnoller

    You can make cus­tom sub­classes as needed. The core libraries do not need to account for 3rd party libraries.

What's this?

You are currently reading PEP 3148 Accepted: “futures — execute computations asynchronously” at jessenoller.com.

meta