PEP 3148 Accepted: “futures – execute computations asynchronously”

Back in May, Guido assigned me “single-use” BDFL powers when deciding on the state of PEP 3148. I had been working on and off with Brian since his original proposal to the stdlib-sig mailing list.

Tonight, I “officially accepted” the PEP Brian’s put a lot of work into. You can check out the PEP itself, or play with the code over here. The credit goes to Brian, and the small army of people who gave him feedback including Jeffery Yasskin who spent a fair amount of time iterating and expounding on things.

Below, is a snippet from one of my original emails to python-dev on my reasoning for supporting the library, and also adding in the “concurrent” namespace.

Baloney. A young library providing some syntactic sugar which uses
primitives in the standard library to implement a common pattern is
fine for a PEP. We’ve hashed this out pretty heavily on the stdlib-sig
list prior to bringing it here. By the same argument, we should shunt
all of the recent unittest changes and improvements into space, since
golly, other people did it, why should we.

This is something relatively simple, which I would gladly add in an
instant to the multiprocessing package – but Brian’s one-upped me in
that regard and is providing something which works with both threads
and processes handily. Take a look at multiprocessing.Pool for example
- all that is some sugar on top of the primitives, but it’s good
sugar, and is used by a fair number of people.

Let me also state – “my” vision of where futures would live would be
in a concurrent package – for example:

from concurrent import futures

The reason *why* is that I would like to also move the abstractions I
have in multiprocessing *out* of that module, make them work with both
threads and processes (if it makes sense) and reduce the
multiprocessing module to the base primitive Process object. A
concurrent package which implements common patterns built on top of
the primitives we support is an objectively Good Thing.

For example, how many of us have sat down and implemented a thread
pool on top of threading, I would hazard to say that most of us who
use threading have done this, and probably more than once. It stands
to reason that this is a common enough pattern to include in the
standard library.

In any case; consider me a strong +1 to adding it.

With that done, I am back to being only the benevolent dictator of my lawn, which is mostly dead.

Also, for added fun – check out the PyCon AU talk Brian did!

  • (not following the original discussions)
    I hope the new concurent package would also be usefull with gevent/greenlets too
  • You can make custom subclasses as needed. The core libraries do not need to account for 3rd party libraries.
  • tmalsburg
    Why "if future.exception() is not None"? (I'm referring to this example:http://code.google.com/p/pythonfutures/ ) Is Python becoming the next C? Could the exception not be thrown when the result is accessed, i.e. on future.result().
    I find it really ugly that this proposal bypasses Python's exception handling and comes up with a new mechanism.
  • Baczek
    i hope this was done with at least some cooperation with twisted guys...
  • The twisted folk provided feedback
  • Baczek
    glad to hear that.
  • Nick Coghlan
    I recommend that talk - it did a nice job of showing how easy the PEP makes it to take an iterative algorithm using map and turn it into a parallel algorithm using futures.
  • reidkleckner
    Hooray for new code! :)
  • Yay!
  • Anonymous
    minor gripe -- not a fan of the long, camelcase names in this module. Please don't let java naming atrocities infect python/modules.
  • Those are class names, and classes in python are typically camel case.
blog comments powered by Disqus