Comments on: Multiprocessing in hindsight.

By: Eli

Eli — Tue, 03 Feb 2009 23:07:43 +0000

As other commenters have said, multiprocessing is one of the greatest new features in 2.6 - it's the main reason I'm itching so hard to switch to 2.6 (though unfortunately I can't, until some libs I use do).

And having bugs is not a sin. Especially for 2.6, which is a version most production-heavy users haven't switched to yet, so it's a good testing ground.

By: Eli

Eli — Tue, 03 Feb 2009 18:07:43 +0000

By: jnoller

jnoller — Sun, 01 Feb 2009 12:39:11 +0000

I was limiting my “damn magic” comment to MPs test suite. Way too much magic.

By: Brett

Brett — Sun, 01 Feb 2009 03:58:16 +0000

So, I screwed up and importlib only has 127 tests; my test auto-discovery code was executing both the source code as well as the bytecode files. Oops. =)

And I don't know if your "god damn magic" comment is limited to MP or Python's entire test suite, but I definitely want to remove the magic from Python's own test suite. I think that will be my next big project after importlib, DVCS, and the dev docs.

By: Brian 2

Brian 2 — Thu, 29 Jan 2009 19:18:11 +0000

Python badly needed a standardized way to use multiple processors. The official response to complaints about the GIL was “threads are bad, use processes”, which is fine, but there was no convenient way to actually do that. Getting multiprocessing into 2.6 and 3.0 even with known issues is a much better result than having no multicore solution at all until 2.7/3.1. I’ll definitely be taking advantage of it, and thank you very much for your efforts.

By: Seun Osewa

Seun Osewa — Thu, 29 Jan 2009 18:05:41 +0000

I also want to say the same thing. If a mistake was made, it was that it wasn’t accepted *earlier*. Not including it at all would have been a worse outcome.

By: Larry Hastings

Larry Hastings — Thu, 29 Jan 2009 02:54:48 +0000

multiprocessing : Python 2.6 :: STL : 1994 ANSI C++ Draft Standard

By: jnoller

jnoller — Wed, 28 Jan 2009 21:10:01 +0000

Personally? I think it’s ready now for many applications, but yes — there’s a handful of bug fixes I need to get in for the next 2.6 rel and 3.xxx

By: jnoller

jnoller — Wed, 28 Jan 2009 21:06:33 +0000

Nah, the GIL naysayers are still there — trust me. I try to avoid the bike shed burning and repainting as much as possible.

As for the tests — way to RAIN ON MY PARADE :) We do need coverage results, I still don’t have accurate numbers on the MP code. Not to mention, the current test suite might as well be called “god damn magic” sometimes. Needs less magic.

By: Brett C.

Brett C. — Wed, 28 Jan 2009 21:01:21 +0000

Yes it probably would have been easier had we waited, but there is always some snafu in a release. Considering how much slack CPython gets over the GIL I appreciate the module as the GIL nay-sayers seem to have shut up for once.

And 124 tests? Importlib has 226! Come on, you can do better than that! =) But obviously the real question is what kind of code coverage you get. Which reminds me, I still think we need to get regular test coverage results for trunk and py3k so everyone can know where we come up short in our tests.