The 2.6/3.0 Multiprocessing todo list

July 30th, 2008 § 6 comments

With the lat­est beta’s out in the wild — and the test suite being 100% less locky with the fix for issue874900 — it’s time to but­ton down and start polishing/closing out the remain­ing bugs. Unfor­tu­nately, some of them are, well — “complicated”.

This is more of a per­sonal track­ing for what’s remain­ing — any­one who wants to pitch in and help is more than wel­come (in fact, it’s encour­aged). Work­ing on this stuff — like most of the python-core team is a part time job for me, so the more eyes the merrier.

  • issue3256: Mul­ti­pro­cess­ing docs are not 3.0-ready
    • I need to basi­cally review/apply the patch and run through the exam­ples on both 2.6 and 3.0. Andrii rocks.
  • issue874900:thread­ing mod­ule can dead­lock after fork
    • This bug isn’t in my camp, but I want to see it to com­ple­tion as it very obvi­ously directly impacted the mp package.
  • issue3419: mul­ti­pro­cess­ing mod­ule is racy
    • The con­nec­tion refused error has been resolved, and I need to review Mark’s pro­posed patch to the incref part of the bug
  • issue3321:_multiprocessing.Connection() doesn’t check handle
    • This is from Victor’s fuzzing of the mp mod­ule. Still need to review/test — I attached his test updates to the bug today.
    • Need a windows-happy fix.
  • issue3311:block oper­a­tion on closed socket/pipe for multiprocessing
    • Another fuzzing bug cour­tesy of Victor.
  • issue3352 : Defi­cien­cies in multiprocessing/threading API
    • Work has started on this one cour­tesy of Ben Peter­son — I am going to start focus­ing on this as well.
  • issue3125: test_multiprocessing causes test_ctypes to fail
    • This one is done, pend­ing a merge of the fix to py3k — see issue3385
  • issue3270: test_multiprocessing: test_listener_client flakiness
    • Just pend­ing an added doc warn­ing for the 0.0.0.0 issue trent brought up.
  • issue3350 : mul­ti­pro­cess­ing adds built-in types to the global copyreg.dispatch_table
    • Can prob­a­bly be closed with the fix to issue3125
  • issue3206:Mul­ti­pro­cess­ing Array and sharedctypes.Array error in docs/implementation
    • Doc issue, patch pend­ing — I need to review. Andrii again, rocks.
  • issue3111 : mul­ti­pro­cess­ing ppc Debian/ ia64 Ubuntu com­pi­la­tion error
    • I added a check to sim­ply not run the tests that were fail­ing on the plat­forms which raise an excep­tion when cre­at­ing the RLock — not the best, or final solu­tion. Need to revisit.
  • issue3110 : Mul­ti­pro­cess­ing pack­age build prob­lem on Solaris 10
    • Solaris build prob­lem: I lack a solaris machine to test with — and my solaris-fu is weak sauce.
  • issue3149 : mul­ti­pro­cess­ing build fails on Solaris 10
  • issue3578: ‘dic­tio­nary changed size’ error in test_multiprocessing
    • Antoine may have found the issue — need to research
  • issue3518: mul­ti­pro­cess­ing: BaseManager.from_address doc­u­mented but doesn’t exist
    • Doc issue
  • issue3518: Mis­lead­ing names for mul­ti­pro­cess­ing “con­ve­nience” functions
    • Easy to fix.

    Last Updated: Aug 18 2008

    And that con­cludes “the list” — I still need to update the PEP with some minor edits, expand the test suite/make it sim­pler among other things. Since this is more of a hit-list for me, I’ll keep this post updated with the move­ment of all of the tasks.

    So far, the most chal­leng­ing part of this is not the tech­ni­cal aspect: It’s the making-the-time-to-get-it-done aspect. In the grand scheme of life — it’s home(baby) > work > open source. Good times.

    • Brett

      Any rea­son you didn’t do the py3k merge your­self for issue3125?

    • jnoller

      Because Alexan­dre removed dis­patch from pickle: “Here is a pro­posed patch for py3k gen­er­ated from an svn merge of amaury’s patch into py3k. This is cur­rently blocked due to issue3385.” — Basi­cally the cPickle->pickle con­ver­sion was lossy — Alexan­dre is work­ing on a patch

    • http://bitheap.org/ Brodie Rao

      For mem­ory issues on OS X I rec­om­mend using Guard Mal­loc. I ran into a weird mem­ory allo­ca­tion bug in Python recently that caused seem­ingly ran­dom crashes (see issue3242). I was able to nar­row down the behav­ior to a con­sis­tent and more use­ful kind of crash with it.

    • http://pybites.blogspot.com Ben­jamin

      You are a very coura­geous fel­low! Keep up the good work!

    • jnoller

      Don’t con­fuse igno­rance with courage. :)

    • http://jessenoller.com jnoller

      Don’t con­fuse igno­rance with courage. :)

    • What's this?

    You are currently reading The 2.6/3.0 Multiprocessing todo list at jessenoller.com.

    meta