The 2.6/3.0 Multiprocessing todo list

With the latest beta’s out in the wild – and the test suite being 100% less locky with the fix for issue874900 – it’s time to button down and start polishing/closing out the remaining bugs. Unfortunately, some of them are, well – “complicated”.

This is more of a personal tracking for what’s remaining – anyone who wants to pitch in and help is more than welcome (in fact, it’s encouraged). Working on this stuff – like most of the python-core team is a part time job for me, so the more eyes the merrier.

  • issue3256: Multiprocessing docs are not 3.0-ready
    • I need to basically review/apply the patch and run through the examples on both 2.6 and 3.0. Andrii rocks.
  • issue874900:threading module can deadlock after fork
    • This bug isn’t in my camp, but I want to see it to completion as it very obviously directly impacted the mp package.
  • issue3419: multiprocessing module is racy
    • The connection refused error has been resolved, and I need to review Mark’s proposed patch to the incref part of the bug
  • issue3321:_multiprocessing.Connection() doesn’t check handle
    • This is from Victor’s fuzzing of the mp module. Still need to review/test – I attached his test updates to the bug today.
    • Need a windows-happy fix.
  • issue3311:block operation on closed socket/pipe for multiprocessing
    • Another fuzzing bug courtesy of Victor.
  • issue3352 : Deficiencies in multiprocessing/threading API
    • Work has started on this one courtesy of Ben Peterson – I am going to start focusing on this as well.
  • issue3125: test_multiprocessing causes test_ctypes to fail
    • This one is done, pending a merge of the fix to py3k – see issue3385
  • issue3270: test_multiprocessing: test_listener_client flakiness
    • Just pending an added doc warning for the 0.0.0.0 issue trent brought up.
  • issue3350 : multiprocessing adds built-in types to the global copyreg.dispatch_table
    • Can probably be closed with the fix to issue3125
  • issue3206:Multiprocessing Array and sharedctypes.Array error in docs/implementation
    • Doc issue, patch pending – I need to review. Andrii again, rocks.
  • issue3111 : multiprocessing ppc Debian/ ia64 Ubuntu compilation error
    • I added a check to simply not run the tests that were failing on the platforms which raise an exception when creating the RLock – not the best, or final solution. Need to revisit.
  • issue3110 : Multiprocessing package build problem on Solaris 10
    • Solaris build problem: I lack a solaris machine to test with – and my solaris-fu is weak sauce.
  • issue3149 : multiprocessing build fails on Solaris 10
  • issue3578: ‘dictionary changed size’ error in test_multiprocessing
    • Antoine may have found the issue – need to research
  • issue3518: multiprocessing: BaseManager.from_address documented but doesn’t exist
    • Doc issue
  • issue3518: Misleading names for multiprocessing “convenience” functions
    • Easy to fix.

    Last Updated: Aug 18 2008

    And that concludes “the list” – I still need to update the PEP with some minor edits, expand the test suite/make it simpler among other things. Since this is more of a hit-list for me, I’ll keep this post updated with the movement of all of the tasks.

    So far, the most challenging part of this is not the technical aspect: It’s the making-the-time-to-get-it-done aspect. In the grand scheme of life – it’s home(baby) > work > open source. Good times.

    • You are a very courageous fellow! Keep up the good work!
    • Don't confuse ignorance with courage. :)
    • For memory issues on OS X I recommend using Guard Malloc. I ran into a weird memory allocation bug in Python recently that caused seemingly random crashes (see issue3242). I was able to narrow down the behavior to a consistent and more useful kind of crash with it.
    • Brett
      Any reason you didn't do the py3k merge yourself for issue3125?
    • Because Alexandre removed dispatch from pickle: "Here is a proposed patch for py3k generated from an svn merge of amaury's patch into py3k. This is currently blocked due to issue3385." - Basically the cPickle->pickle conversion was lossy - Alexandre is working on a patch
    blog comments powered by Disqus