multiprocessing.Pool and KeyboardInterrupt

January 8th, 2009 § 15 comments

Some­one pinged me recently — he was hav­ing a prob­lem han­dling a key­board inter­rupt when using multiprocessing.Pool.

Fun­da­men­tally, the prob­lem comes with the fact that if a process gets an excep­tion, it just throws the excep­tion instead of return­ing or becom­ing .join()-able by the par­ent. You want to use pool.terminate() in the main func­tion — but you also need to cap­ture and han­dle the excep­tion in the child so that when then excep­tion is caught — the process can exit “gracefully”:

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

import multiprocessing
import time
 
def create():
    try:
        time.sleep(10)
    except KeyboardInterrupt:
        return 
    return 'hi mom'
 
def main():
    def cb(what):
        print what
 
    pool = multiprocessing.Pool(2)
    try:
        for i in range(2):
            pool.apply_async(create, args=(), callback=cb)
        pool.close()
        pool.join()
    except KeyboardInterrupt:
        print 'control-c presd butan'
        pool.terminate()
 
if __name__ == "__main__":
    main()

If the excep­tion is thrown within the child, and it just raises — it becomes unman­age­able by the par­ent. As you can see from this piece of code (from pool.py, python trunk) the Pool is attempt­ing to join() the child — which is of course, unjoin­able (sort of like me, when I was a teenager):

?View Code PYTHON
1
2
3
4
5
6
7
8

    @classmethod
    def _terminate_pool(cls, taskqueue, inqueue, outqueue, pool,
                        task_handler, result_handler, cache):
        ...
        if pool and hasattr(pool[0], 'terminate'):
            debug('joining pool workers')
            for p in pool:
                p.join()

  • Ben

    You almost made me laugh with that joke… almost…

  • jnoller

    Nerd humor for the loose.

  • http://jessenoller.com jnoller

    Actu­ally, let me amend this. The library *does* cor­rectly han­dle excep­tions thrown in the child — if you mod­ify the exam­ple in my post and just add a bro­ken import (don’t hit control-c) the pool will exit gracefully.

  • Jean-Paul

    Why doesn’t apply_async have a means for receiv­ing noti­fi­ca­tion of errors from the child process? This isn’t just related to C-c, right? Any excep­tion raised in the child will cause the process to go into an unman­age­able state — that seems like a ter­ri­ble way to deal with errors.

  • jnoller

    I don’t dis­agree. I’m think­ing about how to change it under the cov­ers. I may file an enhance­ment against myself to track it.

  • Jean-Paul

    Great. :)

  • jnoller

    See below: excep­tions within the chil­dren are han­dled prop­erly in some cases.

  • gaius.julius

    any­ways, what should I do to grace­fully han­dle Key­board­Inter­rupt and other sim­i­lar cases when using Pool?

  • jnoller

    See the main post. A key­board inter­rupt is not han­dled the same as other excep­tions. At least in py2.5-forward the solu­tion in my main blog post works.

  • Aki

    Inter­est­ing.
    I did it dif­fer­ently.
    I dis­abled SIGINT to any child process using signal.signal(signal.SIGINT, signal.SIG_IGN)
    The par­ent takes SIGINT then send a mes­sage (or set an event) to each child ask­ing it to shoudown.

  • Shim­mer

    I tried your exam­ple and got errors with­out the nice mes­sages as expected:

    None
    None
    Trace­back (most recent call last):
    File “testpool.py”, line 26, in <mod­ule>
    main()
    File “testpool.py”, line 23, in main
    pool.join()
    File “C:Python26libmultiprocessingpool.py”, line 340, in join
    self._result_handler.join()
    File “C:Python26libthreading.py”, line 634, in join
    self.__block.wait()
    File “C:Python26libthreading.py”, line 237, in wait
    waiter.acquire()
    Key­board­Inter­rupt

  • http://jessenoller.com jnoller

    Some­thing has changed. The nice mes­sages should be being raised.

  • Joe

    If you change range(2) to range(200) it does not exit cor­rectly. How would you solve this?

  • Casey

    What and how is _terminate_pool sup­posed to be used? Can you add some more detail?

  • Casey

    What and how is _terminate_pool sup­posed to be used? Can you add some more detail?

What's this?

You are currently reading multiprocessing.Pool and KeyboardInterrupt at jessenoller.com.

meta