Threads can’t be serialized?!

Note the subject is tongue-in-cheek – I realize you can’t pickle threading.Thread and multiprocessing.Process, but for some (possibly sick) reason I want to do an implementation for both that implements __getstate__ and __setstate__ and changed the __repr__ for both to not refer to any sort of run-time state.

You don’t need to serialize the function thats passed in: the assumption here is that you’re serializing them prior to calling start() (because serializing a running thread would be awesome – in that “hey my brain just melted” sort of way).

The best (read, only sane) use case would be to allow threads/processes to be defined in say, a YAML file and generated at parse-time (the client could call .start()) or sending a blob of serialized threads to a remote client and having it call start() to act as a slave.

And that concludes the random thought for the day.

Edit: And in the comments, Ben Hayden posted a link to a version of threading.Thread which can be serialized. You can see it here. I need to make a multiprocessing.Process version of it too, just, well, cause.

  • Oh my, it's full of stars. Thanks for this - I was mostly being
    tongue in cheek with my post, I knew what needed to be done, but you
    actually doing it is awesome.

    I'll do a processing version tomorrow if I have a some time at lunch
  • Miguel S Filipe
    Look for "Call with Current Continuation" on Lisp or lisp like languages.
    the name of what you are looking for is: continuations.
  • No, I don't think continuations are what I want - I know what a continuation is, and in my case I want to be able to generate a pool of threads with assigned function/args and push them over the wire (or parse them from a config file) and "use them blindly".
  • That's one of the cool things about Stackless Python, no? you can pickle microthreads, and ISTR move them between machines...
  • Yeah, but that's like saying "twisted can do that" - I'd like to not to have to add in a large external dependency. Of course, if I went stackless/twisted, I'd probably never want for anything ever again! :)
  • x
    stackless can!
  • Yes, but I don't want to have to go to stackless!
blog comments powered by Disqus