Threads can’t be serialized?!

October 8th, 2008 § 10 comments

Note the sub­ject is tongue-in-cheek — I real­ize you can’t pickle threading.Thread and multiprocessing.Process, but for some (pos­si­bly sick) rea­son I want to do an imple­men­ta­tion for both that imple­ments __getstate__ and __setstate__ and changed the __repr__ for both to not refer to any sort of run-time state.

You don’t need to seri­al­ize the func­tion thats passed in: the assump­tion here is that you’re seri­al­iz­ing them prior to call­ing start() (because seri­al­iz­ing a run­ning thread would be awe­some — in that “hey my brain just melted” sort of way).

The best (read, only sane) use case would be to allow threads/processes to be defined in say, a YAML file and gen­er­ated at parse-time (the client could call .start()) or send­ing a blob of seri­al­ized threads to a remote client and hav­ing it call start() to act as a slave.

And that con­cludes the ran­dom thought for the day.

Edit: And in the com­ments, Ben Hay­den posted a link to a ver­sion of threading.Thread which can be seri­al­ized. You can see it here. I need to make a multiprocessing.Process ver­sion of it too, just, well, cause.

  • x

    stack­less can!

  • http://www.thok.org/bloggery _Mark_

    That’s one of the cool things about Stack­less Python, no? you can pickle microthreads, and ISTR move them between machines…

  • jnoller

    Yes, but I don’t want to have to go to stackless!

  • jnoller

    Yeah, but that’s like say­ing “twisted can do that” — I’d like to not to have to add in a large exter­nal depen­dency. Of course, if I went stackless/twisted, I’d prob­a­bly never want for any­thing ever again! :)

  • http://bob.pythonmac.org/ Bob Ippolito
  • Miguel S Filipe

    Look for “Call with Cur­rent Con­tin­u­a­tion” on Lisp or lisp like lan­guages.
    the name of what you are look­ing for is: continuations.

  • http://code.google.com/p/benjhayden/ Ben Hay­den
  • jnoller

    Oh my, it’s full of stars. Thanks for this — I was mostly being
    tongue in cheek with my post, I knew what needed to be done, but you
    actu­ally doing it is awesome.

    I’ll do a pro­cess­ing ver­sion tomor­row if I have a some time at lunch

  • jnoller

    No, I don’t think con­tin­u­a­tions are what I want — I know what a con­tin­u­a­tion is, and in my case I want to be able to gen­er­ate a pool of threads with assigned function/args and push them over the wire (or parse them from a con­fig file) and “use them blindly”.

  • http://jessenoller.com jnoller

    No, I don’t think con­tin­u­a­tions are what I want — I know what a con­tin­u­a­tion is, and in my case I want to be able to gen­er­ate a pool of threads with assigned function/args and push them over the wire (or parse them from a con­fig file) and “use them blindly”.

What's this?

You are currently reading Threads can’t be serialized?! at jessenoller.com.

meta