jessenoller.com

Recently, I made an off­hand com­ment here about:

I’ve actu­ally started think­ing about/sketching an actor model build on top of MP, using con­cepts from actors/monitors and things in the ecosys­tem today

The ensu­ing com­ments and dis­cus­sion were pretty good — but last night Michael Sparks (of Kamaelia) posted a darned nice comment:

Are you aware that a com­plete mini-axon using gen­er­a­tors is tiny and the rest is opti­mi­sa­tions and extra stuff that you find use­ful in real world sys­tems? By tiny, I mean this small:
* http://www.kamaelia.org/MiniAxonFull

A mini-axon using processes would be equally light­weight (shorter prob­a­bly) and pretty awesome.

Also, it’s easy to con­fuse the two halves of Kamaelia. If you think of Kamaelia as just an actor-type imple­men­ta­tion, then it’s actu­ally more an actor-like imple­men­ta­tion, with STM & an inter­nal SOA sys­tem of just over 2000 lines (which is how big Axon actu­ally is, exclud­ing com­ments & docs), with 80,000 lines of examples…

How­ever, per­son­ally I view it as a mech­a­nism for build­ing com­po­nents which hap­pen to be best used in a con­cur­rent fash­ion. ie rather than view­ing it as “a mech­a­nism for using con­cur­rency”, I view it as “OK, assume we have con­cur­rency, how can we use this to assist in build­ing and main­tain­ing sys­tems”. Axon also gives you the tools for tak­ing these con­cur­rent sys­tems, and inter­fac­ing between con­cur­rent sys­tems and stan­dard code. (http://www.kamaelia.org/AxonHandle)

As a result, I view Axon as a library which pro­vides you with the tools wrap­ping up idioms use­ful for build­ing col­lec­tions of com­po­nents which be a framework.

Any­way, potato/potato, tomato/tomato — if you like, you like, if you don’t, you don’t.

I’d love to replace our exist­ing process based stuff btw with a mul­ti­pro­cess­ing based ver­sion though. If I was going to go down this route, I’d fol­low our mini axon tuto­r­ial to do so, largely becauseit’s essen­tially the start­ing point I took with the mul­ti­process stuff recently and it worked out pretty well.

Beyond this basic stuff though, I’ve noted that peo­ple gen­er­ally start talk­ing about co-ordination lan­guages and build­ing up pat­tern repos­i­to­ries. The inter­est­ing inter­sec­tion between these two which you get if you call things com­po­nents rather than actors is it becomes nat­ural to cre­ate com­po­nents called a chas­sis. These chas­sis often instan­ti­ate directly in con­crete usable form con­cepts that you’d nor­mally refer to as a pat­tern — Pipeline, Graphline, Carousel, Back­plane, Seq, TPipe, etc.

On a ran­dom note, you may want to check out MASCOT “Mod­u­lar Approach to Soft­ware Con­struc­tion, Oper­a­tion and Test”. I heard about it late last year, and it appears to have the same sort of archi­tec­ture as Kamaelia. Inter­est­ingly (to me) it makes the same key deci­sion — when you send a mes­sage out­side your com­po­nent, you don’t know who is going to receive it. This then enables (and requires) a higher level sys­tem for con­nect­ing com­po­nents together. The upshot is highly reusable com­po­nents. This doesn’t entirely sur­prise me — my ethos came from recog­nis­ing that asyn­chro­nous hard­ware sys­tems & net­work sys­tems look strik­ingly sim­i­lar… (cf http://www.slideshare.net/kamaelian/sociable-so…)

Any­way, ref­er­ence for MASCOT: http://async.org.uk/Hugo.Simpson/ — skip down to the end of the page for this PDF: http://async.org.uk/Hugo.Simpson/MASCOT-3.1-Man… I was really pleased to be pointed at MASCOT, largely because it showed a large num­ber of other domains where the same basic model has been used for well over 30 years… Just with non-existent expo­sure, and slightly dif­fer­ent metaphors. Though we, like it, also have mech­a­nisms for auto­mat­i­cally visu­al­is­ing sys­tems, with a 1:1 cor­re­spon­dence. Beyond that this also gives us a model that matches Edward Lee’s “The Prob­lem with threads” — we’d released run­ning code long before that paper was published :-)

Any­way, I’m glad that you’re look­ing at what we’ve done. If you use it, that’d be great, and I’ll hap­pily merge any­thing you’d like to have a life. (the only com­ment I’d make there is metaphors and acces­si­bil­ity count — this is surely the point of python? :-) If you don’t take what we use etc but it helps you solid­ify your thoughts to “No, I don’t want that, I want this”, then like­wise, I’m equally glad. If you do that, I’d love to know what you do try, since I like to merge best prac­tice con­cur­rency ideas into Axon :-)

I’d *REALLY* sug­gest look­ing at MASCOT though. Really made my Christ­mas last year when I was pointed at it.

We’re cur­rently hav­ing lots of fun using con­cur­rency, pri­mar­ily by allow­ing it to make our lives eas­ier, and for­get­table about :) It’d be nice to see some­thing sim­i­lar on top of mul­ti­pro­cess­ing (which we’ll do if you don’t, but it’d be great if you did — but I’d under­stand if your view was that you pre­fer a pure actor (sender knows receiver) model.

Orig­i­nally posted as a com­ment by Michael Sparks on jessenoller.com com­ments using Dis­qus.

I wanted to pull that com­ment out and show­case what Michael has to say, and in some way, respond. First, yes — I am look­ing at Kamaelia (from here on out, I’m going to call it “Kam” — I keep trans­pos­ing the ae). I actu­ally ran through the mini-axon tuto­r­ial, and when I have time, I’m try­ing to tease apart the inter­nals to bet­ter under­stand it.

Fun­da­men­tally, I agree with you (Michael) about the aspects of mak­ing con­cur­rency eas­ier (and safer). Right now, I think Kam is a pretty darned good start — for a frame­work *grin*.

When I made the com­ment I made, I didn’t think it would get the response it got. There has been a dog-pile of dis­cus­sions about con­cur­rency best practices/etc and in fact, there’s a dis­cus­sion still going on on the python-list about con­cur­rency stuff going on right now.

Per­son­ally, I only work on the con­cur­rency stuff part-part time — this includes my minor work on Python-Core. My day job is a test engi­neer — while I am build­ing highly con­cur­rent (and dis­trib­uted) tests and I use mul­ti­pro­cess­ing and thread­ing daily, it’s not my full time pur­suit. I am pas­sion­ate about it, and I am pas­sion­ate about improv­ing python as a lan­guage, and library — and if I can do it as a day job and open source it, or get com­pany time to do it, by golly I will.

When I said “I want to build an actor model” — I was not nec­es­sar­ily talk­ing about doing an imple­men­ta­tion for python-core. I’m a big believer in learning-through-implementation — so when I said “build x”, not only did I mean “build some­thing for the world” — I also meant “build some­thing for my own ben­e­fit” so I can deep-dive into the con­cepts, prob­lems etc.

This is why I am adverse to jump­ing in and sim­ply “using” a frame­work — not because I don’t think it does some­thing exceed­ingly well — trust me, I am an oppor­tunis­tic devel­oper — if I can find a library that does what I need *right now* — I’ll use it.

That being said — I am explor­ing Kamaelia, and yes hope­fully I can steal some time to actu­ally do an imple­men­ta­tion of the process-based stuff with mul­ti­pro­cess­ing. I want to explore every­thing in the ecosys­tem today — my dis­cus­sions with Adam Olsen around this stuff (and around python-safethread) and oth­ers has made me really want to explore solu­tions that help every­one, and take the best ideas and con­cepts and rolls it into some­thing wor­thy of Python core.

As I have said before — I believe there is room within Python as a lan­guage, and CPython as an imple­men­ta­tion of that lan­guage — for a vir­tual buf­fet of help­ful libraries for con­cur­rency and dis­trib­uted sys­tems. Right now, we have thread­ing, async* and mul­ti­pro­cess­ing. There is plenty of room to grow. Maybe one day I can steal time to grab more of the con­cepts from java.util.concurrent and pro­pose them via a PEP. Heck — maybe we can work as a group to pro­pose an actor/monitor imple­men­ta­tion for Python-Core.

So — per­son­ally, and by way of respond­ing to Michael in a more con­crete way: I’m, per­son­ally look­ing at any­thing I can to learn more about imple­men­ta­tions and strate­gies. If some­thing were to come out of it, and I felt strongly enough to pro­pose inclu­sion in core, I would write and post a PEP — and not run in blindly.

I’ve got more than enough stuff to work on in addi­tion to the day job and being a Dad, and yard work. Oh and the day job, which doesn’t involve nearly as much distributed-and-concurrent sys­tems in Python as I’d like :).

Heck — think­ing about it we’d need a good mes­sag­ing imple­men­ta­tion too. I’ll put that on the pile too.

Quick Rant (slightly off topic):

Also, can we stop talk­ing about the damned GIL? Yes, you need locks, No, you prob­a­bly don’t care about the GIL. Stop yam­mer­ing about how “bro­ken” CPython is because of it– CPython is an imple­men­ta­tion, not the final one and not the only one. If the GIL really gets you excited, either drop to a C mod­ule, use mul­ti­pro­cess­ing or some­thing else. The GIL is here to stay for some time — either pro­pose a PEP (and a Patch) that doesn’t break CPython or hush. Enough bike-shedding — dis­cus­sion is great, espe­cially when some­thing comes out of it, but con­stantly berating/lamenting things is just a bike shed. The shed is pur­ple, now move on. Purple!

Required Read­ing ( in addi­tion to Michael’s links):

• Kamaelia
• python-safethread
• Twisted
• Drama­tis
• Par­ley
• Mes­sage Pass­ing Con­c­cur­rency (Actor Model) in Python
• Toward Bet­ter Con­cur­rency
• The Pil­lars of Con­cur­rency

Recently, I made an offhand comment here about: I've actually started thinking about/sketching an actor model build on top of MP, using concepts from actors/monitors and things in the ecosystem today The ensuing comments and discussion were pretty good - but last night Michael Sparks (of Kamaelia) posted a darned nice comment: Are you aware that a ...