jessenoller.com

I chose the title for today’s foray into Duck/Latent typ­ing from the (in)famous thought exper­i­ment Schrödinger’s Cat wherein a cat is placed in a box with a radioac­tive iso­tope, and one can not observe the state (type) of the cat with­out irrev­o­ca­bly alter­ing said state — I felt it was a duti­fully ironic view of the lack of “con­trac­tual enforce­ment” of types except dur­ing run­time within Python.

My “type(Duck)’ing: On Duck vs. Sta­tic Typ­ing” post received some minor atten­tion, but as with all things of an obses­sive nature, I wanted to fol­lowup and delve more into the Python(ic) aspects of typ­ing as I under­stand them⁴⁴.

Henry Story — the author of the arti­cle which trig­gered my last post made sev­eral com­ments — the last of which stuck out the most for me:

…snip “But in fact these lan­guages don’t do this. they just look to see that there is a method that is named “Quack”. If there is it gets called as if it was clear that was it means was the sound. But why could a dog not have a “quack” method that meant to kill a cat? There would be no inter­face bro­ken here. Just a clash of words, and we have those a lot in the eng­lish lan­guage. Things such as “bank” (the place you deposit your money at) and “bank” of a river… We dis­am­biguate eng­lish because we take con­text into account, just as a pro­gram­mer makes sure that when he gives objects to a method that will duck type on “quack” he makes sure never to give the method objects where “quack” means some­thing else. Notice how this has trou­ble scal­ing though.

Any­way, I would be glad to be shown to be wrong. I just looked up a book on Ruby, and that con­firmed my think­ing. Do you have a pointer to a piece of code that could resolve the issue?”

As oth­ers pointed out seman­tics and con­text mat­ter when pro­gram­ming. There is sim­ply no way around con­text being rel­e­vant — much like lan­guage⁴⁴ con­text and intent give rel­e­vance and inflec­tion. Yes — a word can mean many dif­fer­ent things — hell, some peo­ple make stuff up.

Again ignor­ing the fact that Henry is try­ing to address the seman­tic web with the con­cept of a URI/heirarchy based “typ­ing enforce­ment scheme” I fig­ured I would delve into, well, Python’s fac­ul­ties in this area (hint: name­spaces and usage give con­text, and ergo you can derive type). But first a digres­sion into namespaces.

I could speak to a per­son and say “How was your Day”. In their crazy moon-speak, maybe they rede­fined “Day” to mean “Under­pants”. So, I just asked them how their under­pants are doing — while amus­ing, in most places I would get slapped or fired. This is how inter­net flame­wars break out — some­one has a bro­ken under­stand­ing of the word/world — or they don’t have con­text and/or rel­e­vance. Who is in fun­da­men­tal vio­la­tion of the con­tract of lan­guage? Me or the per­son who rede­fined the inter­face (word)?

Wel­come to Sta­tic Typ­ing — because in the sta­tic world we do not trust that other pro­gram­mers, peo­ple, scripts, appli­ca­tions or for that mat­ter — our­selves — under­stand intent or con­text⁴⁴.

Duck typ­ing allows for fudge -⁴⁴ sure, crazy McCraz­er­ton thinks that Day means pants, Duck means Dog and all sorts of wrong things — but what if his def­i­n­i­tions weren’t explic­itly “wrong”? What if they were “close enough”.

By “close enough” I mean that given the con­text of what I was say­ing — he could fig­ure out I wasn’t address­ing his pri­vate parts — but rather I was inquir­ing as to his cur­rent state. Sure! His word has all sorts of weight behind it. When he says “Day” or “Dog” — the word can bring all sorts of inter­est­ing inter­faces with it — but who cares what he tacked onto the damned word: to me, if it’s “Day” — then I can at least have an idea of what the hell is going on.

If it walks like a duck: Quacks like a duck, has wings, then it could be a Dog in a Duck cos­tume. Who cares?

In our case — the Type in the Cat, the Name­space is the Box, and the iso­tope in the box is the inferred con­text of the type when we “open the box”.

Code time! Yay!

At Python’s very core is the con­cept of name­space. That’s an impor­tant thing as every­thing within Python is a name­space and has scope and “pri­vacy⁴⁴″ inher­ent in that design.

For instance — when you per­form an import, the pack­age loader walks into the tar­get package’s name­space and works on find­ing the tar­get — for example:

1

import animal.species.dog.quack

This means that the inter­preter walks into ani­mal, then species and dog and snags the quack.py mod­ule con­tained there. It then com­piles the asso­ci­ated python code to byte-code and now has some­thing within the global scope point­ing to the imported mod­ule. For exam­ple given this mod­ule “struc­ture”:

woot:~/tmp/ jesse$ mkdir -p animal/species/dog
woot:~/tmp/ jesse$ cd animal/
woot:~/tmp/animal jesse$ touch __init__.py
woot:~/tmp/animal jesse$ cd species/
woot:~/tmp/animal/species jesse$ touch __init__.py
woot:~/tmp/animal/species jesse$ cd dog/
woot:~/tmp/animal/species/dog jesse$ touch __init__.py
woot:~/tmp/animal/species/dog jesse$ touch quack.py
woot:~/tmp/animal/species/dog jesse$ vi quack.py

And within quack.py, all I put is:

1
2

def quack():
	return "Woof"

I pop into the inter­preter and do this:

woot:~/tmp/duck jesse$ python
Python 2.5 (r25:51918, Sep 19 2006, 08:49:13)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import animal.species.dog.quack
>>> globals()
{'__builtins__': , '__name__': '__main__', '__doc__': None, 'animal': }
>>> quack
Traceback (most recent call last):
File "", line 1, in
NameError: name 'quack' is not defined
>>> animal.species.dog.quack

>>> animal.species.dog.quack()
Traceback (most recent call last):
File "", line 1, in
TypeError: 'module' object is not callable
>>> animal.species.dog.quack.quack()
'Woof'
>>>

As you can see: I told the inter­preter to import⁴⁴ the full “path”/name of the mod­ule from dog I wanted, quack. Of course you can do things like this:

>>> from animal.species.dog.quack import quack
>>> quack()
'Woof'
>>>

But given that import state­ments are in the first few lines of your script⁴⁴, or imme­di­ately near the code which is ref­er­enced, the scope and intent of quack within this session/application is clear. Most of the time peo­ple will pull in the top-level pack­age, or the pack­age right above the foo.py they wish to ref­er­ence, in my case — it would be dog. I would then ref­er­ence dog.quack.quack() when I needed to call the quack method in quack.py.

Now, import tricks, rel­a­tive imports, etc are all inter­est­ing — and by far they give some of the best con­text that you could want about the ini­tial intent of a method, class or object — but import renam­ing is another fan­tas­tic thing:

>>> from animal.species.dog.quack import quack as dogQuack

I’ve just saved myself some heart burn. I don’t like call­ing foo.bar.baz.yourmom — I like call­ing func­tion() or method() with­out the import/namespace fore­play. But I have the decency to rename the import to dogQuack so that when I ref­er­ence it, I am always reminded about whose quack I’m quack­ing⁴⁴.

You should read the import sec­tion in PEP8. No, seriously.

So now that we’ve gone and ram­bled on about the con­text of where a Class/Object or Method might come from, or be inferred we can move on to real things, like types!

Guido van Rossum: In Python, you have an argu­ment passed to a method. You don’t know what your argu­ment is. You’re assum­ing that it sup­ports the read­line method, so you call read­line. Now, it could be that the object doesn’t sup­port the read­line method.

Bill Ven­ners: And then I’ll get an exception.

Guido van Rossum: You’ll get an excep­tion, which is prob­a­bly OK. If this is a main­line piece of code and some­thing could pos­si­bly be passed to you that doesn’t have a read­line method, you’ll dis­cover that early on dur­ing test­ing. Just as much as in a typed lan­guage when you have an inter­face and you know you’re get­ting some­thing that has the right inter­face but doesn’t imple­ment the right thing, or it throws an unex­pected excep­tion. You’ll hope­fully find that dur­ing testing.

In addi­tion in Python, because there aren’t fixed pro­to­cols, some­thing else can be passed that also sup­ports read­line and doesn’t hap­pen to be a file, but does exactly what you need. All you need at that point is some­thing that returns lines.

This quote hints at one of the key things about Python: Excep­tions should not pass silently. More on that later.

Python has some basic object types — you know, your run of the mill int(), float(), str(). One of the first things you learn in python is what each one means. For the sake of this dis­cus­sion, we’ll focus on dict() (dic­tio­nary) — Python’s hash/map type.

If we have an object — let’s say baz and we declare it to be a dict — We can politely inquire as to it’s type:

>>> baz = {}
>>> type(baz)

>>>

The builtin type() method returns an object rep­re­sent­ing the type of the object you passed to it. Not the object itself, also not the string rep­re­sen­ta­tion of the object’s type. If we wanted to be banal about this, we could do this:

>>> baz = {} # assume someone passed us baz
>>> foo = {} # we make foo into an empty dict...
>>> type(baz) == type(foo)
True
>>>

And yes, this sort of check works if one actu­ally has some­thing in it:

>>> foo = {'hi':'mom'}
>>> type(baz) == type(foo) # yes I know, isinstance() - I'll get to that.
True
>>>

But if we com­pare the two objects directly (instead of the type objects returned by type()) we see that they are dif­fer­ent:

>>> foo == baz
False
>>>

So — back to the topic of ducks — what makes our dict quack()? The meth­ods it sup­ports. Again — if it looks like a dict, smells like a dict — then it works like a dict it is a dict. Right⁴⁴?

Let’s look at the meth­ods a dict has, shall we?

>>> foo = {}
>>> dir(foo)
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']
>>>

Now, I want to make my own object — say, MyDuck⁴⁴:

>>> class MyDuck(dict):
... def __init__(self):
... pass
...
>>> dir(MyDuck)
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__str__', '__weakref__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']

Behold: MyDuck sup­ports all of the same attrib­utes of dict() — and this brings us full circle.

If I cre­ate a Quack object in animal.species.duck or animal.species.duck.quack — then you can be sure that what you are look­ing at is a duck’s Quack — you have the con­text — you can look at the attrib­utes of the Quack() to ensure that is in fact, Quack.quack-able.

If some genius makes a animal.species.dog Quack object — the if you want the duck’s quack — why are look­ing at a dog to sup­ply your much needed quack? Maybe you want the spe­cial dog quack that uses the ducks’s quack:

>>> class MyDog(MyDuck):
... def __init__(self):
... pass
...
>>> dir(MyDog)
['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__dict__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__str__', '__weakref__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']
>>>

We now have MyDog, which has all of the attrib­utes of MyDuck, and all of the attrib­utes of a Dict object. This is fan­tas­tic as we can remove/override/extend things from MyDuck, or Dict inside of MyDog and move on with our lives.

What if the meth­ods clash you say? What if both MyDuck and MyDog both have the .quack() inter­face, but in the case of MyDog, a woof is returned instead of a quack? The answer: Why are you using Dog’s quack when you know it returns some­thing you don’t want, i.e: a ducks quack?

But what if you don’t know that MyDog’s quack is dif­fer­ent than MyDuck’s⁴⁴? Easy — when you fool­ishly pass MyDog() into your script’s make­Quack­ing­Noises() func­tion, and you look for the returned ‘quack’ value — you’re going to get an excep­tion — and excep­tions should not pass silently in the night.

But you say: A com­piler would have told me this ahead of time! To which my reply would be: A com­piler pro­tects you from the most base ver­sion of human error — the typo. Whether the typo is inten­tional (you changed the method with­out watch­ing what some­one was pass­ing you) or unin­ten­tional (you meant Dog but typed Duck), the com­piler can’t pro­tect you from more seri­ous “pilot errors” (you pass in some­thing that works, sort of). Ergo — Testing!

I know Python is not per­fect⁴⁴ and as I have said before, there is some­thing nice about sta­tic con­trac­tual enforce­ment — that’s why I like python 3000’s ABCs (or even the Roles implementation).

Also, yes — method sig­na­ture enforce­ment can get con­fus­ing some­times — I had to chase down a bug I checked in yes­ter­day that was the direct result of me play­ing fast and loose with the rules around method argu­ments⁴⁴.

Note that I’d also like to point out Collin Winter’s type­check mod­ule you can also add in.

Going back to the orig­i­nal points how­ever — Duck typ­ing is flex­i­ble, pow­er­ful and yes, like all things involv­ing those — it can be dan­ger­ous but name­space pro­vide your con­text, and the inqui­si­tion of objects at run­time is easy to do, you can enforce the required con­trac­tual oblig­a­tions as much as you need or want to.

Python’s power comes from it’s flex­i­bil­ity — and believe it or not, duck typ­ing. With­out it, we would lack some of the grace and power that comes with the lan­guage. A dynamic lan­guage should be exactly that: Dynamic. Sta­tic type enforcement/interfaces has ben­e­fits but it only pro­tects you against sim­ple bugs that a com­piler can be smart enough to test.

For more thoughts/information on this — also see isin­stance() con­sid­ered harm­ful. As well as the links in my pre­vi­ous post. 4

1. Dis­claimer: I am not always cor­rect, I too am con­stantly learn­ing and adapt­ing. What may seem clear now may change as tech­nol­ogy changes or as I learn more444
2. If you want more on Lan­guage vs. Pro­gram­ming, speak to “r0ml” Lefkowitz444
3. I have a feel­ing this is where the ver­bosity cri­tique of Java comes into play444
4. with the imple­men­ta­tion of ABCs, you can make the fudge more solid444
5. see __methodName444
6. see also: Import­ing Python mod­ules444
7. see PEP 8444
8. duck anal­ogy: offi­cially beaten to death444
9. see Library Ref­er­ence for Map­ping Types444
10. yes, I like Boun­cy­Case classes and meth­ods, leave me alone.444
11. why are you coding?444
12. see: python pit­falls, python warts, python gotchas444
13. See: Method sig­na­ture check­ing dec­o­ra­tors444

4

I chose the title for today’s foray into Duck/Latent typ­ing from the (in)famous thought exper­i­ment Schrödinger’s Cat wherein a cat is placed in a box with a radioac­tive iso­tope, and...