Since the news of PEP 3003 being released crossed the ‘web – I’ve seen more than a handful of assertions and statements made that seemed to grossly miss the point.
I figured I’d expound on my last post to further explain the why of the moratorium and some of the other personal reasons I stepped forward to help author it.
The one that really gets me is the assertion that the moratorium was some ego-driven “python is perfect” statement. That Guido/core dev think the language is “done” and that no new syntax or constructs are needed.
False. Categorically false. Quite the opposite is true. Most of us who are involved on python-dev and python-ideas and the various -sig lists are actually quite sure that some changes, whether they be syntactic sugar for existing things, or new fundamental constructs continue to be “needed” (“nice to have”) for Python The Language.
Still yet; many of us do think of new things to add – there is always a small wish list of “nice to have ponies” waiting in the wings to add to the language by a variety of people. So, no, no one asserts that “the language is done”. We all know, accept and look forward to the continuing evolution of the language over time.
However, language changes have side-effects. For example, context managers introduced two new keywords – “with” and “as” – you should take a moment to go back through the archives to see how many people were up at arms at those two new keywords being added. Apparently, a lot of developers use them as variable names.
To this day, I still download packages off of PyPI which throw warnings about those two keywords. Which is amusing, because they just crash under python 2.6. It was added to __future__ in 2.5 – but became syntax in 2.6.
I only mention this to illustrate the consequence of adding new things – does this mean adding new things is bad? No. Does it mean we have to be wary/aware of the ramifications of doing these things? Yes.
In the past 5 years, Python has gone through an accelerated evolution – decorators, context managers, etc. More new syntax, keywords and cool things have been added to the language than normally occurs in other languages. This has consequences.
The first consequence: Users. Users (developers) who want to support many version of python walk a tight rope. Ultimately, they have to write code for the lowest version (say, 2.3) and make sure their code doesn’t use anything (or any keywords) that will conflict with a newer version. That’s life – I think we, as developers, all accept that.
But the second one is more interesting. The second consequence is more of a recent development. For many years, CPython has been the implementation of Python. When you asked a person “hey, what python interpreter are you running” they typically say “the one from python.org” or “the one that came with my OS” – this is always CPython.
That’s changing. Take for example my surprise when I asked a friend “so what are you guys running” and his reply was Jython. He then lamented the fact that Jython was behind the curve on syntax changes in the latest version of Python, etc, etc.
More and more we see the fact that Python The Language is more than Python The C Interpreter. We have PyPy, IronPython, Jython, Unladen-Swallow (really, a branch of Cpython) and others. Sure, most of the community is still running good-old CPython (“old reliable”) but we’re at a point where more and more people are running “those other guys”.
We’re also at a point where some of the “cool” interpreter changes, such as JIT, free threading, etc are coming from those other interpreters. We face a possible future wherein CPython is just the reference implementation of the language, but most of the world runs “one of the others”. This is not a bad thing! It’s not that the CPython interpreter is dead – it’s simply that the other interpreters have a greater focus on “cool interpreter stuff” which CPython has a focus on “stable”.
Adding syntax, keywords, and new constructs to the CPython implementation negatively impacts those implementations. For the longest time, Jython was simply aiming at Python 2.5 compatibility – I’m sure us adding a pile of new things in every version doesn’t make their lives any easier. Sure, we can feel free to keep adding things in there, but without all of the implementations being at some parity with one another, every time we do so, and ship it, we set them back and possibly confuse users.
Now consider the fact that in the past 5 years, we’ve done a pile of releases, adding a bevy of features to the language, the latest and greatest being Python 2.6. Stop and realize for the moment most of the world is probably still running Python 2.4 or 2.5. Rumor has it, some people are still running Python 2.3. The fact is many operating system vendors, and normal people using python day to day, aren’t getting exposure to the latest and greatest things we’ve added. This means all the cool stuff we’ve added hasn’t seen the light of day – sure, some of it is “pretty minor” – but when you’re adding things to a language, you generally want to see them get used heavily so you know you “hit the mark” and don’t need to go and fix the thing you just added.
It’s important that we don’t (and shouldn’t) just chuck “really cool ideas” in there and then never look back – sure, some of them might be constructs from other languages which have been proven in that language but Python isn’t that language – Python is Python, and we have to be very self-aware to the consequence to Python a change might incur. It takes time, and adoption (see above) of those features to see how they stand up to real-world usage. Sometimes, we simply get an interface/API wrong, or the behavior isn’t as clear as we would like. It happens, but just adding thing on top of thing without taking a moment to consider how it worked out is bad.
Even with an extensive review/PEP process, even with some of the smartest people I have ever had the pleasure of talking to, mistakes happen. The only thing which allows us to see if a mistake has been made is time.
And now we get to Python 3.
Python 3 is an interesting animal – it breaks backwards compatibility in the name of forward progress – some of the changes made to the language required that break to occur. Now, throw a new, backwards incompatible version of Python onto the pile of things users, packagers, developers, OS Vendors, etc all have to deal with. Remember of course the point above – many people are still running/shipping python2.4/2.5. How are we going to get any runtime on the syntax we added in the 3.x line?
Python 3 is the ultimate syntax change – not to mention, to most users it’s several years off for them, given they’re reliant on third-party packages that haven’t ported, their OS doesn’t ship it, etc. For all the problems a “normal” python release suffers with adoption – Python 3 suffers from them in spades.
So now, we have Python 2.6, and Python 3 – both in the adoption pipeline. Python 2.6 is an easier pill to swallow, Python 3 is a bit more painful as it requires much more intervention to port to it, and maintain a compatible code base.
Let’s also account for the fact that, as of this writing, Python 2.7 (scheduled for next year) is intended to be the End of Life release of the Python 2.x syntax – Python 3 being the next evolutionary step.
Python 2.7 is going to be a good release – there’s going to probably be some Python 3 stuff pulled back into it to help porting to Python 3, there will be some “new” things – minor features back ported from Python 3 as well as more migration helpers to assist in the 2 to 3 jump.
Notice in that last part – almost everything going into 2.7 is a feature/change of Python 3 we’re pulling back to help developers and users alike. Big time features and changes are not typically going into 2.7. In fact, there’s a growing number of changes which are not being pulled back into 2.7, Antoine’s recent Global Interpreter Lock changes (also here) being amongst them.
I haven’t even mentioned the fact that every change to the code base has to go into at least 2 branches, and in many cases 4 individual branches (py26-main, trunk, py3k, py31-maint). Many changes don’t have a clean-merge, given the change of syntax each time you move something between trunk and 3k, you have a non zero chance of having to make decent changes to the patch you’re merging. This causes non-zero pain for core maintainers/developers.
So, here we are – a 2.6 release still being adopted by users, OS vendors, etc. A new 2.x release coming down the pike within the next year and a backwards incompatible release (Python 3) also in the pipeline.
At this point, it only makes sense to put a moratorium on the language – we have to artificially put on the brakes for the syntax and core language to make sure the next few releases (2.7, 3.2) don’t increase the burden for adopters. We have to let the other implementations catch up – we have to focus on things like stdlib improvements/fixes/cleanup, we can focus on interpreter improvements, tests, cleaning out the bug list, etc.
The next few releases of Python won’t introduce new language and syntax – they will be releases focused on bug fixes, cleanup – things that make everyone’s life better. This time will serve as a reflection period for the syntax we have, and allow us to help push along Python 3 adoption, maybe helping bigger projects port, improve porting tools and guides, etc.
I’d also like to add – this moratorium doesn’t apply to the standard library! We can still fix things like the mimetypes module, we can improve warts and fix things which are broken. We can keep cleaning the standard library up – we can polish all of those batteries which help make Python, well, Python. Ultimately Python isn’t just it’s syntax – many people adopt it because of the standard library (over 216 modules, at last count) – and that deserve as much, maybe more, attention than adding cool syntax to the core of the language.
So, don’t be afraid of it – don’t think that Python is evolutionarily dead – it’s not. We’re taking a stability and adoption break, a breather. We’re doing this to help users and developers, not to just be able to say “no” to every random idea sent to python-ideas, and not because we’re done.
We’re doing it for you – the developer who needs stability, the OS vendor still working on a new release, the language/interpreter implementation group still catching up to 2.6/2.7 and to all the people who would like to adopt Python 3 in the next decade.
Python’s not dead people. It’s just outside sitting in the grass thinking about what it’s going to do next.