Skip to content

Miscellanea – Python Sprints, Nasuni, etc.

I’ve obviously been quiet here on my personal blog – as everyone who reads regularly knows I’m neck-deep in a pretty exciting startup call Nasuni as well as doing other projects, like the PSF Sponsored sprints thing. That combined with twitter means my time for other additional long-form content is minimal. So here’s a small roundup of interesting things:

Nasuni

Yup, still running Python and Django! We’re actually pretty proud to be a sponsor for DjangoCon 2010 coming up in September – I’ll be attending, so I hope to see all the familiar Django faces I know, and meet some new ones.

I’ve been blogging semi-regularly for the Nasuni blog itself – my posts are focused on product-things more than anything else. Here’s a small list of posts which I’ve done:

  • The Road to Release – Feature Previews – this is actually my latest one, and the first in a series where I’ll be showing off some of the new features we’re adding in the latest release.
  • Looking at OpenStack, a Rackspace and NASA initiative – For those of you who don’t know, Rackspace and NASA announced OpenStack – the awesome part? It’s all python – I had the swift component (which powers Rackspace’s cloudfiles system) of OpenStack running pretty quickly. I’d recommend snagging the code from launchpad and taking a look. Swift (the storage component) uses eventlet – and Nova (the compute part) uses Tornado and Twisted.
  • Storage Switzerland Test Drives the Filer – This is a response to an article written about the product – I actually used it to preview some of the work going into the next release of the Filer.
  • Thanks to Django – This piece goes into some detail about our use of Django, it’s one of our ways of saying thanks. I still need to rework it so we can send it over for the Django Success Stories page.
  • Thanks to the Supporting Cast – This is an earlier thank you post – but to the other people who have helped out a ton, including Greg Newman, Lincoln Loop, and Revsys.
  • The Donut Solution – This was a fun one, mainly to show that yes – we’re listening hard to customer feedback, and we’re improving/iterating quickly. Also, I get to show off UI improvements.
  • Finally – The Nasuni Blog team – this is the rosetta stone for the authors of the blog, describing who we are. I didn’t write this piece, but it’s good reading to figure out who is who.

If you’re interested in Nasuni – or cloud storage in general – I’d encourage you to sign up for the RSS feed. We’re trying to keep the information useful outside of “just us” (despite my urge and predilection to churning out completely product-related posts) – and if you ever have feedback, drop us a line.

PSF Sponsored Sprints

The project continues on – we’ve funded two sprints so far, and have several more coming down the pike. We’re always in need of volunteers to help us do things like the manuals and site maintenance/content authoring. Here’s some highlights:

  • The call for applications is open – The call for applications is open – and now I suspect we won’t be closing it. Originally, I thought we’d have to do things in waves of apply-approve. As time has progressed, I no longer think this is the case.
  • Montreal Python Packaging sprint wrap up – the wrap up report for our first sprint!
  • Europython core sprint report - another wrap up report for the core sprint we provided funds to.
  • Just added the locations page – we now have people/companies offering up space for sprinters! Check it out!
  • Finally - Sprints at PyOhio – PyOhio is going on this weekend, if you’re in the area you should really go check it out! Catherine has gone above and beyond with the entire “become a contributor” effort going on.

Please! If you’re thinking about holding a sprint - send us an application! Heck, even if we’re not sponsoring it, we’ll help promote you via the blog, and the sprint calendar we have up. A little fact? The sprints we’ve funded so far, and that are on deck for funding are all outside of the US, which is both awesome, and surprising!

PSF Board

Some of you probably know that I’m currently on the board of directors for the PSF – things progress well here, but I mainly wanted to call out the excellent blog Doug Hellmann has been authoring for PSF news. You should really be watching that because yes – we do do things, and hopefully over the next year, we’ll be doing more awesome things.

I’ve actually got a bigger post in the works for what I think the ultimate mission of the PSF is/should be as well as “how do you get money from us” as well. Must find the time!

Posted 30 July 2010 † jesse § Programming § Python § Storage § Technology ‡ View Comments °

Google Testing Blog: “There, but for the grace of testing, go I”

The Google Testing Blog has a good post up right now by James Whittaker called “There, but for the grace of testing, go I” – it’s a good read, and a pertinent one for any of you/us who feel strongly about quality.

Even though I’ve spent more time then not on “the other side” of the table (Developer, noun – “focus on making software (ergo, bugs)”) I find that James’ words ring pretty loudly for me still, especially his part on risk analysis:

I am thankful that the vast majority of bugs that affect entire user populations are generally nuisance-class issues. These are typically bugs concerning awkward UI elements or the occasional misfiring of some feature or another where workarounds and alternatives will suffice until a minor update can be made. Serious bugs tend to have a more localized effect. True recall class bugs, serious failures that affect large populations of users, are far less common. Testers can take advantage of the fact that not all bugs are equally damaging and prioritize their effort to find bugs in the order of their seriousness. The futility of finding every bug can be replaced by an investigation based on risk.

I’d recommend James’ post amongst the others there on that blog – it reminded me of an old rant of mine “The cost of (not) testing software“. Anyone in the business of making something is also in the business of making bugs. It’s important for us to keep that in mind when we deal with our day to day job – and when we think about our customers. It’s also important for us to keep that in mind when criticizing or dragging any person or company or code through the muck.

Posted 17 July 2010 † jesse § Programming § Python § Testing ‡ View Comments °

PEP 3148 Accepted: “futures – execute computations asynchronously”

Back in May, Guido assigned me “single-use” BDFL powers when deciding on the state of PEP 3148. I had been working on and off with Brian since his original proposal to the stdlib-sig mailing list.

Tonight, I “officially accepted” the PEP Brian’s put a lot of work into. You can check out the PEP itself, or play with the code over here. The credit goes to Brian, and the small army of people who gave him feedback including Jeffery Yasskin who spent a fair amount of time iterating and expounding on things.

Below, is a snippet from one of my original emails to python-dev on my reasoning for supporting the library, and also adding in the “concurrent” namespace.

Baloney. A young library providing some syntactic sugar which uses
primitives in the standard library to implement a common pattern is
fine for a PEP. We’ve hashed this out pretty heavily on the stdlib-sig
list prior to bringing it here. By the same argument, we should shunt
all of the recent unittest changes and improvements into space, since
golly, other people did it, why should we.

This is something relatively simple, which I would gladly add in an
instant to the multiprocessing package – but Brian’s one-upped me in
that regard and is providing something which works with both threads
and processes handily. Take a look at multiprocessing.Pool for example
- all that is some sugar on top of the primitives, but it’s good
sugar, and is used by a fair number of people.

Let me also state – “my” vision of where futures would live would be
in a concurrent package – for example:

from concurrent import futures

The reason *why* is that I would like to also move the abstractions I
have in multiprocessing *out* of that module, make them work with both
threads and processes (if it makes sense) and reduce the
multiprocessing module to the base primitive Process object. A
concurrent package which implements common patterns built on top of
the primitives we support is an objectively Good Thing.

For example, how many of us have sat down and implemented a thread
pool on top of threading, I would hazard to say that most of us who
use threading have done this, and probably more than once. It stands
to reason that this is a common enough pattern to include in the
standard library.

In any case; consider me a strong +1 to adding it.

With that done, I am back to being only the benevolent dictator of my lawn, which is mostly dead.

Also, for added fun – check out the PyCon AU talk Brian did!

Posted 11 July 2010 † jesse § Programming § Python ‡ View Comments °

Just in cased you missed it: Numpy / Scipy Python 3 support.

I’m not an active user myself (I’ve never had the need) but Numpy and Scipy are two of <b>the</b> key modules people cite as their reason not to use or port (or support) Python 3 (due to lack of porting). According to this announcement to Numpy-discuss, this will very quickly no longer be the case. Numpy has been ported to Python 3, and Scipy is following quickly (in progress). These are cornerstone libraries for the community – more will follow.

This is great news; most of all, I’m looking forward to picking through the collective knowledge of their porting effort to see if there information which can be given/provided to other porting efforts. You can see their Python3 porting notes here.

And a quick plug – the PSF Sprints project will help fund sprints focused on porting libraries to Python 3 – go here.

Posted 10 July 2010 † jesse § Programming § Python ‡ View Comments °

Call for Applications now open for PSF Sponsored Sprints

I’ve put up the first official Call for Applications over on the Python Sprints blog – go check it out!

Posted 08 July 2010 † jesse § Programming § Python ‡ View Comments °

Announcing: Python Sprint Sponsorship

Otherwise known as “from psf import sprints”.

Awhile ago, I started wondering if there was a way that the Python Software Foundation (the PSF) could reach out, and help encourage python development a little bit more than it currently does (it actually does a lot of stuff). I then spent a fair amount of time thinking about core contribution, and ways of getting people not just to become contributors to say, python-core, but also to the “ecosystem” of big-P-Python. It’s part of my job now that I volunteered to be on the PSF Board.

Then, a few weeks ago, I asked the question – “Why aren’t you contributing (to python)?” – that post alone got my little blog here about 30,000 hits in around 48 hours. Pretty impressive for navel gazing (not even controversial navel gazing, at that). In the days following that post, I’ve had a lot of public and private discussions with people about various ways to help improve things, what could work, what wouldn’t work, etc.

One thing has been stuck in my head – sprints – sprints are places where a bunch of programmers and hackers of all skill levels, sit in a room and work on a particular code domain, it could be a website, it could be python bugs, it could be a module, library, or anything, really. The point is getting a bunch of people together in a room, feed them, water them and enable the hacking.

And finally – there’s Python 3.

Python 3 is a bit of an interesting animal – and while it is the future of the language, and a much brighter one at that, the rate at which people are porting to Python 3 is a little slower than many of us would like. Porting libraries to Python 3 can be relatively simple, but it can also be boring, tedious and require mentorship and guidance.

Given these itches of mine – I’ve decided to try to find (or at least, start the formulation) of a cure. A few weeks ago, I approached the rest of the Python Software Foundation board, and presented them with a “sponsored sprint” proposal. The short version being that the PSF would spin up a committee which would accept and evaluate sprint proposals from the world as a whole, and give each sprint a relatively set amount of money.

The sprints themselves are relatively focused – Python-core, the Python website, docs, bugs – and finally, porting Python 2 libraries to Python 3.

So – all of that said, I’m officially publicly announcing the new program. I’m still working on initial committee membership, and putting together things like the mailing list, but I will soon be announcing a call for applications. Below, you will find the full proposal – please feel free to post comments, or send me private email (jnoller at python dot org) about questions or concerns. If you want to help – please please contact me – I can not possibly make this a success on my own.

“from python import sprint”

This proposal outlines a global “focused sprint” initiative for the Python community (including Python core). The initiative outlined in this proposal is designed to help raise both the profile of the PSF, while also encouraging community, Python 3 adoption, and possible python-core contributions. This proposal does not identify any sort of return on investment or profit for the foundation; nor does it attempt to identify metrics of success. The success of this proposal is determined by the social “buzz”, patches, new contributors, etc. We hope that this will naturally raise the awareness of the PSF, as well as bringing in an overall more diverse contributor group.

This proposal suggests that the PSF-Board approves the following:

  • A small monthly fund, to a maximum of 400 USD per month (to a project maximum of 5000 USD), for “sponsored” sprints, this fund will be allocated by the project committee.
  • Creation of a “Sprint Committee” – the purpose of which is the allocation of the project funds to sprint coordinators, reviewing and approval of sprint applications, and internal and external communications. This committee is part of, and acts on the behalf of, the Python Software Foundation.

Focus of the sprints are:

  • Python Core bug triage and patch submission (on-boarding new contributors)
  • Python Core documentation (including process documentation) improvements
  • Porting libraries/applications to Python 3
  • Python website/wiki content improvements
  • PyPI packaging hosting site improvements
  • Contribution to other “core” projects, such as packaging related issues.

Committee Outline:

  • The “Sprint Committee” will be formed of one or more PSF-Board members, PSF Members, and possibly external volunteers.
  • A blog and/or website will be started by the committee for the external communication and “showcasing” of sprints which have occurred.
  • The committee will be responsible for the coordination and review and approval of sprint proposals.
  • The committee’s head/lead will be responsible for monthly reports to the PSF-Board.
  • A “Sprint Bundle” including the following items will be created, by the committee and other volunteers:
    • “Sprint Guide“, meant to serve as a how-to for coaching sprints
    • “Beginners Guide to Python Core work“, meant to act as handout material for sprint participants
    • “Beginners Guide to Porting to Python 3“, meant to act as handout material for sprint participants
    • “Beginners Guide to the Python Website“, meant to act as handout material for helping out with the website, for sprint participants
    • A page on the Python Wiki will be dedicated for free form “lessons learned from previous sprints” so that participants may provide advice to future participants.
  • Note: The PSF-Members list will be solicited for volunteers and committee members to help with this initiative and generation of collateral. Additionally, members will be asked to be initial Sprint Leaders, or Sprint Coach points of contact/facilitator. In other words, someone to help with issues such as the python bug tracker, and other domains.
  • Note: As needed for legal reasons, the Committee may use an intermediary entity to donate funds to local user groups.

Sprint Details:

  • 1-2 Proposals per month are targeted for approval. If the fund is increased either by the PSF, or external donations, then the number of sprints may be increased keeping the target of ~200USD per sprint (e.g. More smaller ones).
  • The requirements of the proposal would be:
    • Location
    • Number of participants
    • What the focus and goals of the sprint are (e.g. core contribution; porting a project to Python 3, website content)
  • As part of being approved, the sprint leads/coaches agree that they will deliver a report (hopefully, with pictures!) of the sprint to the Sprint Committee, for distribution on the Sprint Team’s and/or the PSF’s blog and websites.
  • Expenses will be handled post-facto. Unless a sprint lead specifically requests funding in advance, they will be asked to submit receipts and they will be reimbursed in a timely manner (as soon as possible).
  • The planning, and execution are the sole responsibility of the sprint coordinator/proposer. The PSF, nor the committee, will execute or directly plan any sprints.

Notes:

  • Status of the initiative will be provided to the board at each PSF Board Meeting by Primary Coordinator.
  • PSF sponsors, or other external companies, will be allowed (possibly solicited) to donate into this fund. If funding is provided externally, that additional funding will be used, if interest and applications permit, to host more sprints per month rather than returning money to the PSF.
  • The scope of the solicited sprints – namely, Python 3 porting, core and website work – may be expanded to other projects or initiative upon approval from the PSF Board, and only by the PSF Board. If the scope is expanded, the membership will be notified accordingly. Note that this may require additional materials to be made available.
    • Such expansions may include a Django sprint, a Twisted sprint, etc.
  • We may be able to solicit (from the membership) the creation of a bare-bones dedicated website with a blog and news updates for this project.

 

Posted 20 May 2010 † jesse § Programming § Python ‡ View Comments °

Why aren’t you contributing (To Python)?

Over the past year or two, I’ve been in a pile of discussions surrounding attempting to increase the number of contributions to Python (as a project and ecosystem) – specifically around bug fixes/patch reviews, filing bugs, pointing out documentation issues, website content, etc. Many of these conversations seem relatively insular (people already involved talking about how they get work done, and how easy it is) or relatively hand-wavy (“let’s make it better! *crickets*”).

We all know that the fact is that there are plenty of areas for anyone (both programmer and not) to contribute to the Python project/ecosystem, this is actually true for most, if not all, open source, and open communities out there. While I am looking at this through the myopic lens of Python-core, the same discussion could be applied anywhere.

In the discussions I’ve had on mailing lists, and in private conversations with people about why they don’t contribute, I’ve found a set of recurring issues cited by people who could be contributors, roughly speaking these are:

  • Don’t know how (documentation issue)
  • Don’t know where (site layout, organization issue)
  • Don’t want to fight/argue with the “but this is the way we’ve always done it” people (perception issue)
  • For many issues, see checkout (svn), build (learn a “new” build system), run tests/write tests, generate patch, file bug, defend bug as a “high bar” (process/tool issue)
  • Don’t feel that they’ll be valued (perception issue)
  • Can’t invest the time to learn the tool chain(s) (make, ReST, regrtest, svnmerge/svndiff, sphinx, etc) (I got nothin here)

And so on.

What interests me more than the things I’ve listed about (which is a distillation of months of discussions) is what, if anything, prevents you, my reader(s) from contributing to the following areas:

  • Filing bug reports (including the stdlib and documentation)
  • Filing patches for bugs/improvements (including the stdlib and documentation)
  • Fixing issues/sending patches on website content
  • Proposing new website or documentation content
  • Contributing to website maintenance
  • Contributing to the Documentation

And anything else you can think of. What I’m asking for, is for you to give me thoughts, ideas and feedback on how we can lower the bar and reduce the friction of any of this so that you too can help out. Everything is fair game, even “I find the method of arguing for PEPs to cause lower intestinal irritation” (although, I hope to get more constructive comments than that).

Here’s the reason I’m asking/reaching out – I firmly, and totally believe that every single person who uses Python today can become a contributor to the project and ecosystem beyond that of a user. While it is true that philosophically, everyone who uses Python is contributing, I would like to see if we can make it so that even the most “basic” user feels that they can file a bug, submit a patch, fix the documentation, fix/add website content, etc.

Much like I think anyone can, and should be empowered to contribute back to the “Python Project” – I believe the culture of “everyone contributes” is an important one. Citing the wonderful keynote from PyCon 2010 by Antonio Rodriguez (video here), I maintain this is should be a goal for everyone, not just companies – but open source projects as well.

There are lots of things we can change/fix/improve – but before we can do that, we have to be able to identify the “bugs” in what we have now. Do we need a giant flowchart with flashing text (I joke!) or do we need simplified “how do I…” text to the website? Everything’s fair game (except “just put it on github”).

So, do you have any thoughts? If you don’t feel comfortable airing them publicly, you can email comments to jnoller at (gmail.com|python.org) as you see fit. I promise not to share any identifiable information, but I reserve the right to share the feedback you provide, albeit anonymously. I also promise to ask meaningful questions so I can squirrel out more detail if needed.

Posted 22 April 2010 † jesse § Programming § Python ‡ View Comments °

PyCon 2010: My Nasuni lightning talk – Saying thanks

One of the posts I’ve been meaning to do is this one – a more extended version outlining what I spoke about in my five minute lightning talk I did at PyCon 2010 (more general thoughts on the conference later). Of course, meaning to post it – and actually getting a chance to are two different things.

So – my goal of the lightning talk was to say thank you to the python community, not only from me, but from the point of view of the company I work for (Nasuni) which recently launched. Thanks can be given back to the community in a variety of ways – money, code, sponsorship, programs, etc.

Sometimes, a simple Thank You can be a good start, and that was my goal. Unfortunately, I also completely deleted my slides (intentionally, what was I thinking?) after my talk was done.

What is Nasuni? We’re pretty small – obviously much smaller when starting (5 core engineers). The goal was to build a simple to manage, secure, reliable cloud-storage backed virtual storage appliance. Everyone helped code and design the product. We had to build it fast – when you’re a startup on VC money, you’ve got to be fast – you have finite time. Limited time, money and people.

From a technology standpoint – it simply made sense to put the core data path in C and C++ – you need speed, security, and the compactness C offers you when you’re talking speed. Not to mention, this is a device which has a finite amount of space, so keeping it relatively compact is a Good Thing. So the brain and central nervous system are in C – but we’re lacking something:

The rest of the person. D’oh. You know, arms and legs and stuff.

Let’s look at all of the other stuff we needed:

  • tests
  • tools/glue scripts
  • prototypes
  • user interface(s)
  • other system daemons
  • infrastructure
  • web site
  • deployment tools

And that’s just the stuff I could think of off the top of my head right now. So, we needed to compliment our super strong core with something pretty flexible, something that could also easily mesh with the core system as well.

When picking the technology; we needed speed of development, a broad ecosystem to build from, and it needed to ramp up and down. It had to be flexible enough that anyone involved in the project could become immediately effective with it.

Enter Python.

“Duct tape is like the force. It has a light side, a dark side, and it holds the universe together.”

Python Ramps “up”, and it ramps “down” – it’s good for “hard” programmers, and people just trying to get things done. It’s simple enough that anyone who has scripted in the past can get something done, and quickly. My motto has slowly become “there is no such thing as bad code – there’s just code that works, and code that doesn’t”. Code that comes out of someone who is just learning, and gets the job done is just as valuable as code that is well thought out and designed.

Python, and it’s broad ecosystem. It’s the glue, the face, the hands, the legs. It, plus insanely smart engineers let us move quickly, adapt to anything that popped up and put together a kick ass product.

8 months.

With Python’s help, we put together a ready-to-ship simple to manage, secure, reliable cloud-storage backed virtual storage appliance. 8 months! This isn’t a web app, or a another web service. This is something just shy of putting actual hardware into a datacenter! We’ve got python deep in our DNA. (And heck – the president of the company wrote a good chunk of it! How awesome is that?)

The moral of the story, however brief it is – when you need to sell Python to your boss, or that client – or if you’re just starting out. Don’t sell it with words. Sell it by showing it’s effectiveness, it’s speed. Just do it – that’s what Python is here for. helping you get your job done and succeed.

Python is not meant to be the prettiest, or the fastest-to-run, or the one with the latest feature du-jour. It’s there to let you get your job done. That’s why I love it, and much of the community around it – it’s the pragmatism, how will this let people be effective.

I wanted to take a moment not just to cheerlead – but to also specifically thank the projects and tools out there in the community that helped us get the job done.

  • Python (duh)
  • The Python standard library: This can not be stressed enough! Despite anything else we use from the ecosystem, the standard library is rich enough that it’s a massive time, and life saver. This is why I don’t believe that “getting rid of it” makes any sense at all. Python’s success is not just a by product of the language itself – the standard library is such a big selling point it can’t be stated enough.
  • Django
  • Virtualenv / Virtualenvwrapper
  • Nose
  • pip
  • fabric
  • paramiko
  • lxml
  • pycurl

And many, many others (including obviously non-python projects). Thank you, thank you, thank you!

In closing – Thank You!. Without Python, as a language – Python as a community, Django, and so many others – we would not have been able to achieve our goals. There’s so much to be proud of as a community, and sometimes we loose sight of it while debating frameworks, this, that or the next thing. Python is the perfect secret weapon for people and companies just trying to get things done

Thank you all – hopefully we can give back as much as we’ve gotten.

Posted 16 March 2010 † jesse § Programming § Python § pycon 2010 ‡ View Comments °

Say Hello – Nasuni Launches Today!

nasuni_final.png The company I’ve worked for since July of last year – Nasuni Corporation (a startup in Massachusetts) has gone live! This is the culmination of a lot of hard, but exceedingly fun and exciting work over the past months.

The Nasuni team is an excellent one – and one I am very, very proud to be a part of. Our product is called the Nasuni Filer – a simple-to-use, versioned, encrypted and cloud-storage backed virtual NAS (network attached storage) server (click here for more information).

Without going into all of the features, our goal in making this was to make cloud storage simple, accessible and secure – and I know we’ve accomplished all three. All you do is download it, boot it and start using it – once you do so you have access to truly unlimited storage. It’s an unlimited filesystem for the cloud. Here’s the elevator pitch:

Nasuni has developed a virtual file server, called the Nasuni Filer, that delivers unlimited file storage and complete file protection for businesses. Working in partnership with leading cloud storage vendors, the Nasuni Filer leverages the vast capacity of the cloud to store and protect company files offsite, while retaining the local functionality and performance of a traditional NAS.

This technology allows businesses to use the cloud provider of their choice as a replacement for traditional primary storage. Snapshots, file versioning, and offsite storage are integrated into the file server itself – ensuring business file are safe and secure at all times. No need to manage complex backup and DR schemes – if the file server is running, files are protected.

We’ve launched the Beta of the product today – anyone can sign up, download and use it. Anyone can give us feedback and suggestions – I encourage all of you who might need something like this to download and give it a try. If you want – go check out the videos we’ve put together showcasing the Filer (and better yet – check out the awesome animated cartoon we have on the front page).

Most of you know that my blog is mainly Python oriented. Suffice it to say, Nasuni – and the Nasuni Filer make use of Python for a wide range of tasks. We use Python, Django and as much of the Python ecosystem as we can to drive everything from the website, to the GUI on the appliance itself – Python is part of the DNA of the company, and it has served us well. Without Open Source and Python – I don’t think it would have been possible to build what we have built in as little time as we have.

We have a strong dedication to not just Python, but open source in general (and a fair number of us will be at PyCon this month). As time progresses, now that we’re exiting stealth mode we plan on possibly open sourcing stuff we feel would benefit the community. Some of us already push patches back where and when we can, but as I said – as time progresses this involvement will only increase.

So not only am I proud to announce the product, be part of this team and to see what we’ve made, I’m also happy to thank so many people in the Python and OSS community which have helped us reach this point.

So go – check it out, let us know what you think.

Posted 09 February 2010 † jesse § Personal § Programming § Storage ‡ View Comments °

Unladen Swallow: Python 3′s Best Feature.

holy-shit-awesome-2.jpgWe all know (well – unless you’ve under a rock) about Unladen-Swallow, the semi-Google-Sponsored optimization-focused branch of Python 2.x. Collin, Jeffrey and many others have been working tirelessly on porting the CPython interpreter over to LLVM, applying optimization patches, writing tests, etc all aimed at speeding up real-world operations and code since before last year’s PyCon – at that time, the first release was already working hard, rendering YouTube’s templates. (See the Project Plan for more information)

You probably also saw the 2009Q3 testing/performance results (here), showing a hefty speed increase for various operation. The 2009Q4 will be coming soon.

What you probably don’t know – yet, is that a PEP is coming, proposing Unladen Swallow be merged back to Python-core – specifically, the Python 3k branch. Yeah, that’s right – a PEP to merge Unladen Swallow back into the mothership, but to Py3k only. Talk about a shot in the arm!

Quoting Collin Winter:

Python 3 is the natural place for something like Unladen Swallow to land: Python 3 is clearly the future, and I think improved performance will be a strong selling point in the language’s favor. What we and others have done for Python 2 applies directly to Python 3.

…

One of the things we’ve really focused on with Unladen Swallow is creating a solid foundation that the wider CPython developer community can build on top of. We’ve fixed some nasty problems in the x86-64 JIT, for example, so that the CPython community can focus on the fun stuff — more optimizations! Having the basic JIT infrastructure in place and well-tested opens up a world of opportunities that didn’t exist before.

…

I think PyPy’s focus on research is incredibly valuable. Unladen Swallow has always been about, “what can we achieve right now?” One of our guiding principles has been, Do nothing original, don’t be innovative. PyPy is looking 10 years down the road and thinking, What can we do if we question everything? I think having those different perspectives is important to the community. So, no, I don’t see us in competition (with PyPy) at all.

There’s been a lot of talk about how slow adoption of 3k has been – a backwards incompatible implementation of a widely used language will always have slow(er) adoption then releases of the same language, backwards compatible – but I’d argue that to an extent, part of the reason is that there “haven’t been enough carrots” in the Py3k cart to provide enough of an incentive for projects, libraries, etc to move to it at an accelerated rate.

This, well – this changes things. A significantly faster interpreter, a more rational GIL thanks to Antoine Pitrou – these are the improvements to Python 3 which make it the perfect target to aim for for new projects and libraries. It should drastically help provide the impetus for larger libraries to move over. Things like these are exactly why something like the moratorium make sense, efforts can be focused on the ecosystem of python outside of the syntax, on the interpreter, the standard library, etc.

This is huge; and with a little luck, a lot of discussion and debate – and a ton of work, by the end of 2010, we’ll have not only the best implementation of Python, as a language to date in Python 3 – but the best interpreter as well.

The PEP should go out for review soon, probably within the next two weeks – and more details/information will be coming out at PyCon in February (you’re coming, right?!).

Also see Michael Foord’s post about this news too.

Posted 06 January 2010 † jesse § Programming § Python ‡ View Comments °