I hinted at this on twitter earlier - but now that I've had a chance to ssh into the box finally, and, well - run some benchmarks and translate PyPy - I am very happy to announce that the PSF has acquired quite the beefy machine for the planned speed.python.org site discussed at the Python VM summit at PyCon 2011.
The goal of "speed.python.org" is to port the PyPy Speed Center from "just" a PyPy benchmark tool/suite/site and into one for PyPy, CPython, etc - essentially, a massive source of benchmark information for a range of full Python interpreters. These benchmarks are real world micro and macro benchmarks - something we have Unladen Swallow to thank for - quoting Reid:
I'm also proud of our macro-benchmark suite of real world Python applications which lives on and PyPy uses it for speed.pypy.org. In all the performance work I've done before and after Unladen, I have to say that our macro benchmark suite was the most useful. Every performance change was easy to check with a before and after text snippet.
I'll also note, some of the benchmark suite lives in the Python HG repo - PyPy has done a lot of customizations and tweaks - most of all, they've turned it into an awesome site and resource, and having this data, comparing say - Python 2.6/2.7/PyPy/Python3/Jython/etc with historical / per build data (so we could find performance regressions for all of these interpreters will be amazing.
There's already work occurring on the PyPy side on codespeed (the tool) to make it more generic and usable outside of just PyPy. There's also two Google Summer Of Code projects which should help - "Turn Codespeed into a multi-project, statistically savvy application" and "Developing a benchmark suite (for Python 3.x)".
My part to play in all of this was to act, as the magical PSF fairy and sprinkle resource dust into the idea. Fundamentally, I view Python.org not as "CPython.org" - I view it as Python-the-Community.org. I view it as a resource where all interpreters share equal footing, where all of our resources should live and grow (I have a side project for that too...). This single, shared machine is step one in realizing that vision (although I'd be remiss in not noting that plenty of non-cpython things also share our HG repository, and use the mailman infrastructure we have today).
That all said - I finally get to "finish" playing the magical PSF fairy for this, and announce, that due to an amazing donation to the PSF from the HP Open Source Program Office (Thanks Bob!) of a huge machine, and the generous, free racking, stacking hosting and monitoring from the kick-ass team at OSU/OSL we now have this mega machine.
Originally, the PSF was going to fund the purchase of this machine from it's own pockets - but Van Lindberg connected me to Bob at the HP Open Source Program Office, and bam - magic happened. HP's generosity got us the following machine:
- DL380 HP DL380G7 X5670 LFF (2U)
- Dual HP NC382i Dual Port Multifunction Gigabit Server Adapters
- HP Smart Array P410i/1GB FBWC Controller
- 4x 4GB (1x4GB) Dual Rank x4 PC3-10600 (DDR3-1333) Registered CAS-9 Memory Kit
- 2x HP 750W Common Slot Gold Hot Plug Power Supply Kit
- HP iLO Advanced including 1yr 24x7 Technical Support and Updates Electronic License
- 4x HP 300GB 6G SAS 15K rpm LFF (3.5-inch) Dual Port Enterprise 3yr Warranty Hard Drive
- 2 HP DL380 G7 Intel® Xeon® X5680 (3.33GHz/6-core/130W/12MB) FIO Processor Kit
Yes. HP donated a box with 12 cores (24 with hyperthreading enabled) and 16GB of ram, and a high speed RAID to us for this project. And man, does it fly:
Hardware:Processor: 2 x Intel Xeon X5680 @ 3.33GHz (24 Cores), Motherboard: HP ProLiant DL380 G7, Chipset: Intel 5520 I/O + ICH10, Memory: 4 x 4096 MB, Disk: 1200GB LOGICAL VOLUME, Graphics: ATI ES1000
Software:OS: Ubuntu 11.04, Kernel: 2.6.38-8-server (x86_64)
OSU/OSL is going to be our host - they assembled the machine, racked, stacked and they're going to be our hosts. The entire team - Lance and Peter especially, have been patient, understanding and incredibly helpful. They got this machine up for us today, and helped me work through some auth issues - returning emails within minutes. I'm really looking forward to continuing to work with them - the open source lab at oregon state is an astounding resource for open source. Astounding.
Over the next few weeks, I will be identifying the people who will be taking this from my largely incapable hands and turning it into the resource it will become - in addition to running benchmarks, it's probably going to help Brett Cannon generate in depth code coverage reports for the standard library - something we intend on breaking out and sharing amongst the interpreters as a common code base.
This is great news - and I'm looking forward to the next few months as the hard part of the project takes off - getting the speed site up and running and working for the community. For the record, I had to test the speed of this beast somehow - so I picked the most resources intensive thing I could think of - translating PyPy using CPython 2.7:
[Timer] Timings: [Timer] annotate --- 732.2 s [Timer] rtype_lltype --- 482.5 s [Timer] pyjitpl_lltype --- 533.4 s [Timer] backendopt_lltype --- 146.4 s [Timer] stackcheckinsertion_lltype --- 38.9 s [Timer] database_c --- 184.9 s [Timer] source_c --- 344.7 s [Timer] compile_c --- 137.5 s [Timer] =========================================== [Timer] Total: --- 2600.4 s
As a point of comparison - checking the latest build bot result from PyPy:
[Timer] Timings: [Timer] annotate --- 1452.8 s [Timer] rtype_lltype --- 748.6 s [Timer] pyjitpl_lltype --- 1206.6 s [Timer] backendopt_lltype --- 560.5 s [Timer] stackcheckinsertion_lltype --- 109.9 s [Timer] database_c --- 712.8 s [Timer] source_c --- 1295.1 s [Timer] compile_c --- 472.6 s [Timer] =========================================== [Timer] Total: --- 6558.8 s
So, yeah - not apples to apples, but there's a slight speedup.
We now have the foundation for an amazing, real world bench mark and testing site for Python/Python's interpreters. This is going to be awesome, and it could not have been done without the work and discussion by the people at the PyCon language summit, the PyPy folks for putting together an awesome site, and without the support of the PSF - but most of all, the generous support from HP and OSU/OSL. My thanks goes out to everyone involved, and I'm honored to be able to help get this project off the ground finally. Let's show everyone what Python - as a community, working on concert with one another is capable of doing.