jessenoller.com

I hinted at this on twit­ter ear­lier — but now that I’ve had a chance to ssh into the box finally, and, well — run some bench­marks and trans­late PyPy — I am very happy to announce that the PSF has acquired quite the beefy machine for the planned speed.python.org site dis­cussed at the Python VM sum­mit at PyCon 2011.

The goal of “speed.python.org” is to port the PyPy Speed Cen­ter from “just” a PyPy bench­mark tool/suite/site and into one for PyPy, CPython, etc — essen­tially, a mas­sive source of bench­mark infor­ma­tion for a range of full Python inter­preters. These bench­marks are real world micro and macro bench­marks — some­thing we have Unladen Swal­low to thank for — quot­ing Reid:

I’m also proud of our macro-benchmark suite of real world Python appli­ca­tions which lives on and PyPy uses it for speed.pypy.org. In all the per­for­mance work I’ve done before and after Unladen, I have to say that our macro bench­mark suite was the most use­ful. Every per­for­mance change was easy to check with a before and after text snippet.

I’ll also note, some of the bench­mark suite lives in the Python HG repo — PyPy has done a lot of cus­tomiza­tions and tweaks — most of all, they’ve turned it into an awe­some site and resource, and hav­ing this data, com­par­ing say — Python 2.6/2.7/PyPy/Python3/Jython/etc with his­tor­i­cal / per build data (so we could find per­for­mance regres­sions for all of these inter­preters will be amaz­ing.

There’s already work occur­ring on the PyPy side on code­speed (the tool) to make it more generic and usable out­side of just PyPy. There’s also two Google Sum­mer Of Code projects which should help — “Turn Code­speed into a multi-project, sta­tis­ti­cally savvy appli­ca­tion” and “Devel­op­ing a bench­mark suite (for Python 3.x)”.

My part to play in all of this was to act, as the mag­i­cal PSF fairy and sprin­kle resource dust into the idea. Fun­da­men­tally, I view Python.org not as “CPython.org” — I view it as Python-the-Community.org. I view it as a resource where all inter­preters share equal foot­ing, where all of our resources should live and grow (I have a side project for that too…). This sin­gle, shared machine is step one in real­iz­ing that vision (although I’d be remiss in not not­ing that plenty of non-cpython things also share our HG repos­i­tory, and use the mail­man infra­struc­ture we have today).

That all said — I finally get to “fin­ish” play­ing the mag­i­cal PSF fairy for this, and announce, that due to an amaz­ing dona­tion to the PSF from the HP Open Source Pro­gram Office (Thanks Bob!) of a huge machine, and the gen­er­ous, free rack­ing, stack­ing host­ing and mon­i­tor­ing from the kick-ass team at OSU/OSL we now have this mega machine.

Orig­i­nally, the PSF was going to fund the pur­chase of this machine from it’s own pock­ets — but Van Lind­berg con­nected me to Bob at the HP Open Source Pro­gram Office, and bam — magic hap­pened. HP’s gen­eros­ity got us the fol­low­ing machine:

• DL380 HP DL380G7 X5670 LFF (2U)
• Dual HP NC382i Dual Port Mul­ti­func­tion Giga­bit Server Adapters
• HP Smart Array P410i/1GB FBWC Controller
• 4x 4GB (1x4GB) Dual Rank x4 PC3-10600 (DDR3-1333) Reg­is­tered CAS-9 Mem­ory Kit
• 2x HP 750W Com­mon Slot Gold Hot Plug Power Sup­ply Kit
• HP iLO Advanced includ­ing 1yr 24x7 Tech­ni­cal Sup­port and Updates Elec­tronic  License
• 4x HP 300GB 6G SAS 15K rpm LFF (3.5-inch) Dual Port Enter­prise 3yr War­ranty Hard Drive
• 2   HP DL380 G7 Intel® Xeon® X5680 (3.33GHz/6-core/130W/12MB) FIO Proces­sor Kit

Yes. HP donated a box with 12 cores (24 with hyper­thread­ing enabled) and 16GB of ram, and a high speed RAID to us for this project. And man, does it fly:

Hardware:Processor: 2 x Intel Xeon X5680 @ 3.33GHz (24 Cores), Moth­er­board: HP Pro­Liant DL380 G7, Chipset: Intel 5520 I/O + ICH10, Mem­ory: 4 x 4096 MB, Disk: 1200GB LOGICAL VOLUME, Graph­ics: ATI ES1000
Software:OS: Ubuntu 11.04, Ker­nel: 2.6.38–8-server (x86_64)

OSU/OSL is going to be our host — they assem­bled the machine, racked, stacked and they’re going to be our hosts. The entire team — Lance and Peter espe­cially, have been patient, under­stand­ing and incred­i­bly help­ful. They got this machine up for us today, and helped me work through some auth issues — return­ing emails within min­utes. I’m really look­ing for­ward to con­tin­u­ing to work with them — the open source lab at ore­gon state is an astound­ing resource for open source. Astounding.

Over the next few weeks, I will be iden­ti­fy­ing the peo­ple who will be tak­ing this from my largely inca­pable hands and turn­ing it into the resource it will become — in addi­tion to run­ning bench­marks, it’s prob­a­bly going to help Brett Can­non gen­er­ate in depth code cov­er­age reports for the stan­dard library — some­thing we intend on break­ing out and shar­ing amongst the inter­preters as a com­mon code base.

This is great news — and I’m look­ing for­ward to the next few months as the hard part of the project takes off — get­ting the speed site up and run­ning and work­ing for the com­mu­nity. For the record, I had to test the speed of this beast some­how — so I picked the most resources inten­sive thing I could think of — trans­lat­ing PyPy using CPython 2.7:

[Timer] Timings:
[Timer] annotate                       ---  732.2 s
[Timer] rtype_lltype                   ---  482.5 s
[Timer] pyjitpl_lltype                 ---  533.4 s
[Timer] backendopt_lltype              ---  146.4 s
[Timer] stackcheckinsertion_lltype     ---   38.9 s
[Timer] database_c                     ---  184.9 s
[Timer] source_c                       ---  344.7 s
[Timer] compile_c                      ---  137.5 s
[Timer] ===========================================
[Timer] Total:                         --- 2600.4 s

As a point of com­par­i­son — check­ing the lat­est build bot result from PyPy:

[Timer] Timings:
[Timer] annotate                       --- 1452.8 s
[Timer] rtype_lltype                   ---  748.6 s
[Timer] pyjitpl_lltype                 --- 1206.6 s
[Timer] backendopt_lltype              ---  560.5 s
[Timer] stackcheckinsertion_lltype     ---  109.9 s
[Timer] database_c                     ---  712.8 s
[Timer] source_c                       --- 1295.1 s
[Timer] compile_c                      ---  472.6 s
[Timer] ===========================================
[Timer] Total:                         --- 6558.8 s

So, yeah — not apples to apples, but there’s a slight speedup.

We now have the foun­da­tion for an amaz­ing, real world bench mark and test­ing site for Python/Python’s inter­preters. This is going to be awe­some, and it could not have been done with­out the work and dis­cus­sion by the peo­ple at the PyCon lan­guage sum­mit, the PyPy folks for putting together an awe­some site, and with­out the sup­port of the PSF — but most of all, the gen­er­ous sup­port from HP and OSU/OSL. My thanks goes out to every­one involved, and I’m hon­ored to be able to help get this project off the ground finally. Let’s show every­one what Python — as a com­mu­nity, work­ing on con­cert with one another is capa­ble of doing.

PS: Here’s a run of the basic tests from the cpython repo bench­mark tool

I hinted at this on twit­ter ear­lier — but now that I’ve had a chance to ssh into the box finally, and, well — run some bench­marks and trans­late PyPy...