Comments on: Have GIL: Want Benchmarks. http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/ python, programming and other things Thu, 11 Feb 2010 00:42:02 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: jessenoller.com - Benchmark followup: Google-code Edition http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-6088 jessenoller.com - Benchmark followup: Google-code Edition Mon, 17 Sep 2007 00:06:12 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-6088 [...] last weeks tempest in a blogpot, and my subsequent post "Have GIL: Want Benchmarks" I've been doing a lot of reading, planning and discussion with [...] [...] last weeks tempest in a blogpot, and my subsequent post “Have GIL: Want Benchmarks” I’ve been doing a lot of reading, planning and discussion with [...]

]]> By: Eli http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5701 Eli Fri, 14 Sep 2007 19:16:52 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5701 I'm really glad you're doing this. Python definitely needs good libraries for running multiple processes on different cpus, and your benchmarking efforts will definitely help this happen. I’m really glad you’re doing this. Python definitely needs good libraries for running multiple processes on different cpus, and your benchmarking efforts will definitely help this happen.

]]> By: Michael Dillon http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5535 Michael Dillon Thu, 13 Sep 2007 21:13:21 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5535 Please include Pyro in your tests: http://pyro.sourceforge.net/ Since Parallel Python uses threading inside the module, it really isn't a pure non-threading solution. If you set up a similar test with Pyro using two worker processes on one machine, and a process on another machine to control it all, that would completely remove threading from the equation. It may turn out to have a minimal difference from Parallel Python but that is the kind of thing we want to learn from benchmarking. Also, I think that the benchmark should be structured so that the master thread/process does a fair amount of communication, i.e. to supply the parameters for each loop. That way you can run the test in two ways, first with the Fibonacci computations, and second with a null computation. The second style of test will only measure the overhead. Theoretically, the variance in overhead would be the only difference between the Fibonacci runs, but, as always, you need to run the benchmarks to learn this. Once you have it all running, please distribute it so that people can try it on different CPU and OS types. Benchmarks on a single CPU or OS type can often be deceptive. Please include Pyro in your tests: http://pyro.sourceforge.net/

Since Parallel Python uses threading inside the module, it really isn’t a pure non-threading solution. If you set up a similar test with Pyro using two worker processes on one machine, and a process on another machine to control it all, that would completely remove threading from the equation. It may turn out to have a minimal difference from Parallel Python but that is the kind of thing we want to learn from benchmarking.

Also, I think that the benchmark should be structured so that the master thread/process does a fair amount of communication, i.e. to supply the parameters for each loop. That way you can run the test in two ways, first with the Fibonacci computations, and second with a null computation. The second style of test will only measure the overhead. Theoretically, the variance in overhead would be the only difference between the Fibonacci runs, but, as always, you need to run the benchmarks to learn this.

Once you have it all running, please distribute it so that people can try it on different CPU and OS types. Benchmarks on a single CPU or OS type can often be deceptive.

]]> By: Nick Gerner http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5531 Nick Gerner Thu, 13 Sep 2007 20:42:49 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5531 Thanks for taking the time to verify results. This is the kind of thing you (or me, or anyone) could screw up really easily. And it's important we get solid results on this issue. Thanks for taking the time to verify results. This is the kind of thing you (or me, or anyone) could screw up really easily. And it’s important we get solid results on this issue.

]]> By: jesse http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5504 jesse Thu, 13 Sep 2007 14:40:54 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5504 I got the email and replied, I also linked to Watkins' blog post in mine, I will be following up with him as soon as I have a moment. I got the email and replied, I also linked to Watkins’ blog post in mine, I will be following up with him as soon as I have a moment.

]]> By: Miguel Sousa Filipe http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5503 Miguel Sousa Filipe Thu, 13 Sep 2007 14:37:48 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5503 I agree with you that alternatives must be measured and tested. Check out this other blog post about testing something similar with what you did: http://blogs.warwick.ac.uk/dwatkins/entry/benchmarking_parallel_python_1_2/ Speccially, have a lot at what Mr. Guido Van Rossum said. (9th comment) I've also sent you a email elaborating more on my thoughts about this whole "thinking, testing and choosing" a alternative "parallel/concurrency" API for python. Best regards, Miguel Sousa Filipe miguel dot-thingie0 filipe at-thingie1 gmail dot-thingie2 com I agree with you that alternatives must be measured and tested.

Check out this other blog post about testing something similar with what you did:
http://blogs.warwick.ac.uk/dwatkins/entry/benchmarking_parallel_python_1_2/
Speccially, have a lot at what Mr. Guido Van Rossum said. (9th comment)

I’ve also sent you a email elaborating more on my thoughts about this whole “thinking, testing and choosing” a alternative “parallel/concurrency” API for python.

Best regards,

Miguel Sousa Filipe
miguel dot-thingie0 filipe at-thingie1 gmail dot-thingie2 com

]]> By: jesse http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5496 jesse Thu, 13 Sep 2007 13:40:06 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5496 I will be posting the results + code once I get all my ducks in a row. I also want to pass it by a few people to verify I didn't do something monumentally screwy. I will be posting the results + code once I get all my ducks in a row. I also want to pass it by a few people to verify I didn’t do something monumentally screwy.

]]> By: Steve http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5451 Steve Thu, 13 Sep 2007 09:49:49 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5451 Have read blog post: want your results. Have read blog post: want your results.

]]> By: Richard Jones http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/comment-page-1/#comment-5395 Richard Jones Thu, 13 Sep 2007 00:07:27 +0000 http://jessenoller.com/2007/09/12/have-gil-want-benchmarks/#comment-5395 Interestingly enough, I've just implemented a GUI-based file downloader using both threading and processing on OS X on a G4 (single core). Using urllib2 to download (reading chunks of 1MB at a time) I found that the threading implementation played better with the GUI (with some semi-icky hacks) than the processing approach. Both approaches had the download worker writing directly to disk, so the traffic back from the worker to the parent was minimal (a "size received" message after each read()). This was downloading from a LAN server, so the download speed was roughly 10MB/s. The semi-icky hack in the threading approach: I had to introduce a semaphore in the worker main loop which only allowed the worker to "fire" once per every two GUI update loops, otherwise the GUI was starved for CPU. So for me it wasn't totally about how to make things go the fastest, but also about how to best share the CPU in an interactive application. Interestingly enough, I’ve just implemented a GUI-based file downloader using both threading and processing on OS X on a G4 (single core). Using urllib2 to download (reading chunks of 1MB at a time) I found that the threading implementation played better with the GUI (with some semi-icky hacks) than the processing approach.

Both approaches had the download worker writing directly to disk, so the traffic back from the worker to the parent was minimal (a “size received” message after each read()).

This was downloading from a LAN server, so the download speed was roughly 10MB/s.

The semi-icky hack in the threading approach: I had to introduce a semaphore in the worker main loop which only allowed the worker to “fire” once per every two GUI update loops, otherwise the GUI was starved for CPU.

So for me it wasn’t totally about how to make things go the fastest, but also about how to best share the CPU in an interactive application.

]]>