Comments on: An Interview With Adam Olsen, Author of Safe Threading | Completely Different http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/ python, programming and other things Thu, 11 Feb 2010 00:42:02 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Adam Olsen http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-139155 Adam Olsen Wed, 04 Feb 2009 06:55:58 +0000 http://jessenoller.com/?p=451#comment-139155 The locks aren't the issue — contention of atomic refcounting is.<br><br>Changing the core to use a tracing GC (note: a generational GC is just one variant), not built on refcounting, is possible. The issue is that, for an exact GC, it'd be a rather massive diff (as safethread is). The resulting code would also be rather ugly. Between the two there isn't enough impetus to do it.<br><br>A conservative GC, such as the Boehm-Demers-Weiser GC, would require less changes. However, retaining the Py_INCREF/Py_DECREF API for third-party extensions would be harder, and there's a risk of compatibility issues (pointers stored in untracked memory, etc). The locks aren't the issue — contention of atomic refcounting is.

Changing the core to use a tracing GC (note: a generational GC is just one variant), not built on refcounting, is possible. The issue is that, for an exact GC, it'd be a rather massive diff (as safethread is). The resulting code would also be rather ugly. Between the two there isn't enough impetus to do it.

A conservative GC, such as the Boehm-Demers-Weiser GC, would require less changes. However, retaining the Py_INCREF/Py_DECREF API for third-party extensions would be harder, and there's a risk of compatibility issues (pointers stored in untracked memory, etc).

]]> By: Alex http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-139154 Alex Wed, 04 Feb 2009 04:22:08 +0000 http://jessenoller.com/?p=451#comment-139154 Isn't one possible solution(and I know it's been discussed on python-dev) replacing the ref counting with a generational GC, since this means you don't need all those locks? If yes could this hypothetically be handled as a diff outside of svn just as safe-threading was? Isn't one possible solution(and I know it's been discussed on python-dev) replacing the ref counting with a generational GC, since this means you don't need all those locks? If yes could this hypothetically be handled as a diff outside of svn just as safe-threading was?

]]> By: Adam Olsen http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-64048 Adam Olsen Wed, 04 Feb 2009 01:55:58 +0000 http://jessenoller.com/?p=451#comment-64048 The locks aren't the issue — contention of atomic refcounting is.<br><br>Changing the core to use a tracing GC (note: a generational GC is just one variant), not built on refcounting, is possible. The issue is that, for an exact GC, it'd be a rather massive diff (as safethread is). The resulting code would also be rather ugly. Between the two there isn't enough impetus to do it.<br><br>A conservative GC, such as the Boehm-Demers-Weiser GC, would require less changes. However, retaining the Py_INCREF/Py_DECREF API for third-party extensions would be harder, and there's a risk of compatibility issues (pointers stored in untracked memory, etc). The locks aren't the issue — contention of atomic refcounting is.

Changing the core to use a tracing GC (note: a generational GC is just one variant), not built on refcounting, is possible. The issue is that, for an exact GC, it'd be a rather massive diff (as safethread is). The resulting code would also be rather ugly. Between the two there isn't enough impetus to do it.

A conservative GC, such as the Boehm-Demers-Weiser GC, would require less changes. However, retaining the Py_INCREF/Py_DECREF API for third-party extensions would be harder, and there's a risk of compatibility issues (pointers stored in untracked memory, etc).

]]> By: Alex http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-64014 Alex Tue, 03 Feb 2009 23:22:08 +0000 http://jessenoller.com/?p=451#comment-64014 Isn't one possible solution(and I know it's been discussed on python-dev) replacing the ref counting with a generational GC, since this means you don't need all those locks? If yes could this hypothetically be handled as a diff outside of svn just as safe-threading was? Isn't one possible solution(and I know it's been discussed on python-dev) replacing the ref counting with a generational GC, since this means you don't need all those locks? If yes could this hypothetically be handled as a diff outside of svn just as safe-threading was?

]]> By: Adam Olsen http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-63981 Adam Olsen Tue, 03 Feb 2009 08:07:10 +0000 http://jessenoller.com/?p=451#comment-63981 Unfortunately, my GC changes didn't pan out. They looked promising on my old dual core laptop, but very disappointing on my newer quad core desktop.<br><br>I almost started separating the semantics of branching and monitors from the GC and GIL removal, but it's not nearly as interesting. Most people want performance, not safety and ease of use. Unfortunately, my GC changes didn't pan out. They looked promising on my old dual core laptop, but very disappointing on my newer quad core desktop.

I almost started separating the semantics of branching and monitors from the GC and GIL removal, but it's not nearly as interesting. Most people want performance, not safety and ease of use.

]]> By: Jospeh Lisee http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-63979 Jospeh Lisee Tue, 03 Feb 2009 04:54:46 +0000 http://jessenoller.com/?p=451#comment-63979 This is sad, I was really hoping for your work to make it into core. Maybe you should of (still should?) try for a branch in official python SVN so your work could of (still could?) been integrated better and worked on by more Python devs. <br><br>Yes, I have a multi-threaded application and pythons GIL cop-out requires me to use too much C++ code. I also don't think things like multiprocessing are the answer. This is sad, I was really hoping for your work to make it into core. Maybe you should of (still should?) try for a branch in official python SVN so your work could of (still could?) been integrated better and worked on by more Python devs.

Yes, I have a multi-threaded application and pythons GIL cop-out requires me to use too much C++ code. I also don't think things like multiprocessing are the answer.

]]> By: Adam Olsen http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-63966 Adam Olsen Mon, 02 Feb 2009 23:56:31 +0000 http://jessenoller.com/?p=451#comment-63966 Wait for PyPy or wait for me to do an alternate implementation. Hard to say which'll be first. ;) Wait for PyPy or wait for me to do an alternate implementation. Hard to say which'll be first. ;)

]]> By: Stan http://jessenoller.com/2009/02/02/an-interview-with-adam-olsen-author-of-safe-threading-completely-different/comment-page-1/#comment-63959 Stan Mon, 02 Feb 2009 21:17:18 +0000 http://jessenoller.com/?p=451#comment-63959 A very interesting read, but it sounds like python-safethread is already dead, according to the front page:<br><br>"Note: As an implementation, python-safethread is dead. It is not worth the effort to continue rewriting CPython to have a true tracing GC.<br><br>However, the semantics presented are still viable and I intend to reuse them in future projects."<br><br>Does this mean we will have to wait for PyPy in the far future to generate a "CPython" (of course, not *the* CPython) implementation without a GIL? A very interesting read, but it sounds like python-safethread is already dead, according to the front page:

“Note: As an implementation, python-safethread is dead. It is not worth the effort to continue rewriting CPython to have a true tracing GC.

However, the semantics presented are still viable and I intend to reuse them in future projects.”

Does this mean we will have to wait for PyPy in the far future to generate a “CPython” (of course, not *the* CPython) implementation without a GIL?

]]>