Comments on: Cleaning out the inbox: Updated Import pseudo code, filesystems and workingenv http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/ python, programming and other things Thu, 24 Jul 2008 05:34:55 +0000 http://wordpress.org/?v=2.5.1 By: jesse http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-127 jesse Thu, 28 Jun 2007 17:21:43 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-127 <b>So do you mean for B) that you want to search through all of sys.path, not where all of the matching files are, and then select the last one? Otherwise I don’t quite follow what you are after.</b> Yes, that's what I meant - use the last instance of a name in the sys.path rather than the first. Admittedly, this has a lot of problems associated with it and is easily fixed with a sys.path.insert(0, 'path') As for the rest: I need to dig my heels into PEP304 - ultimately it could be mental dead-end. As for this... <b>I would like to know how a sqlite3-backed bytcode cache would perform.</b> So would I :) So do you mean for B) that you want to search through all of sys.path, not where all of the matching files are, and then select the last one? Otherwise I don’t quite follow what you are after.

Yes, that’s what I meant - use the last instance of a name in the sys.path rather than the first. Admittedly, this has a lot of problems associated with it and is easily fixed with a sys.path.insert(0, ‘path’)

As for the rest: I need to dig my heels into PEP304 - ultimately it could be mental dead-end.

As for this…

I would like to know how a sqlite3-backed bytcode cache would perform.

So would I :)

]]> By: jesse http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-122 jesse Wed, 27 Jun 2007 18:24:22 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-122 I am going to look into resurrecting this patch and pep. May take me a little time, but I really think something like this would be very useful. I am going to look into resurrecting this patch and pep. May take me a little time, but I really think something like this would be very useful.

]]> By: jessenoller.com - Pep 304 followup http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-121 jessenoller.com - Pep 304 followup Wed, 27 Jun 2007 18:22:04 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-121 [...] going to be following up on the comments from Brett and others about python bytecode location for this post as soon as I find sanity again - "Operation Pending Baby 1" has caused a mild for of insanity to set [...] [...] going to be following up on the comments from Brett and others about python bytecode location for this post as soon as I find sanity again - “Operation Pending Baby 1″ has caused a mild for of insanity to set [...]

]]> By: jesse http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-117 jesse Tue, 26 Jun 2007 14:55:09 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-117 Good point Phillip - I will correct my post - I meant to say the $CWD of the script is position 0 Good point Phillip - I will correct my post - I meant to say the $CWD of the script is position 0

]]> By: Phillip J. Eby http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-116 Phillip J. Eby Tue, 26 Jun 2007 14:46:59 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-116 """your current working directory is always position 0 on sys.path""" No, it isn't. That only happens when you run a -c command or start the interpreter interactively. When you run a script, *the directory of the script* is what's in position 0 -- which may or may not be the same as the current directory. “”"your current working directory is always position 0 on sys.path”"”

No, it isn’t. That only happens when you run a -c command or start the interpreter interactively. When you run a script, *the directory of the script* is what’s in position 0 — which may or may not be the same as the current directory.

]]> By: Andrew Dalke http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-115 Andrew Dalke Tue, 26 Jun 2007 11:02:24 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-115 PEP 304 (http://www.python.org/dev/peps/pep-0304/) "Controlling Generation of Bytecode Files" was meant to control where the .pyc files were stored. I think the conclusion and reason for it being withdrawn was that people didn't need it, there were other ways to solve the problem, and it was too coarse grained. Eg, see http://mail.python.org/pipermail/python-dev/2005-June/054419.html . Ah, comment in version control "Authors withdrew some PEPs by mail on python-dev (Apr 26, 2006)." That's Martin's comment in http://mail.python.org/pipermail/python-dev/2006-April/064395.html which says: <blockquote> It's not that the feature is undesirable or the specific approach at solving the problem - just nobody is interested to work on it. So future contributors shouldn't get the impression that this was discussed and rejected, but that it was discussed and abandoned. </blockquote> PEP 304 (http://www.python.org/dev/peps/pep-0304/) “Controlling Generation of Bytecode Files” was meant to control where the .pyc files were stored. I think the conclusion and reason for it being withdrawn was that people didn’t need it, there were other ways to solve the problem, and it was too coarse grained. Eg, see http://mail.python.org/pipermail/python-dev/2005-June/054419.html .

Ah, comment in version control “Authors withdrew some PEPs by mail on python-dev (Apr 26, 2006).” That’s Martin’s comment in http://mail.python.org/pipermail/python-dev/2006-April/064395.html which says:

It’s not that the feature is undesirable or the specific
approach at solving the problem - just nobody is interested to work
on it. So future contributors shouldn’t get the impression that this
was discussed and rejected, but that it was discussed and abandoned.

]]> By: Brett http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-112 Brett Tue, 26 Jun 2007 01:42:04 +0000 http://jessenoller.com/2007/06/25/cleaning-out-the-inbox-updated-import-pseudo-code-filesystems-and-workingenv/#comment-112 So do you mean for B) that you want to search through all of sys.path, not where all of the matching files are, and then select the last one? Otherwise I don't quite follow what you are after. As for the .pyc redirection, that might be doable as an importer/loader (and the rough semantics are already specified in PEP 304). The trick is making sure that package import is handled properly. A package might be found at /some/path/, and then you import from /some/path/subpkg/. But now you need to map that to /tmp._pythonCache/subpkg/ somehow. Guessing off the top of my head you could just have the importer know it is working with a package, denote the root location where the package is rooted off of sys.path, and then handle the .pyc location as needed. But now that I think about it, I am not sure how you would be able to signify that a path is a package cleanly. When you import a package its __path__ entries end up in sys.path_importer_cache just like any other entry on sys.path. You could find the best suffix on sys.path for the __path__ entry, but what if the path is really different and in no way on sys.path (e.g., tacking on a platform-specific directory on to __path__ and thus has a funky directory location like /some/path/x86/subpkg/)? So handling package properly becomes tricky. I guess you could cheat and have the bytecode write out in a naive fashion into the directory (i.e., note the the absolute file name is pkg.subpkg, somehow know that it is an __init__ and not a submodule, and write it to /tmp/._pythonCache/pkg/subpkg/__init__.pyc). I would like to know how a sqlite3-backed bytcode cache would perform. Anyway, I have rambled enough on this topic. =) I have way too many importers to write know. So do you mean for B) that you want to search through all of sys.path, not where all of the matching files are, and then select the last one? Otherwise I don’t quite follow what you are after.

As for the .pyc redirection, that might be doable as an importer/loader (and the rough semantics are already specified in PEP 304). The trick is making sure that package import is handled properly. A package might be found at /some/path/, and then you import from /some/path/subpkg/. But now you need to map that to /tmp._pythonCache/subpkg/ somehow.

Guessing off the top of my head you could just have the importer know it is working with a package, denote the root location where the package is rooted off of sys.path, and then handle the .pyc location as needed.

But now that I think about it, I am not sure how you would be able to signify that a path is a package cleanly. When you import a package its __path__ entries end up in sys.path_importer_cache just like any other entry on sys.path. You could find the best suffix on sys.path for the __path__ entry, but what if the path is really different and in no way on sys.path (e.g., tacking on a platform-specific directory on to __path__ and thus has a funky directory location like /some/path/x86/subpkg/)? So handling package properly becomes tricky. I guess you could cheat and have the bytecode write out in a naive fashion into the directory (i.e., note the the absolute file name is pkg.subpkg, somehow know that it is an __init__ and not a submodule, and write it to /tmp/._pythonCache/pkg/subpkg/__init__.pyc).

I would like to know how a sqlite3-backed bytcode cache would perform.

Anyway, I have rambled enough on this topic. =) I have way too many importers to write know.

]]>