| Subcribe via RSS
... Sent to peps@python.org - now more discussion/debate and stuff shall occur.
For your own edification, here is what I just sent out. Special thanks to all of the early reviewers.
Edit: The PEP is on the official site: http://www.python.org/dev/peps/pep-0371/
I shot an email out to Python-Dev earlier this week asking for comments/questions regarding my push to get the Processing into the standard library. There's been some decent discussion about target releases and other meta-issues around getting it in.
Right now, it looks like I am going to try to target 2.7 and 3.1 - this makes sense for a few reasons.
That all being said - it doesn't look unfeasible to accomplish - and the response both on list and to me privately has been 95% +1 and 5% -.5 and -1 - the positive response really does make me feel that this is the right approach to take.
I am currently working on revised benchmarks for processing vs. threads vs. pp vs. other right now - I'll be publishing those as soon as I complete them to both here and the mailing list discussion as a counterpoint to some of the open questions.
I'd like to see if any of you, oh internet people, have anything else you'd like to have answered for this or anything you'd like to add to the discussion.
Note, I am not trying to solve the "distributed" problem with the inclusion of this - the remote capabilities of the processing module are a side-benefit - not the primary benefit to trying to get this in. I am taking some of the distributed stuff mentally into account - but the goal is to scratch one specific itch - not to solve everyones problem with a single addition.
Now all I have to get over is some bizarre errors with parallel python ramming into ulimit, uh, limits. Luckily I have everything from a dual core to an eight core to hack on!
So, I am pondering going all-out with Nose, and I am wondering what plugins people find the most useful for it, and also how people are using it.
I see two aspects of nose/any test execution mechanism: Unit testing "native" (i.e: python code) and running tests that are more functional in nature (i.e: not testing python, but instead testing a web interface).
What are the features of nose you found the most useful?
Too bad I couldn't find a decent nose picking graphic for this one.
Barry sent the email out last night that both Python 2.6a3 and 3.0a5 are released - these are the final alphas for both. I'd go and grab em while they're still hot off the presses... Provided you're not already sync'ing from svn/bzr/mercurial/wtf.
... And it's just the unittest I wrote for an application I'm cooking up:
Python is consuming much in the way of processor time. I <3 the processing module. All the workers are doing is tossing integers from one queue to another (in, out, result) and doing random multiplication on those integers.
Delicious.
Guido just shot out email to the dev list(s) announcing he has accepted Christian Helmes' Per user site-packages directory PEP. This is awesome as it will provide a lot of functionality for those of us who don't want to sully the system package tree with modules we install. It could also help simplify things like zc.buildout and virtualenv.py
For giggles, you should read the discussion thread - please ignore my less-than-caffenated contribution to it.
I've finally been given the "gift of time" to work on the processing module PEP - when I was reviewing the text template, I ran across this:
- Leave the little Emacs turd at the end of this file alone,
including the formfeed character ("^L", or \f).
I should really switch to a turd-friendly editor. I think VIM or Textmate are scatologically disinclined.
So here's a question for the lazyweb - I'm looking into doing some command-line/scripted video manipulation and I am looking for pointers to current/well-documented modules/libraries to use.
Specifically - I am looking to be able to stream-edit movies programmatically for a variety of formats (one of which is quicktime for a personal project) - ideally I would be able to load the video file and edit the stream (insert bit from other videos, append videos, etc) inside of Python.
The simplest use-case I can think of is building a montage of various videos, for example - say I downloaded non-flv (or flv formatted) videos of funny-baby scenes from youtube and wanted to chain them together into one big funny-baby montage.
Anyone have any links/pointers - the ones I have glanced at are:
Ideally, the library would allow stream editing/manipulation and be cross-platform. I'm willing to sack the xplatform requirement if the library is really good.
I saw this on the 'tubes today: Concurrency with Python, Twisted, and Flex by Bruce Eckel.
Ever since I read it, I've gotten that tickle again to tackle twisted. He offers a pretty elegant solution to the problem of chunking up the work and spreading it across CPUs.
The draw for me to his solution is the asynchronous way in which the results are dispatched/captured (I'm going to be doing a lot of async work soon). It's a pretty good mini-intro into the semi-unpenetrable fort of Twisted.
Personally, I would have probably approached this using the processing module ala (note, I assumed access to Bruce's methods):
from processing import Pool, TimeoutError pool = Pool(detectCPUs()) # ... skipping building the worklist, in bruce's case it's $cores * steps TASKS = [ step1, step2, step3 ] results = [pool.apply_async(t for t in TASKS] imap_it = pool.imap(TASKS) imap_unordered_it = pool.imap_unordered(TASKS) print 'Ordered results using pool.apply_async():' for r in results: print '\t', r.get() print
Note, of course - I lifted the above right from the documentation examples for the module. Yes - I know, it lacks the network-ability that Bruce's twisted-based solution has innately (which does make his solution quite nice). You could do the same with the processing Client/Listeners in theory, but twisted is doing all of the connection management for him under the covers.
Dangit Bruce gave me the twisted itch again.
And later I'll do a more complete solution with the processing module, and compare the speeds of both against one another. I prefer the single-module solution over the twisted.* approach - but I can be swayed. :)
...and finding Quality Assurance people with Python experience (especially advanced Test Engineering skills) is even harder.
The white oak guys at pycon talked about how they used python experience as a measure of skill/intent. Personally, having spent the better half of the last five years looking for people with good Python skill sets, and now looking to build out a serious team - I'm almost always in the position of hiring on people who know "just enough" and teaching them Python.
Teaching python is easy - but not always what you want to end up doing when you're going fast in a startup (although, I've lost track of how many times I've done this).
Excuse me while I go grumble - either you teach QA people automation/test engineering, or you try to find a programmer who wants to learn/do test engineering and teaching them python. No - I don't want programmers to "do QA" - I want them to write code which proves the product.
It's a hard sell to both parties. You also don't want QA people who view QA as subservient to Development, and Developers who don't see QA as subservient to the same. I technically view QA as one discipline, Development as another, but Test Engineering as the Hybrid of the two - and you need a strong background in both. You are writing complex code to test and prove the product - code which can sometimes be as complex as the product itself.
I'm willing to teach anyone anything I know to help them - doing so helps me in the long term build a strong team. Just finding the right people is not-so-easy.
I said in an interview recently - when asked if I wanted to be a Software Engineer or a Software Test Engineer - that if I am writing code in either one of the roles, I don't see the difference. You're writing tools/code and trying to develop new/interesting things in both roles. If you are writing product code: you still have to write test code. If you write tools for testing: you're writing "product code" (that needs testing code). The same pattern of write-test-write-test applies no matter what code you're writing.
There's nothing "less" about writing code that tests a product, just as there is nothing "unique" about writing the product code itself. Both must be tested, both are code. Would I like to write more product code? Sure - but I have to also write the tests that prove it anyway.
Just as an addendum - Corey made a post, and Terry made a post too both discussing this issue. There's also an awesome comment from another test engineer on the reddit discussion here.
|
