| Subcribe via RSS

OLPC: Shipping a Beta?

January 7th, 2008 | | Posted in Programming, Python, Technology

Some of you might remember, coming out of the last PyCon I was (and have been) jazzed about the OLPC project's potential.

Being exposed to technology changed my life, and taught me a new way to learn, expand and grow. I've seen it happen to many other people, in many other walks of life. While it is true that many of the countries the OLPC targets need "other" supplies than computers - it is also true that from a technologist standpoint - the OLPC project offers a way for those of us who have more technical skills to give back.

Donating money to large intangible organizations - or to the people you see parading the same pictures of starving children on TV is the simple route. Going to those countries and helping directly - while infinitely more concrete - is not reasonable in most cases. The OLPC stands out in the middle - something I, as a programmer can contribute my most important asset (my brain (I hope)) to and hope to make a difference.

Like all projects, the OLPC has had it's share of political issues - and I can't say I agree with everything that's come from the less technical arm of the project, but for the greater good, I'm willing to set that aside.

Unfortunately, I was not able to take advantage of the buy one-give one campaign they did last year, it just was not feasible given what was going on. I know, and encouraged many other people to do so however.

This weekend though - a post by someone I highly respect entitled "The OLPC nightmare" made me dig a little bit more. You can tell - by that post - that he's a little miffed at having bought what seems to be a Beta version of a laptop:

In fact just about the only thing you can do well with the laptop is drop it— which is good because I promise that after a couple of hours with it, you'll want to test its resilience to impact.

In that post, he also links to a review done by the Economist (also one of my favorite reads) entitled "One Clunky Laptop Per Child" they too make some of the same assertions Antonio does.

Yes - I do not have one of these laptops - I am stuck only playing with the system via virtual images on a sickly powerful MacBook Pro, but I do remember playing with the prototype at PyCon last year, and based off of these articles and others, it feels like what got shipped was not terribly far from the prototype I played with early last year. That's unfortunate for many reasons - not the least of which is something the Economist wisely points out:

Ultimately the OLPC initiative will be remembered less for what it produced than the products it spawned. The initiative is like running the four-minute mile: no one could do it, until someone actually did it. Then many people did.

The OLPC has spawned a lot of knock-offs, competitors, etc. None of those have goals as lofty and "pure" as those of the OLPC, but the landscape has changed. Shipping a Beta labeled as "GA" never works out well (as commercial software vendors know, but many open source projects never learn).

I'll be interested to see the opinions of others, especially those I know who bought the laptop excited at the prospects and that have no moral or political collateral invested in the project except as technologists.

PyObjC: Run in Python automator action.

December 15th, 2007 | | Posted in Apple, Programming, Python

This is an excellent post from yesterday - the author outlines how, using 10.5 PyObjC bindings and the Leopard Automator, you can add a "run python script" automator action.

I particularly like his followup post on where he outlines how to use the new 'run in python' action to ship files off to Amazon's S3 storage.

The latter of the two posts actually hooked me up with the "boto" python module - which looks like an excellent module for interacting with Amazon's web services, something I've been examining recently for a side-project.

PyObjc 2: Leopard, Python 2.5.1 and You.

December 14th, 2007 | | Posted in Apple, Programming, Python

So, most everyone know that Apple released OS/X Leopard a bit ago - I was on the early adopter train, and have not had any problems (except for the occasional spaces crash because I mash buttons fast).

One of the cooler things that came out with Leopard was a completely overhauled PyObjC built-in. Including Webkit bindings, and many other things (see here and here for more details), a built-in version of twisted and many other things.

However, I was a moron. I was knee-deep in debugging a problem and compulsively replaced my came-with-leopard version of Python with the python.org python 2.5.1 build. (Anyone with tips on how to undo this, please let me know).

Once I dropped in the 2.5.1 version: poof. I managed to wipe out all of the delicious stuff built into my shiny new OS, which made me a sad panda. Once the PyObjC 2.0 stuff hit the PyObjC SVN tree (here) though, I was filled with hope that I could at least restore the PyObjC stuff.

Alas! It was not to be. Come to find out, there's a hard coded string in the macpython makefile which causes the pyobjc build to use the wrong SDK and puke and die. After much wailing and gnashing of teeth, I gave up not realizing what the problem was. You should see a problem like this:

C compiler: gcc -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -fno-common -dynamic -DNDEBUG -g -O3

compile options: '-I/Library/Frameworks/Python.framework/Versions/2.5/include/python2.5 -c'
gcc: Modules/_Foundation_nscoder.m
Modules/_Foundation_nscoder.m: In function ‘imp_NSCoder_encodeArrayOfObjCType_count_at_’:
Modules/_Foundation_nscoder.m:220: error: ‘NSUInteger’ undeclared (first use in this function)
Modules/_Foundation_nscoder.m:220: error: (Each undeclared identifier is reported only once
Modules/_Foundation_nscoder.m:220: error: for each function it appears in.)
Modules/_Foundation_nscoder.m:220: error: syntax error before ‘count’
Modules/_Foundation_nscoder.m:228: error: syntax error before ‘i’
Modules/_Foundation_nscoder.m:249: error: ‘count’ undeclared (first use in this function)
Modules/_Foundation_nscoder.m:256: error: ‘i’ undeclared (first use in this function)

However, last night I was snowed-in at work and in between bug fixes for work, I managed to dig up a post (with fix) on the issue I was having (thanks Barry).

The short answer is sync the tree from subversion, and then edit the following file:

/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/config/Makefile

And remove the "-isysroot /Developer/SDKs/MacOSX10.4u.sdk" chunk of text. Drop into the sync'ed svn tree and run the 02-develop-all.sh script. Now I can happily import WebKit:

Python 2.5.1 (r251:54869, Apr 18 2007, 22:08:04)
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import WebKit
>>>

The T in IT: Mr. T endorses Hitachi Gear

August 22nd, 2007 | | Posted in Comedy, Personal, Storage

Before I give the links to the videos, I want to give the typical disclaimer:

Disclaimer: The opinions expressed here are my personal opinions, views, discussions, etc. Content published here is not read or approved in advance by HDS, my wife or anyone else for that matter and does not - in any way - reflect the views and opinions, positions/etc of my employer. This is my personal, largely python-related blog. Not my employers.

That being said: A few months ago, I discovered (much by accident) that HDS (Hitachi Data Systems) has started a viral marketing compaign involving Mr. T - yes, the man from the A-Team (whose face graced my lunchbox as a child). Note that massive "lulz" were attained when watching these.

Without passing judgement or in any way stating a direct opinion, here are the videos, in order of creation:

For additional amusement, I will direct you to the Archivas (before we were bought by HDS) viral/spoof/etc video that made it to youtube, here.

Posting from iPhone send food

July 1st, 2007 | 1 Comment | Posted in Apple, Linux, Personal, Technology

After much waiting my iphone is all setup and configured i am going to have to get used to typing on it though. I have fat thumbs.

And yes - it is as good as you've heard.

Bugzilla in Python? Yes, Please.

May 30th, 2007 | 4 Comments | Posted in Programming, Python, Technology

Via various places, I've noted that there has been discussion about the possibility of implementing Bugzilla in alternative languages (:cough:python:cough:). I doubt that the core bugzilla team would pull up their roots and really switch but a python-version of bugzilla (trac doesn't count) in Django would be really, really hot.

I linked to the main wiki page up top - but here's a link to the discussion page.

The primary page lists come common (albeit silly for the most part) "cons" that python is viewed as having. I think the discussion page is more interesting personally, but in either case - who is up for doing an implementation anyway?

I would shave every hair off my body and paint myself blue for a Bugzilla/Testopia implementation in Django it's been on my "list of things to do one day" ever since I started poking at Django a year or so ago.

Google’s Drive Study

February 19th, 2007 | | Posted in Storage, Technology


I saw this post on Slashdot the other day - it's a paper called " Failure Trends in a Large Disk Drive Population". It's a good read for anyone in the storage business - hell, it's a good read for anyone interested in computer. In section 5, under conclusions, they state:

In this study we report on the failure characteristics of consumer-grade disk drives. To our knowledge, the study is unprecedented in that it uses a much larger population size than has been previously reported and presents a comprehensive analysis of the correlation between failures and several parameters that are believed to affect disk lifetime. Such analysis is made possible by a new highly parallel health data collection and analysis infrastructure, and by the sheer size of our computing deployment.
One of our key findings has been the lack of a consistent pattern of higher failure rates for higher temperature drives or for those drives at higher utilization levels. Such correlations have been repeatedly highlighted by previous studies, but we are unable to confirm them by observing our population. Although our data do not allow us to conclude that there is no such correlation, it provides strong evidence to suggest that other effects
may be more prominent in affecting disk drive reliability in the context of a professionally managed data center deployment.

These two points are interesting. In some of the labs I've worked in, an astonishing number of drives die regularly. The manufacturer/distributor excuse has always been "heat issues" or "use cases". Admittedly, the temp. range Google capped at was 50 celsius (122 Fahrenheit). In a rack with densely stacked servers (1-2U machines, rack filled) and with those machines running close to 75% and above CPU load with non-stop disk I/O (read, write, delete/format) and constant machine power cycles the temp. inside the racks could spike far past the 122 mark at which point the failure-trend Google marks starts to spike again.

Of course, in the labs I've been in, we were using these as test bed machines - total/high reliability was not something direly important for the simple fact that these machines were disposable.

Even with that in mind: You should always assume your disk drives are going to fail sooner than you expect. The MTBF on a large enough pool of disks not configured in a "smart" configuration (i.e. raid, arrays, etc). I'm not talking about consumer-use patterns (although, I just had a drive go south on my laptop) - I'm talking about datacenter/IT/etc use cases.

The Google paper is a good reference case, but you should remember that all use patterns are different. An application/test or system that really puts the disks to use can cause drive failures much earlier than you (or any paper) might assume. A good chunk of the "storage industry" realized this long ago - this is why companies (cough) work on software applications and intelligent hardware "wrappers" (arrays, raids, etc) to work around the basic assumption that in a large enough pool of drives, you're going to have near constant drive failure. People might disagree with the prices or methodology, but the fact remains that the basic assumption is true.

Of course, that reasoning can be held for any piece of hardware in the typical data center. Apply too much heat/load to a pool of machines and your failure rate it going to be high unless the machines were designed with high-reliability in mind (which normal indicates RAID/Fiber/etc storage).

In any case, the paper is a good read. I've gone and started rambling. If you're looking for some tools to test drives/filesystems in general, I'd take a look at the standard Bonnie/Bonnie++ and other tools, but also take a look at Rugg (built in python) and also remember that it's important to stress a drive below the filesystem layer. Typically, this means raw-writing to the device - if you're job is to test drive speed/reliability or test the reliability of drive drivers for your operating system, that's a step you can't forget.

Update: StorageMojo has a more detailed breakdown.

Hitachi buys Archivas.

February 6th, 2007 | | Posted in Storage, Technology


That's right. My awesome employer, a storage startup called Archivas is being purchased by Hitachi Data Systems, a wholly owned subsidiary of Hitachi Limited.

This is of course, awesome new for us as a company - but on a personal level this purchase shows a belief and commitment on the part of Hitachi when it comes to the product and technology I've put blood and sweat and tears into for the past 3 years.

Some of the news links:
From the HDS site
Search Storage
eWeek Coverage
MarketWatch

When I go to pyCon this month - I get to go as an HDS employee. This is truly awesome.

HDS, Archivas team on fixed-content archiving

February 27th, 2006 | | Posted in Storage, Technology

HDS, Archivas team on fixed-content archiving
Hitachi Data Systems and Archivas Forge Global Partnership to Create Solutions for a New “Active Archive” Market Space
And that's all I am going to say about that.

The essence of a long-term digital archive

January 16th, 2006 | | Posted in Storage, Technology

You should read this: The essence of a long-term digital archive

An excellent article describing digital archives in the modern IT field. For most people, this article might seem irrelevant (in the scope of things I have talked about previously - it is). However, given that this is the field I work in, it's a subject near and dear to me.

Of course, in the interest of full disclosure, I work in this field, and I also happen to know the article's author.