Broken Bulb

With apologies to Johnny Cash: "Flash, I hate every inch of you."

Falling for Git

You know what? Git must have come a long way in the last year. I keep reading that Git is hard to learn, has rough documentation, etc. But it’s really been quite nice in comparison to many things.

It’s especially nice once you quickly learn that the HTML man pages for Git follow a simple pattern (as I guess many online man page collections must). Just change the end of the URL from git-cvsimport.html to git-push.html or git-pull.html to look up documentation.

That I’ve been able to play around with Git quite successfully and easily just makes my frustration with some Python tools (like easy_install and zc.buildout, particularly its recipes) even more …. frustrating.

And, I’ve totally fallen in love with Git. Yes, I know there are alternatives written in Python that are quite comparable. But Git’s actually been easier to install and figure out (particularly the CVS interaction that I must currently suffer). And people who know me know that I’m no “Kernel monkey”. I’m really impressed with Git’s implementation and general behavior. Very impressed with the implementation.

By the way: if you’re having to work two ways with a CVS repository, this post has been absolutely invaluable. This collection of Git articles has been invaluable in getting some good defaults established, and some tips for building on Mac OS X (with a nice tip to download and untar the man pages directly instead of trying to build them with the asciidoc tool and its terrible dependency on troublesome XML libraries. Goddamn, how I hate XML).

AppleScripting Across the Universe

After a long day at work, I wrote a long message in Basecamp about what I had accomplished, how to access it, etc. But I forgot to submit the message! Crud. I wanted to send it out before morning and didn’t want to go into the office. I couldn’t get any screen sharing connection to go between the machines. I just had a handful of SSH leaps.

AppleScript to the rescue!

This is probably the most AppleScript that I’ve ever written. Fortunately, Safari supports the command do JavaScript ... in tab. After some floundering around with a similar setup on my local machine, I finally figured out AppleScript’s interesting reference notation and was able to ferret out the window and tab containing the unsent message, add some text to the message’s textarea element, submit the form, and return the extended value.

tell application "Safari"
    set message_tab to current tab of window named "Web site > New message"
    set extended to ".... Fun fact - i wrote this before i left the office and forgot to submit it. as a result, i now know how to submit forms like this via AppleScript."
    set post_body_value to "$('post_body').value"
    set extend_value to post_body_value & " += '" & extended & "';"
    do JavaScript extend_value in message_tab
    set body_value to do JavaScript post_body_value in message_tab
    do JavaScript "document.forms[0].submit();" in message_tab
    return body_value
end tell

I pasted the above code into VIM and ran it with the command line osascript command. Worked like a champ.

And because sleep is for the weak, I decided to track down how to do the equivalent in Python. Mac OS X 10.5 provides a “Scripting Bridge” for Python and Ruby (and potentially others), which causes many frameworks and other objects to be dynamically exposed. Without the need (for better or worse) of yet-another-virtual-machine. Anyways, I cobbled the following together:

from Foundation import *
from ScriptingBridge import *
safari = SBApplication.applicationWithBundleIdentifier_('com.apple.Safari')
def find_window_named(name):
    for win in safari.windows():
        if win.name() == name:
            return win
window = find_window_named("Web site > New message")
message_tab = window.currentTab()
print safari.doJavaScript_in_("$('post_body').value", message_tab)
safari.doJavaScript_in_("document.forms[0].submit()", message_tab)

There may be a better way to do the find_window_named method, but I didn’t have the time to track it down. As it was, I was able to do do the above by playing around with everybody’s favorite Python tool, dir(), which verified my suspicion that many of the commands exposed to AppleScript were also available via the Scripting Bridge. This is evidenced by the currentTab() method of a Safari window, analogous to the current tab of window ... AppleScript. And I imagine most of these are just Objective C methods. And since AppleScript editor’s Dictionary browser told me about the do Javascript [v] in tab [t] command, it stood to reason that it would exist on the Safari object. It was there when I did pprint(dir(safari)), and I knew that I’d need to pass in a Tab object.

In any case, it’s awesome that Apple has embraced Python and Ruby and has tied them in to the Cocoa runtime. Historical note: the first Python - Objective C bindings that I know of where commissioned by a NeXT Developer who wanted to use Python and Bobo (zope.publisher) to do web work with NeXT’s Enterprise Objects Framework, without the weight and cost of WebObjects. I think that means that Python was bridged into the Objective C runtime and NeXTStep frameworks before Jython ever got going. I believe that work was done by the developer who later released Objective Everything which bridged into Perl and TCL as well as Python.

Of course, traditional MacPython from the classic Mac OS was also natively tied in to the AppleScript of that era; AppleScript has always supported other dialects (FrontierScript was a common one).

But it’s nice now to see support coming out of both Apple and Microsoft (and Sun too, I guess) for these languages. The above scripting of Safari was surprisingly easy. As was an earlier experiment to fish around my calendar store for incomplete To-Do items. Quite nice.

But what’s especially nice is that I was able to SSH into my office Mac and tell Safari to submit that form that I had neglected earlier.

Java Crybabies

So Mac OS X 10.5 (Leopard) doesn’t ship with Java 6. And now Java people are all sad and mad and yelling at Apple for dropping the ball on this.

Why should Apple go out of their way to provide Java 6? After the aborted Java - Objective C bridge experiments, what else is there to do? “Native” Java applications have still never come close to feeling like a native or near native piece of the operating system. Why should Apple keep throwing engineering efforts at this system?

And whatever happened to OpenStep for Solaris by the way?

Honestly, Apple has a dynamic object runtime environment heavily tied to a C based language already. It’s done much of what Java and the .Net framework are now doing, and has been doing that since the latter half of the eighties. It’s interesting to look back to the mid nineties and at the criticisms of NeXTStep/OpenStep. “Why are you using this Objective C thing? Why not C++?”

Because what NeXT understood that others didn’t was that the runtime is what’s important. NeXTStep came closest to providing a Smalltalk-style runtime of dynamic collaborating objects without being the alien and self contained environment that Smalltalk can often be. Initially tied in to a Unix operating system, it later went multi-platform (at least to an extent - I think FoundationKit, Enterprise Objects Framework, WebObjects, and PDO (Portable Distributed Objects) ran on HP-UX and Solaris, while all of those plus AppKit ran on NT, along with D’OLE (distributing COM/OLE while Microsoft was still struggling to provide DCOM)).

In those same mid nineties, there were other attempts to provide some of the power of the NeXTStep / OpenStep platform. IBM perhaps came closes with their CORBA (Corba 1.x) based SOM, which was also to be at the heart of OpenDoc. SOM was the heart of the fascinating OS/2 2.0 and Warp. It also was used in the classic Mac OS. Of course, it was there in OpenDoc’s brief life. But beyond that, SOM was used to provide contextual menu plug-ins, interestingly enough. But it was still fairly heavy, as CORBA could be. Too much wringing and wrangling to help non-dynamic languages function in a semi-dynamic world.

And then there was Microsoft’s Cairo. Never shipped. Some of its technologies found their way into NT… But the big features? One of the big features was to be an “object oriented file system”. This resurfaced as WinFS for Vista. Twelve-plus years later, and it’s still not done.

And of course, there was Taligent. Initially, Taligent was going to be an all-new Operating System, aggressively object oriented, etc. Apple and IBM together, to make a NeXTStep for the rest of us, perhaps? Except instead of a dynamic language, they decided to go for C++. But they apparently had to pump in a lot of work to overhaul the C++ runtime and try to provide some of the dynamic loading options (of Smalltalk, NeXTStep/Objective C, etc). It was a lot of time wasted, I’m sure. And they eventually had to pull back from the all-new operating system plan. That was probably wise, considering the environment of the time. Apple never was able to complete Copland and Gershwin, and Microsoft never got Cairo finished; BeOS never found a substantial market; and even early NeXT-era Apple wasn’t able to sell the idea of a NeXTStep based Mac OS until they provided the Carbon migration path for the classic Mac APIs.

So Taligent shifted to providing, like OpenStep, differing layers that would provide these object features on top of differing host operating systems. Still never happened. Which is a bit of a bummer - they had some paradigms that would have been interesting to see.

In 1994, Jon Udell wrote a short article titled “A Taligent Update,” subtitled Will systemwide object frameworks reinvent programming?

Well, while Taligent never delivered; and OpenStep faded into WebObjects (providing the OpenStep developer tools on NT, NeXTStep Mach, and Mac OS X Server 1.x, aka Rhapsody); this seems to have actually, finally, come to pass. Cocoa is a killer framework for Mac OS X, with many fans. There are bridges to many other languages (Ruby, Perl, Python, among others). It’s not quite the pervasive system-wide framework that it was in the NeXTStep days, but in Mac OS X Leopard Cocoa looks as though it’s reclaiming its position as king of the hill. (For a while, there were many Carbon APIs that were a bitch to use from Cocoa; or at least those used to the comparative ease of Cocoa programming).

And Microsoft’s .NET framework has delivered similar in the Windows market. Of course, it doesn’t have anything like Interface Builder; but it still seems to have a much better share-and-reuse model than anything that’s come before it in Windows programming. And it’s built on: dynamic object systems. Unlike Java (which I’ll get to in a minute), the .NET framework and core languages (C#) appear to be taking cues from Objective C and purely dynamic languages/systems like Smalltalk: dynamic class extension, for example, is a new feature in C# 3.0. It’s also been possible in Objective C. This can be a dangerous feature; but also quite useful and usable. But it’s nice to see this in languages and systems that try to combine C, which has the benefits of familiarity, with the power of dynamic object-filled worlds.

And it’s much better than the heavy and strained world of COM and CORBA.

So anyways - NeXT, and now Apple, has been ahead of this game for quite some time. Granted, if it weren’t for Apple, NeXT would be another blip like Taligent. Except with a shipping product. But still - they survived. And their system wide dynamic object framework idea seems to have been vindicated.

So what of Java?

Java is the bastard child here. I’ve never been comfortable with it. It’s not cross platform: Java IS the platform. And it’s awkward. Even in its best desktop guise - Eclipse - it’s still a foreign environment on Mac OS X. Even Firefox is starting to feel more natural (and Firefox 3 looks to be trying even harder in this area). Why would we want it? Sun has never seemed to care that much about Java on Macs, except to try to showcase their “see, multi-platform!” message. But Windows has always gotten the lions share of the attention, even though Microsoft long ago stopped caring about Java.

On a side note, we’re seeing the same thing happen now with Flash. Ugh. It’s a memory hog on Mac OS X. Makes poor use of
resources. If Adobe and Sun don’t seem to care enough about providing a truly killer Mac experience, is it any wonder that they’re being kicked off of the island? Apple’s got the dynamic object system language and frameworks (Objective C, Cocoa); and it’s got an increasingly impressive web environment (WebKit, with Canvas support); it has bridges into and out of AppleScript, Python, Ruby, and Perl, all included with Leopard - bridges too!

One just has to look at how huge the “Java in a Nutshell” books have become to know that Java is no small undertaking. And again, I think that Apple probably has stopped caring. They’ve tried to be good Java citizens - from the Java - Objective C bridge to the all-Java implementation of WebObjects; they’ve tried to be make the Cocoa framework appealing to Java developers and probably tried to make Java appealing to Cocoa developers. But it must not have ever happened. It’s all deprecated now.

And I don’t know what’s going on, really. I just see Apple as having other priorities. It’s not like they’re (purely) a not-invented-here company. They have, after all, built in Sun’s DTrace technology. And Apple builds on and gives back to Open Source, with projects like launchd, the new calendar server, bridge support, WebKit, etc.

But Java…? Apple has no real stake in it any more. The last Java application I ran was Eclipse, months ago, just trying to see what life in a fancy IDE would be like. It was disappointing. Desktop Java just doesn’t figure into a Mac user’s life all that much. I’d rather see Apple focus on improving their primary object language (which inspired Java), focus on improving their APIs and offering more features for programmers (all done handily in Mac OS X 10.5 - from a programmer’s standpoint, it’s extremely impressive) and providing a smooth, fast, and natural platform experience (again done handily in Mac OS X 10.5 - see the still-unequaled Interface Builder 3.0; see CoreAnimation; see a sea of new UI object offerings from Apple).

Why would they spend their time fighting uphill to support a platform whose chief aim is to be an anti-platform? At best, desktop Java on the Mac could mean “runs a windows-like-application-almost-as-good-as-windows, maybe.” Well, we now have Boot Camp and Parallels/VMWare for that. Apple wants to provide a killer alternative platform, and they’ve learned that the best way to do that is to be in control. When Mac browsers were suffering - IE on Mac OS X was slow and strange (compared to near excellent behavior in OS 9), Camino and Firefox were big and slow and non-native (Camino did a decent job, but it had different widget implementations for browser skin and in-page rendering), OmniWeb looked beautiful but its support for new HTML and Javascript was far behind, etc - Apple took control of their destiny by building Safari. And they built in on a toolkit that would let them plug in the right widgets and behavior for a native experience. Hence, all Safari users have long enjoyed having spell checking support in their TextAreas. We get that from Cocoa. As of Mac OS X 10.5, we can turn on grammar checking as well. And now, WebKit is leading much of the HTML 5 charge, recently announcing preliminary support for client side database storage, and were among the first (if not THE first) to put forward tags like canvas which may make SVG usable and make ultimately take care of many uses of Flash (remember when Flash was purely an animation tool?). Apple has all of this in their control. They don’t have to put up with lackluster players / viewers from Adobe.

And I think it’s become pretty clear that Apple’s preferred solution at this time for rich cross-platform(ish) UI and Code is - the web. HTML, CSS, Javascript. It powers Dashboard; it’s been a big selling point of the iPhone (although it is admittedly laughable that Apple said “Web 2.0 apps are your iPhone API!”, but it’s still impressive; people have built some very impressive apps with that very system).

Why does AJAX / DHTML succeed where Applets have failed? And even Flash does poorly? It’s not just because it’s “everywhere”, without a need to install and deal with a JRE. It’s because AJAX is part of the web page. It’s not a self contained rectangle that can do really cool things - within the realm of that rectangle. It’s because AJAX leverages the browser’s toolkit, so that a text field in IE behaves like a text field in IE; and a text field on a Mac behaves like a text field on a Mac. Java performance has never been that great on the Mac. Which must mean that no one cared enough to really try to make it shine, or that the technology is really heavy and inferior. I remember groaning every time I used to check the snow report page at one of the ski resorts because of the extra amount of time taken to load up Java and all that accompanies it just to render some scrolling-headlines; scrolling headlines whose text didn’t match (or even anti-alias with) the surrounding text. It’s absolutely useless and pointless. Granted - I have seen some impressive Java applets in the science arenas, and IBM had some cool chess ones in their old Deep Blue v Kasparov (?) challenge. But again, those have been few and far between.

The two desktop Java apps that I’ve used heavily at points in the past, besides my experiments with Eclipse, were with a UML tool, and then LimeWire. Both of these used on Macs. The UML tool was tolerable, but barely. Limewire was terribly slow - click and wait instead of click and point. Desktop Java is dead to me. I don’t know why you’d want to write in it. Cocoa is just an all out balls out better environment, especially for UI programming as (again) Interface Builder remains peerless. And .NET, especially with the Mono implementation, is a far more interesting playground that seems keen on taking in new dynamic, declarative, and functional features, (LINQ, F#, etc), while Java just feels like a big stack of alphabet soup and static typing and not much else. (Although I do understand that Java 6 has started to break this mold).

And as far as languages go: please. Python, Perl, and Ruby alone offer better cross platform capabilities than Java: especially on the server side where Java is supposed to shine. I didn’t have to wait for Apple’s blessing to use Python 2.5 on my desktop Mac. I didn’t have to wait for Apple’s blessing to draw native widgets with it either. I didn’t have to wait for Apple’s blessing to use Python 1.x to control other applications across the scripting bridges of Mac OS 7, 8, and 9.

Now that Java is Open Source (it is, isn’t it?), maybe the Java community can look into what it would take to provide a good Java experience on Mac OS X. I think that is its only hope. It has to become leaner. The dynamic “scripting” language crowd have all been able to find ways to take advantage of different platforms. Why is it on Apple’s head to provide Mac Java? Going back to what I asked earlier - is Sun going to try to get OpenStep going again on Solaris? Is Microsoft going to provide Java 6 for Windows? Is Apple going to provide .NET 3.5 for the Mac?

A response I’ve seen in the Java community is that Apple is arrogant for not shipping Java 6 with Leopard, and for withdrawing any development downloads and many topics related to Java on the Mac. I think that it’s arrogant of the Java community to think that they matter enough for Apple to continue to sink engineering resources into the platform. They’ve sunk a lot in over the years, and there’s never been a huge payoff. I see no reason for them to continue. They have far better alternatives.

IBM or Sun or anyone else out in the Java / Open Source community should take it upon themselves to provide a good platform if they really care. They’re obviously doing it for Linux and Windows.

Sorry, this is a long and rambling post and now it’s quite late at night when I swore I would be going to bed early. But seriously - Java has not mattered to me as a developer or Mac user for years. It’s a dead weight for Apple. Support for Mac OS “Classic” got the boot in Leopard, and even Carbon is looking like its days are numbered. If those two can be cut off, what the hell chance would Java have?

Catching Up

These periods between posts keep getting longer, don’t they?

I’ve got nothing earth-shattering to talk about. Work’s been very busy, and we continue to be served well by Zope 3. I’m still royally confused by things like setuptools and eggs, mostly in regards to how they work in a Zope 3 world when you’ve already got long entrenched ways of doing software. I could not get a good answer from anyone I asked (in fact, I often got wildly competing opinions). So I’m sticking with our internal make-rake-like-ish toolkit which is primarily helpful for automating checkouts from internal and external repositories. I did have some success with zc.buildout, but I don’t yet foresee a time when I can use it to deploy whole sites/applications. I can barely see a time when I can use it on anything but small projects that are relatively stand-alone. There’s just a big gap between The Way Things Have Been Done and The Way That It Seems That Maybe Things Should Be Done In The Future.

Of course, neither setuptools nor zc.buildout seem to have “proper” releases. zc.buildout is in an endless 1.0 beta (beta-30 at this point), and setuptools is at 0.6c7. Does that mean that it’s not even at release 0.6 quality yet? None of this instills confidence in this hurried developer.

The big problem is the legacy code, which is in CVS. Some of it is being extracted out into individual packages that have the proper ‘setup.py’, ‘buildout.cfg’, etc. Finally. But I have no idea how to apply it to the bigger picture, and I’ve found very little written words that target our situation.

The biggest downside of being so busy with customer related work is that it’s very difficult to keep up with discussions, conversations, plans, etc. And I’m sure that my frustrations with lack of documentation, seemingly unfinished releases, and so on, are really the fruit of other hurried developers. I admire them for at least releasing something. It’s more than I’ve done in a long time. It’s more than I see myself being able to do for quite some time.

Anyways, the revolving door of Javascript toolkits keeps turning. I’m now deeply enamored with jQuery. “Write less, do more”. I like it. I like that it doesn’t trample all over Javascript, and thus plays well with others (especially others that play well with others, like MochiKit). MochiKit is just so big… I think I might make a stab at writing, at least for internal use, a lightweight version that brings many of its best concepts out without overlapping jQuery’s functionality. MochiKit brings many wonderful Python-ic functions and tools to the Javascript table that make general development much easier.

I’m also deeply enamored with zc.resourcelibrary which is a Zope 3 add-on that makes it much easier to manage javascript and CSS resources and their relations to each other. Among other things, it helps save resources when they’re not needed. For example::

if len(rendered_boxes) <= 3:
    return self.just_render_the_damn_boxes(rendered_boxes)
else:
    zc.resourcelibrary.need('fancy.scrolling.library')
    return self.render_the_advanced_widget(rendered_boxes)

I’ve also adjusted my coding style, returning to the underscore_separated_words style instead of the camelCasedWords style, at least for functions, attributes, and methods. This is closer in style to PEP 8 (the main style guide for Python code). The Zope style guide differs on this point, using camelCased instead. And PEP 8 does say that it’s OK, if not downright preferred, to stay true to the style around you.

But one thing I learned from looking through Rails code was that the underscore_style was easier to read, since the underscore acts like a space. And I’ve become a big fan of writing code that communicates intent; that reads like a story (somewhat). Extract Method is your friend. I’ve grown very distrustful of excessive nesting, or of having very long bodies inside of a ‘for’ or ‘if’ block.

That’s about it. Hell of an update, huh? Well, work’s really started to become work, and is quite enjoyable. I’ve got a good flow going and don’t feel I have as much need (nor place) to be an advocate or crank. As I’ve mentioned before, we’ve gotten incredible levels of code re-use by building our internal libraries and applications on top of Zope 3, and we’ve been able to grow them so much that they’re really the first level of framework. It was such a struggle to do this in Zope 2, but in Zope 3 it does fall (fairly) neatly into place. Nothing else in the Python web-framework-whatsit world comes close.

The only toolkit that’s even better? SQLAlchemy. It’s pretty much the only way I’ll interact with RDBMS systems in Python from this point out. And I don’t mean I’ll be writing every RDBMS interaction as an object-relational mapping. SQLAlchemy is great because it provides a good connection / pooling infrastructure; a good Pythonic query building infrastructure; and then a good ORM infrastructure that is capable of complex queries and mappings (as well as some pretty stone-simple ones).

Numbers

I don’t know when I last used a spreadsheet for its actual spreadsheet capabilities, on a sheet I designed myself. I think it may go back to AppleWorks (on the Apple II)! Sure, I’ve used sheets like time cards and travel requests that others had made where I just had to fill in the holes. And I’ve received more than my fair share of spreadsheets used like an outliner / lightweight database / structured note list. But I don’t remember how long it’s been since I used a spreadsheet to figure out a budget, to track expenses, or any other dumb mundane thing like that. Until last night.

I was getting ready to pay my mid-month bills, and I was trying to figure out how much I could pay towards one of my credit cards and still have enough cash to cover the expenses remaining in the month. I also realized that I’ve been spending quite a bit at the iTunes Music Store and hadn’t been tracking any of it. I decided that this would be an excellent time to try out Apple’s new spreadsheet application, Numbers. I found a downloadable time trial of Apple’s iWork ‘08 suite, and immediately got to work.

Numbers is pretty damn cool. I don’t know if there are other spreadsheets that behave like this, but in modern times, it seems so obvious: instead of having the big set of cells in one large table, you work in small floating spreadsheets / tables. This is a big deal for so many reasons, with the most obvious being layout. Another great reason is that each table/spreadsheet can be more focused on its job. Already, Numbers felt a lot more intuitive than anything I had used in a long time.

When doing my simple rest-of-the-month budget, my main question was “how much can I pay on this card and still have enough cash on hand for the rest of the month?” Numbers made it easy with its slider option. For just this one cell, I was able to quickly configure it to give me a slider with a range of -700 to -500. When I got the rest of the budget entered, I could then play with the slider and watch its impact on the total-leftover cell. In previous months, I’ve generally done this calculation in my head, or compared where I stood the prior month after paying this particular bill. It was much nicer to whip up a simple spreadsheet where I could make this one particular number interactive and see the results immediately.

So I was able to get a couple of simple but nice looking spreadsheets together quickly that gave me actual data. I could easily play with this data, or just be embarrassed by it (I have spent quite a bit on the iTunes Music Store).

There are still a lot of old-style spreadsheet rules in play, at least in formulas and the like. That’s made a bit easier by being able to use header names (ie, =SUM(Total) or =MINA(Date Purchased)). I think it was Lotus’ Improv, which first appeared on NeXTStep, that worked this way. In fact, I think with Improv, it was the only way you could work: there were no A/B/C/D columns or rows. This was part of a cool feature of Improv wherein you could drag and drop header representations and regroup the data visually without impact on the calculations. I still think that was one of the most forward-thinking spreadsheet applications. But, it’s gone. I believe there’s some open source variation on the idea, possibly written just for GNUStep…?

Still, Numbers is pretty decent. I love the free-floating tables. It does make it much easier to compose complex spreadsheet pages out of multiple tables and data types. It’s pretty easy to refer to other tables as well. And it’s nice to have non-tabular data (text, graphics, etc) floating free from those numbers, making it easier to adjust layouts without impacting cells.

I’m impressed enough that I’m quite likely to buy iWork ‘08, just for Numbers alone. I have a small need for Pages and almost no need for Keynote, but I do find myself needing to get on top of my finances and similar data. Numbers is the first tool I’ve encountered that I think will let me handle my odd needs without requiring a degree or summer course in spreadsheets.

Yahoo TV Is Now Useless

So, I used to use “My Excite” as my little personal portal. It had good TV listings, which is always important. But many years ago, as Excite was was burying its content under more and more ads, I switched to “My Yahoo!”. Which was even better. Great TV listings. So even now, when I don’t have cable or satellite, I found it valuable.

But then they changed. They upgraded to super fancy TV listings full of AJAX-y action.

But do you know what sucks?

THEY CAN’T GET TIMEZONES RIGHT!

It’s been quite a few months now, and they still can’t get my timezone right.

For seven or eight years, I had no problem with localized listings. Although for a few of those years I was in the blessed Eastern Time Zone. But even when I moved out west - no problem.

But now, apparently Yahoo! has invested all of their resources into flashy features but they can’t tell me when a tv show is on according to the time in my area.

Absolutely useless.

For a while, I’ve gotten by with using the TV listings on the “My Yahoo!” page. It didn’t have all of the fancy features, so somehow it managed to get the times right. But as of today, I’m told that I have to use the new fancy-ass “My Yahoo! Beta”.

First thing that happens when I visit the My Yahoo! beta?

“We don’t support Safari….”

Goddammit. Many of their other upgraded YUI based sites work fine in Safari. I can understand them saying this months ago, which was the last time I looked at the new beta. But to still be having an issue? While trying to be a shiny partner of the lovely iPhone? What’s up?

Anyways, I clicked the “live on the edge” button to see the new page. There were the TV listings. In the wrong time zone!

I submitted an email months ago about this and got the “yeah, we know, we’re working on this” response.

I know that time zones are a bitch to work with, but come on: this shit has worked for years. And now it’s been how long since the launch of the fancy new Yahoo TV section? Six months? Seven? Surely someone could have worked this out by now.

I’m so disappointed. Yahoo! was always one of the most reliable web sites. And I appreciate what they’ve given to the developer community with YUI! and other tools. But this little TV listing issue just takes the cake. It makes it absolutely useless, and now “My Yahoo!”, which has been my ‘home page’ for years, is all but useless as this became such a valued resource as their other sources stopped working.

Very frustrating.

And I’m still looking for a good TV listing site. But all I’ve come up with, so far, is pretty much bullshit. Wrong channels, inability to properly remember channels, too many ads, hard to access listings, slow display…. Augh.

Pissed.

Traits / Roles as Alternative to Abstract Base Classes

While digging through the Python-3000 development list archives, trying to figure out the state of thought circling PEP’s 3119 and 3124, I came across this gem:

“Traits/roles instead of ABCs”, by Collin Winter.

With ABCs refererring to Abstract Base Classes (pep 3119).

Winter’s proposal is similar to my recent post, which is that this sort of “capability inference” should be dynamic, and not bound to the rigid nature of the class hierarchy. In my post on this subject, I showed a number of different implementations of a single interface (specification, role, whatever) - only one implementation followed the basic class-instance scenario. All others provided the exact same outward appearance, while internally they were implemented as module-level functions, class or static methods, or a dynamically composed single-use object (a brainless instance was made and had methods dynamically attached).

Winter’s roles/traits system, which refers to roles in Perl 6 and traits in Squeak, is along the same lines. I hope to hell it gains traction.

ABC may be easy as 123, but it can't beat zope.interface

I guess the deadline may have come and gone for getting in PEPs for Python 3000. Guido’s already written up a PEP Parade.

Of particular interest to me has been the appearance of PEPs for Abstract Base Classes (PEP 3119) and the more exhaustive PEP 3124 which covers “Overloading, Generic Functions, Interfaces, and Adaptation.”

Both of these aim to provide ways of saying “this is file-ish”, “this is string-ish,” without requiring subclassing from a concrete “built-in” type/class. But I think they both fall short a little bit, while zope.interface (from the Zope 3 family) provides the best solution.

PEP 3119 (Abstract Base Classes) has a section covering comparisons to alternative techniques, and it specifically mentions “For now, I’ll leave it to proponents of Interfaces to explain why Interfaces are better.” So this is my brief attempt at explaining why.

A quote from PEP 3119 that I particularly like is “Like all other things in Python, these promises are in the nature of a gentlemen’s agreement…” The Interfaces as specified and used in Zope 3 and some other systems are the same way. They are not “bondange and discipline” Interfaces. They are not the ultra-rigid Eiffel contracts, nor are they the rigid and limited Interfaces as used by Java. They are basically a specification, and they can be used (as mentioned in PEP 3119) to provide additional metadata about a specification. There are some simple tools in zope.interface.verify to check an implementation against a specification, but those are often used in test suites; they’re not enforced hard by any system. The agreement might be “I need a seekable file”, which might mean it expects the methods/messages ‘read’, ‘seek’, and ‘tell’. If you only provide ‘read’ and ‘seek’, then it’s your fault for not living up to the agreement. That’s no different than the Python of today. What Interfaces and Abstract Base Classes aim to provide is a better clarification of what’s expected. Sometimes “file-like” in Python (today) means it just needs a ‘read’ method. Sometimes it means the full suite of file methods (read, readlines, seek, tell). Same thing with sequences: sometimes it just means “something iterable”. Other times it means “support append and extend and pop”.

Another side benefit of Interfaces as specification is that they provide a common language for, well, specifications. Many PEPs propose some sort of API, especially informational PEPs like WSGI (PEP 333) or API for Cryptographic Hash Functions (PEP 247). I’ll use PEP 247 as an example for my attempt at explaining why Zope 3’s Interfaces are Better.

A problem with Abstract Base Classes is this: they’re limited to classes. Even when PEP 3119 mentions Interfaces, it does so like this:

“Interfaces” in this context refers to a set of proposals for additional metadata elements attached to a class which are not part of the regular class hierarchy…

It then goes on to mention that such specifications (in some proposals and implementations) may be mutable; and then says that’s a problem since classes are shared state and one could mutate/violate intent. That’s a separate discussion that I’m not going to go into here.

What is important is this severely limited focus on classes. zope.interface works on objects as well, and not just normal ‘instances of a class’ object, but on classes themselves, and also modules.

There are two important verbs in zope.interface: implements and provides. provides is the most important one - it means that this Object, whatever that object may be, provides the specified interface directly.

implements is often used in class definitions. It means “instances of this class will provide the specified interface”. It can also be thought of in terms of Factories and/or Adaptation - “calling this object will give you something that provides the desired interface.”

“What does that matter?” you might ask. Well, there are all sorts of ways to compose objects in Python. A module is an object. It has members. A class is an object. An instance of a class is, of course, an object. Functions and methods are also objects in Python, but for the most part what we care about here are Modules, Classes, and Instances.

Because when it comes down to actual usage in code, it doesn’t particularly matter what an object is. In PEP 3124, the author (Phillip J Eby) shows the following interface:

class IStack(Interface):
    @abstract
    def push(self, ob)
        """Push 'ob' onto the stack"""
    @abstract
    def pop(self):
        """Pop a value and return it"""

Ignore the @abstract decorators, as they’re artifacts of the rest of his PEP and/or related to PEP 3119. What is important is the use of self.

“self” is an artifact of implementation that is invisible to use. Sure, you can write a Stack implementation like this. (Note: I’m going to use zope.interface terminology and style from here on out):

import zope.interface
class Stack(object):
    zope.interface.implements(IStack)
    def __init__(self):
        self._stack = []
    def push(self, ob):
        self._stack.append(ob)
    def pop(self):
        return self._stack.pop()

But when it’s being used, it’s used like this:

def do_something_with_a_stack(stack):
    stack.push(1)
    stack.push(2)
    # ...
    top = stack.pop()
stack_instance = Stack()
IStack.providedBy(stack_instance)
# True
IStack.providedBy(Stack)
# False
do_something_with_a_stack(stack_instance)
# works fine
do_something_with_a_stack(Stack)
# raises an exception because `Stack.push(1)` is passing `1` 
# to `self`.. unbound method, bla bla bla.

Notice that there is no ‘self’ reference visibly used when dealing with the IStack implementation. This is an extremely important detail. What are some other ways that we may provide the IStack interface.

One way is to do it with class methods and properties, effectively making a singleton. (This isn’t a good way to do it, and is just here as an example).

import zope.interface
class StackedClass(object):
    zope.interface.classProvides(IStack)
    _stack = []
    @classmethod
    def push(class_, ob):
        class_._stack.append(obj)
    @classmethod
    def pop(class_):
        return class_._stack.pop()
IStack.providedBy(StackedClass)
# True
do_something_with_a_stack(StackedClass)
# this time it works, because `StackedClass.push(1)` is a class method,
# and is passing `StackedClass` to the `class_` parameter, and `1` 
# to `ob`.

Another variation of the above is using Static Methods:

import zope.interface
class StaticStack(object):
    zope.interface.classProvides(IStack)
    _stack = []
    @staticmethod
    def push(ob):
        StaticStack._stack.append(ob)
    @staticmethod
    def pop():
        return StaticStack._stack.pop()

Again, StaticStack.push(1) and StaticStack.pop() work fine. Now lets try a third way - in a module! Let’s call this module mstack (file - mstack.py)

import zope.interface
zope.interface.moduleProvides(IStack)
_stack = []
def push(ob):
    _stack.push(ob)
def pop():
    return _stack.pop()

Then in other code:

import mstack
IStack.providedBy(mstack)
# True
mstack.push(1)
mstack.push(2)
print mstack.pop()
# 2

So whether we’re dealing with the instance in the first example (stack_instance), the classes in the second two examples (StackedClass and StaticStack), or the module in the last example (mstack), they’re all objects that live up to the IStack agreement. So having self in the Interface is pointless. self is a binding detail.

Jim Fulton, the main author of zope.interface, taught me this a long time ago. Because in Zope 2, you could also make an IStack implementation using a Folder and a pair of Python scripts. Well, those Python scripts (as used in Zope 2 “through-the-web” development) have at least 4 binding arguments. Instead of ‘self’, the initial arguments are context, container, script, traverse_subpath. Just like self is automatically taken care of by the class-instance binding machinery, the four Zope Python Script binding arguments are automatically taken care of by Zope 2’s internal machinery. You never pass those arguments in directly, you just use it like push(ob) and pop().

So there it is - many ways to provide this simple “Stack” Interface. And I believe that both [PEP 3119] and [PEP 3124] are short sighted by focusing on the class-instance relationship exclusively (or so it appears).

And since many objects, particularly instances, are mutable, one could compose an IStack implementation on the fly.

class Prototype(object):
    """ Can be anything... """
pstack = Prototype()
pstack._stack = []
def pstack_push(ob):
    pstack._stack.append(ob)
def pstack_pop():
    return pstack._stack.pop()
pstack.push = pstack_push
pstack.pop = pstack_pop
# Now we can say that this particular instance provides the IStack
# interface directly - has no impact on the `Prototype` class
zope.interface.directlyProvides(pstack, IStack)
pstack.push(1)
pstack.push(2)
print pstack.pop()
2
# We can remove support as well
del pstack.push
zope.interface.noLongerProvides(pstack, IStack)

Examples of dynamically constructed objects in the real world - a network services client, particularly one that’s in an overwraught distributed object system (CORBA, SOAP, and other things that make you cry in the night). Dynamic local ‘stub’ objects may be created at run time, but those could still be said to provide a certain interface.

So now let’s look at whether it matters that you’re dealing with a class or not:

@implementer(IStack)
def PStack():
    pstack = Prototype()
    pstack._stack = []
    def pstack_push(ob):
        pstack._stack.append(ob)
    def pstack_pop():
        return pstack._stack.pop()
    pstack.push = pstack_push
    pstack.pop = pstack_pop
    zope.interface.directlyProvides(pstack, IStack)
    return pstack
@implementer(IStack)
def StackFactory():
    # Returns a new `Stack` instance from the earlier example
    return Stack()
import mstack
import random
@implementer(IStack)
def RandomStatic():
    # chooses between the two class based versions and module
    return random.choice([StackedClass, StaticStack, mstack])

All three are factories that will return an object that provides an IStack implementation, which is exactly the same as the Stack class in the first example. That also claimed that it implements(IStack). When the class is instantiated / called, a new object is made that provides the IStack interface. In Python, another thing that doesn’t really matter is whether something is a class or function. All of the following lines of code yield a result that is the same to the consumer. The internal details of what is returned may vary, but the IStack interface works on all of them:

Stack()         # class
PStack()        # 'Prototype' dynamically constructed object
StackFactory()  # Wrapper around basic class
RandomStatic()  # Chooses one of the class/static method implementations.

And whether we’re looking at the class implementation, or any of the factory based implementations, the result should be the same:

IStack.implementedBy(Stack) # class
# True
IStack.providedBy(Stack)
# False
IStack.providedBy(Stack())
# True
IStack.implementedBy(PStack)    # Factory
# True
IStack.providedBy(PStack)
# False
IStack.providedBy(PStack())
# True

No matter which method of instantiation is used, they should all pass the verifyObject check, which checks to see whether all of the specified members are provided and that the method/function signatures match the specification

from zope.interface import verifyObject
verify_stack = partial(verifyObject, IStack)
all(verify_stack, [Stack(), PStack(), StackFactory(), RandomStatic()])
# True

Now the class-based options will fail on the implementedBy check, because it’s the Class that provides the implementation, not an instance like with Stack

IStack.implementedBy(StackedClass)
# False
IStack.providedBy(StackedClass)
# True
IStack.providedBy(StackedClass())
# False

“OK”, you might say, “but still, why does it matter? Why might we really care about whether these abstract specifications work only with classes? It seems smaller, simpler.”

The main advantage is that specification should (generally) make no assumptions about implementation. If the specification, aka “gentlemen’s agreement” is generally met, it shouldn’t matter whether it’s provided by a Class, an instance, a module, an extension module, or some dynamically constructed object. The specification language should be the same

Going back to PEP 247, the “cryptographic hash API”: there is a specification in that module about what the ‘module’ must provide, and for what the hash objects must provide. Consider also the WSGI spec, the DB-API specs, and all of the other formal and informal specs that are floating around just in the PEPs. Using zope.interface, those specifications can be spelled out in the same fashion. WSGI just cares about a particular function name signature. It can be provided by a single function in a simple module, or as a method from an object put together by a large system like the full Zope 3 application framework and server. It just wants a callable. This is a little bit ugly in zope.interface… but in reality, actually, I think it works. Here’s how it could be specified:

class IWSGIApplication(Interface):
    def __call__(environ, start_response):
        """ Document the function """
    # and/or use tagged values to set additional metadata

This just means that a WSGIApplication must be a callable object taking environ and start_response arguments. A callable object may be a function (taken from PEP 333):

def simple_app(environ, start_response):
    """Simplest possible application object"""
    status = '200 OK'
    response_headers = [('Content-type','text/plain')]
    start_response(status, response_headers)
    return ['Hello world!\n']

Or a class (the __init__ is what is callable here). Maybe the WSGI spec might also state that the result “should be iterable (support __iter__)” Maybe that’s loosely enforced, but the following example shows how the class can make separate declarations about what the class directly provides, and what its instances implement. Instead of using any decorators or magic-ish “class decorators” (the implements, classProvides calls above), we’ll make the declarations for both AppClass and simple_app in the same manner, which matches the style in PEP 3124.

class AppClass(object):
    def __init__(self, environ, start_response):
        self.environ = environ
        self.start = start_response
    def __iter__(self):
        status = '200 OK'
        response_headers = [('Content-type','text/plain')]
        self.start(status, response_headers)
        yield "Hello world!\n"
from zope.interface import directlyProvides, classImplements
# Both 'simple_app' and 'AppClass' are callable with the same arguments,
# so they both *provide* the IWSGIApplication interface
directlyProvides(simple_app, IWSGIApplication)
directlyProvides(AppClass, IWSGIApplication)
# And we can state that AppClass instances are iterable by supporting
# some phantom IIterable interface
classImplements(AppClass, IIterable)

What are the benefits of this, beyond just having a common way of spelling specifications? Instead of, or in addition to, abstract base classes, the core Python libraries can include all of these specs, even if they don’t provide any concrete implementation. Then I could have a unit test in my code that uses verifyClass or verifyObject to ensure I stay inline with the specification.

def test_verifySpec(self):
    verifyClass(ICryptoHash, MyHashClass)

Then, if the specification changes in a new version of Python or in a new version of someone elses library or framework, I can be notified.

Of if the specification undergoes a big change, a new spec could be written, such as IWSGI2Application. Then by process of adaptation (not covered in this post) or interface querying, a WSGI Server could respond appropriately to implementations of the earlier spec:

if IWSGI2Application.providedBy(app):
    # Yay! We don't have to do anything extra!
    # ... do wsgi 2 work
elif IWSGIApplication.providedBy(app):
    # We have to set up the old `start_response` object
    # ... do wsgi 1 work
else:
    raise UnsupportedOrUndeclaredImplementation(app)

Adaptation could provide a means of doing the above… (still, not going into the details.. trying not to!)

@implementer(IWSGI2Application)
@adapts(IWSGIApplication)
def wsgi1_to_wsgi2(app):
    return wsgi2wrapper(app)
# And then, replacing the `if, else` above:
wsgi_app = IWSGI2Application(app, None)
if app is None:
    raise UnsupportedOrUndeclaredImplementation(app)
# ... do wsgi2 work

When you have both specification and adaptation, then you can write your code against the spec. In the above example, the main code does IWSGI2Application(app, None) which means “for the object app, give me an object that provides IWSGI2Application, or None if there is no means of providing that interface.”

If app provides that interface directly, then app is returned directly. Otherwise an adaptation registry is found, and it’s queried for a callable object (an adapter) that will take ‘app’ as its argument and return an object that provides IWSGI2Application.

Another example: knowing that Python 3000 is going to change a lot of core specifications and implementations, such as the attributes for functions (func_code, func_defaults, etc). If an IPy2Function interface were made (and zope.interface or something like it was added to Python 2.x), then code that works with function object internals could program against their preferred spec by adding a line of code:

func = IPy2Function(func)
if my_sniffer(func.func_code):
    raise Unsafe(func)

On Python 2, you’d get the regular function straight through. In Python 3000 / 3.0, an adapter would translate __code__ into func_code, for example. I don’t expect this to happen in reality, but it’s an example of how migration paths could be made between two major software versions, allowing code to run in both.

By taking advantage of this system, my company has seen more re-use with Zope 3 than at any time in our company history. And because (most of) Zope 3 is programmed against specification, we’ve been able to plug in or completely make over the whole system by providing alternative implementations of core specs. This is very hard to do in native Zope 2 (the CMF, on which Plone is based, was probably the first Zope system that started these concepts, which Plone and others were able to take advantage of by providing new tools that matched the provided spec).

At the heart of it, again, is the gentlemen’s agreement, but brought out in full: it doesn’t matter who you are or where you came from (ie, it doesn’t matter what classes are in your family tree or if you are a simple module), as long as you get the job done. There’s a simple contract, and as long as the contract is fulfilled, then everybody is happy.

But if the gentlemen involved can only come from the class system, then there’s still a nasty aristocracy that excludes a large chunk of the populace, all of whom can potentially fulfill the contract. Let’s not cause an uprising, OK?

plispy

Sometimes, it just happens...

>>> pprint(
...     sorted(
...         map(
...             linecount,
...             path('.').walkfiles('*.py')
... )))