A New Project, Part 19


sticky, part 19



I will be out of town and offline for the next few days, but I wanted to give a quick update on the new project before I left.

I have some new heros in the technology world. They are the men and women that build rich browser-based client-side applications. First-rate examples include Gmail, the PressDisplay newspaper viewer, and the del.icio.us director . Also, rumor has it that we'll be seeing richer web email clients from Microsoft and Yahoo in the near future as well.

So why are these people my heros, and not the developers that build the back-end functionality that the applications depend on? Simply because building rich browser-based user interfaces is really, really hard. I've been architecting and authoring large-scale web applications for the past decade and I haven't run head first into anything this challenging since they day I started teaching myself C++ as an undergrad. Maybe it's just familiarity, but I rarely run across something I just don't know how to accomplish on the backend. In fact, I usually know of a half-dozen ways of solving most of those problems -- the biggest challenge is picking the best solution. But on the client-side I feel lost and alone in a dark forest knotted with thorny briars and twisty paths to nowhere.

Here's an example: The other day I decided to teach myself a bit about Ruby on Rails, the darling application framework du jour. I visited the website, read through the introductory material, skimmed the API, read an article or two published at ONLamp, and more-or-less felt that I could get something up and running with the framework within a few hours. The patterns were logical, the design followed contemporary best practices, and everything was extremely well documented. (It turns out that I didn't find a reason to personally use Rails, but I learned more than enough to comfortably make that evaluation.)

Contrast that with trying to figure out how to start building with the rich client-side application that I have in mind for the new project. Conceptually I understand AJAX just fine. I get the theory behind the DOM and interpreted languages. I'm proficient at XML and XSL and remote method invocation. I'm even willing to say I know a thing or two about good REST design. I can bang out code in a half-dozen languages without cracking a reference manual -- in a few of those I may even know what I'm doing. But getting started with client-side JavaScript and AJAX is an altogether different story. Here's why:

For starters, I tend to do most of my learning on the web. I like almost all of the O'Reilly book series, but for the most part I've found them to simply be a higher-quality and better-edited version of what is already available online. Besides, reference books are heavy and expensive and I never know what I'll need in advance. So when I need to learn about a new technology, or need to brush up on one I'm working with, I just hop over to whatever canonical website deals with the topic and start pouring through specifications and APIs. (In particular I spend a lot of time over at w3.org reading the formal specs. I find that to be the most consistently valuable way to really figure things out.) So when I started hunting for the "canonical" material for AJAX I came up far short. Worse, I couldn't even find a really good canonical site for JavaScript -- remarkable for a uniquitous technology that is nearly a decade old. (In general my feeling is that books are great for detailed discourses into the theory of something, but that any web technology worth using should both be self-documenting and have a dedicated and passionate enough community to document it for free.)

In my quest to better understand JavaScript I started poking through the ".js" files on the websites I like. That's a fruitless path -- not only are most of those files hard to find in the first place (partly due to the history of the <script> tag, partly because the websites don't want you to extract them), but just about every bit of JavaScript is reimplementing the same wheel, and each doing it in their own in-house way. Most sites don't seem to have even internal standards for their JavaScript, much less conform to some external sense of consistency. But it isn't just the code itself -- some of which seems laughably off, even to a novice's eyes -- the patterns employed seem haphazard at best, and non-existent in the worst cases.

Fortunately the flourishing AJAX movement has given the JavaScript community some burgeoning role-models. Blogs like Ajaxian have been an invaluable help in highlighting the best in this new generation of client-side application development. Partly due to reading through Ajaxian I've found Behaviour, Prototype, Script.aculo.us, and Rico -- all projects dedicated to improving client-side development. Each of these efforts should be commended for taking the time to think about and promote rational design patterns and reusable code. In particular I found that Behaviour is an essential part of quality DHTML -- the XML attribute-based model for JavaScript event handling is appalling, and Behaviour at least mitigates that pain.

The documentation for these projects still falls far short of the server-side frameworks I am familiar with. Compared to anything released by Apache Jakarta, on CPAN, or in the Ruby world, the AJAX community lags far behind in figuring out the best way to promote re-use. Patterns are apparently a novelty, with only small clusters of client-side developers picking them up. Automated API generation, such as we've long had in the Java and C++ worlds, still seems to be non-existent in the JavaScript community. (And this isn't a interpreted vs compiled language issue either, the Perl and Python communities do a fantastic job with their shared libraries.) Reusability in the JavaScript world consists of either a) cut-and-paste, or b) copy this .js file, along with its undocumented global variable scoping, to your server and include it in every HTML page. (If my bias isn't clear enough, I don't think that's exactly a scalable long-term solution.)

Interestingly, one of the better documented JavaScript components I ran across was Rico. Perhaps not coincidentally, it was developed by a large corporation. (Incidentally, it was Sabre, built by their Airline Solutions group. Guys -- where were you when I was working there?) But even that is a bit challenged because it is fitting the toolkit into a very narrow box. There are some good ideas in Rico with the AjaxEngine, but it would be better served by building up one small reusable layer at a time. Not everyone is trying to build the same end-user application, and not everyone has control over what format their XML response arrives in.

Another impediment to learning best practices for client-side programming is the sheer diversity of actual clients. Even the building block of Ajax, XmlHttpRequest, is supported differently on each of the major browsers (Firefox, IE, Safari, and Opera). You need to go through hoops to even get started making asynchronous HTTP requests in a cross-platform fashion. And don't get me started on parsing XML or performing XSLT on the client. Even with Google's well-intentioned pure JavaScript XSLT implementation there are so many variables and inconsistencies that it getting even the basics under control is an exercise in patience and luck.

The AJAX world is still young. It has a long way to go before quality client-side application design is intuitive and scalable. Developers today still need to fumble around in the dark to find good solutions to common problems, and I'm sure that many get lost along the way. At the very least, the AJAX community needs to achieve the following:



So am I giving up? Heck no. Like I said, I have some new heros. The people that are blazing the path ahead with applications like Gmail and PressDisplay are an inspiration to me. After jumping in head first I have to admit that if I hadn't seen it done, I would be skeptical that it could be done. But know what I want to do, and knowing that the possibilities are potentially limitless, I still can't wait to use all of these techniques in the new project.

I look forward to your comments. Since I'll be gone for a few days I'll set things up to publish comments without moderation and clean up any spam when I get back. Cheers!