As I started to explain in part 1 of this series, I am in the planning stages of building a new web application. And rather than build it first and then release it with some fanfare, I've decided to publish my thoughts and notes along the way. The odds of finishing the application are slim (time being the limiting factor), so this is a good way of ensuring at least a little public humiliation before the idea fades away.
From a user's perspective, what I want to see is pretty clear. I'd like an application I can access via my web browser. I want to be able to drag-and-drop notes around. I want to be able to just start typing text into a note wherever I see one. I want the application to automatically link notes together. I'd like to be able to search my notes. I'd like to be able to double click on the top of a note and have it do that neat window-shade effect. I'd like to be able to have multiple pages of notes, and ideally be able to navigate back and forth through pages of them. I'd like it to feel fast, like a client-side application. I'd like to be able to share some notes, but keep other ones private. I would like to be able to email links to notes to my friends. I'd also like to be able to tag notes, and filter on those tags.
Well, that's what I'd like to see anyway. But when you go back and look at that list, you realize that it is a mixture of both user experience and base functionality. And if even when you strip out the user experience side of it (such as the window shade effect), you still have a pretty reasonable feature set. In fact, you have a feature set that could power any good second-generation Wiki application. You'd have a base feature set that even a more conventional user interface could easily take advantage of. Or perhaps even better, you'd have a feature set that a much more advanced inteface than I could ever imagine will build on top of.
What does this insight imply?
For starters, it implies that there is a strong separation between the user interface and the underlying functionality. This isn't particularly shocking -- good software design has been making this distinction since beginning. But now that we're in the age of Web 2.0, there are suddenly a whole slew of new names for this paradigm. And even better, most of those new terms are directly applicable to client/server web applications like this one.
In particular, the resurgence of REST (i.e., using HTTP the way it was designed) fits this application perfectly. More than that, one realizes that the entire backend of this application is really just a web service with a rich REST API. The UI itself can be entirely decoupled from the the underlying data services. And, to use another acronym, AJAX is just what we need to build a rich client-side experience.
This reminds me a bit of Flickr, and why that site is so great. It's not just because they let you share photos -- tons of sites do that -- but it is because they built an amazing web services API for their own internal use and then opened it up to the public. Half the fun of Flickr is seeing what crazy applications people are building on top of the Flickr backend.
And to that end, this is exactly what I want this application to do. I'd like to see a powerful and accessible web services API that can not only power this one application, but any other application that might want these features. It's your data -- it shouldn't be locked up on someone else's servers.
So looking at this from above, we see a need for:
- A client. In this case, CSS, JavaScript, and DHTML running in a modern web browser.
- A REST web services API. Running on the server side, this encompasses the "business" logic.
- A persistence layer. Some place to store all of the data.
That sounds easy and familar enough. We've all built many applications like that, and there don't seem to be any surprises here. In fact, the hardest parts (for me) will probably be:
- Getting the data model right. Or at least "right enough."
- Dealing with the vagarities of different web browsers.
- Deciding on the feature set for the first version of the API.
- I don't have a spell checker for WordPress. Or a (human) editor for the grammar.
- Time.
There are a few points there that I'd like to expand on. First, getting the data model right (enough) is probably as simple as finding something that meets the need of the web services API. And the web service API will largely be driven by what the first UI (the one in my imagination) requires. If all goes well there, then the data model should work out for now, and will hopefully be somewhat malleable in the future.
Second, the actual data store for this application is -- in an ideal world -- some sort of attributed filesystem. Most of the data is not actually relational. (And if I could give one bit of advice to any candidate interviewing with me, it would be to use the phrase "don't store data in a relational database unless it is relational" as frequently as possible.) That said, for version 1.0, a relational database -- with it's convenient transaction contexts and support for concurrent modifcations -- is almost certainly what I'll choose.
Third, I'm going to basically have to rule out IE as a client simply because I don't have a Windows machine. I could pretend that this makes me sad, but I'll spare you the drama. Fortunately, a good web services API will allow other people to build a Windows UI for me. (See, that decision is already paying off, and I haven't even written a line of code.)
Fourth, finding a good UML tool on Linux is a chore. Dia works, but it is a little clunky and quirky. And you can run a few tools via Crossover -- a coworker suggested Sparx to me today -- but nothing free and easy. So I'm concerned that I'll under-document the project. Which would be fine (and frankly, par for the course) if not for deciding to post the design process along the way. A lack of good diagrams will be an impediment in helping other people follow along.
Fifth, I'm serious about the spell checker thing. I was a horrible speller in grade school. Never really got better at it, thanks to being adept at working a computer. So now when I write on unto.net I tend to pick words that I think I can spell. Please bear with me.
Six, I was able to write the first part of this before work this morning, and the second part after work. Giving this about an hour or two a day I expect it can be done in a month. Though the UI will probably take 10 times as long as I expect it will. (Ever notice how often the UI is the "long pole" on a big project, and yet people always underestimate it?)
So there you have it. A top-down (or was it middle-out?) design in a handful of paragraphs. And even better, I know just where to start. I will probably co-evolve the data model and the web services API and document those parts next. The code itself will flow pretty easily from the API (also, it helps to defer the decision about programming langauges for a while). I have a handful of notes about some problems that I know will pop up (how to return intra-note hyperlinks? what sort of markup is appropriate? how to solve concurrent edits on public notes?) but hopefully I can defer some of that until later in the development. Trying to solve it all up-front would just turn the first phase into a insurmountable obstacle. And besides, I'd get it wrong anyway (that's not self-deprecation talking, that's common sense), so it would be wasted effort.
Stay tuned. More to follow.