A New Project, Part 14


sticky, part 14



When I first started working on the new project a month or two ago, I had little more in mind than a convenient way of storing short sticky notes. As the project evolved I began to realize that there were some things that I'd like to include that would set it apart: the application should have a rich client UI, it should be client/server based, it should support sharing and multiple users. As I dug deeper into the design, I began to see that certain technical approaches, such as REST and Ajax, not only made great sense for the task at hand, but also positioned the application for much more than a simple note taking tool.

As time has gone on, thanks in large part to the transparent nature of the development process, I am beginning to see the real potential being unlocked here. The reliance on standards and clean APIs means that the future of the new project is not limited to just the original goals. In fact, the project has already been able to grow and adapt as new ideas are brought forward. For instance, the introduction of a distributed authentication model for users and groups has sparked a fire of potential applications.

Just to keep everyone interested, I'd like to throw a couple of these ideas out there, and where I can, outline how they can be rather easily implemented on top of the project's infrastructure. (This isn't meant to be exhaustive by any means -- if you have ideas of your own, please leave a comment.)

A traditional wiki

I've always wanted the project to use Wiki-style linking to build relationships between the notes. But if you add a tool for the community to vet, accept, deny, or rollback edits, then you have the foundation for a solid traditional wiki (in the style of TWiki). Add to that the ability to endlessly customize the presentation layer, from sticky-notes to RSS feeds to mobile clients, and you start looking at an even better Wiki than most. Then mix in a rich framework for users and groups, a native mechanism for searching, tagging, and exporting data and things start getting interesting. The Orchard back-end will essentially come with all of this out of the gate -- all that will be needed is a server-side presentation layer and then you have your wiki.

A blog

It is almost trivial to write a presentation tier that would display a set of notes in a traditional blog format. And again, since the searching and syndication capabilities are built in to the back-end, the presentation layer would have little work left to do other than build a pretty display. In fact, one of the first things I may do with this framework is to integrate it with the WordPress database -- I'd love to be able to use some of the new project's tools to work with that data. (WordPress is a fantastic blogging tool, but one thing it is not is a web services API.)

A mailserver

Not just a mailserver -- but an email gateway that can help tackle the spam epidemic. To be fair, as a spam fighting tool the new project will only be as good as it's adoption rate (which even on my more optimistic days I don't presume to hope for much). But imagine if you could send an email to an Orchard server and "sign" the email with your Orchard account credentials. If the recipient also has an Orchard account (on any Orchard server), then the chain of relationships can be traced with precision. The recipient can elect to route incoming messages differently depending on relationships -- notes from friends and family could be tagged as a priority and delivered immediately (how they are delivered depends on their client -- it could be to their mobile, as a sticky note, in a traditional email interface, or forwarded to another email account), saved in an inbox queue, archived for future review, or discarded outright. Even better, since the whole system is built on a network of invitations, any user or tree of users that abuses their privileges can be easily cut off from the community.

A secure instant messaging system

This one is a little offbeat, but what if you consider a client that also runs an Orchard server. If you can address a message to a user on the the client (acting as server) then you have an easy way of sending them a message, no matter where they are. Granted, there is still thinking to be done about the discovery mechanisms necessary here -- the system thus far has been predicated on the ability to use DNS to map URIs to specific server instances, but conceptually it should be possible to flip the discovery process on its head and build a true distributed messaging system. (Just about a year ago I wrote about a tiered message delivery system. Guess what? This could be it.)

A large scale distributed document storage system

Okay, I'm reaching a bit here. But if "notes" can contain metadata, and that metadata can take the form of arbitrary attributes, then you have enough to migrate small amounts of data back and forth between servers and clients. I've already started to think about ways that the data can easily be exposed over protocols such as DAV -- it's not a complete stretch to think about being able to mount a filesystem that is entirely backed by a remote instance of Orchard. If you really push the envelope you could extend the persistence mechanism to handle large file sizes efficiently (using a P2P swarming mechanism, perhaps?).

A social network

Bah. They're boring. You can build that one... : )

Message boards / "groups" / forums

Similar in spirit to groups.google.com or groups.yahoo.com. Orchard should have all the parts you'd need to turn it into a full-fledged mailing list and forum. And it would be a little better -- you'd be able to put data in and take data out of the system at your convenience. Building additional client intergration should be possible as well. (If you want to read and manage "Unto" groups in Thunderbird, just build a plugin -- the entire API is out there for you to call.)

What do all of these applications have in common? Aside from the fact that they all can be built on top of the new project, they are also all focused on convergence. From blogging to email to instant messaging, these applications all act as bridges between other well-established formats and protocols. Part what makes this work is the fact that the hub of it -- Orchard -- is explicitly designed to act as a data gateway and to do so in a way that it uses as few home-grown mechanisms as possible. The APIs are all REST web services and the data is format neutral -- in fact, the data will be available as everything from unadorned XML to RDF to RSS XML to OpenSearch RSS XML to JSON to YAML. Everywhere I can I'm looking to leverage existing standards -- the less I have to invent the better. Besides, the entire point is to open up the data and the means by which you can access that data -- a stated priority -- no, a stated requirement -- is avoiding data lock-in.

So where are we now? In some ways, we've just begun, and in other ways, we're already there. The last thing I want to do is over-promise and under-deliver anything, so I don't want to get ahead of myself here. At the same time, I do want you to be as excited about the potential for the new project as I am. And by writing everything down more-or-less as it pops in to my head I hope that no effort is lost, even if this particular project never goes anywhere else.

That said, I'm actually about to take a minor detour on the project. For better or for worse, I'm just one normal guy with a normal life and a normal job, so my ability to parallelize the development effort is limited. For the next couple of days I plan on taking Essex and re-vamping the AWS OpenSearch engine. I want to do this partly as a test of the capabilities of Essex and partly to try out a few ideas I have about an alternative interface for product search. In fact, I'm going to probably push this new application a bit further in one direction (Ajax) than I otherwise would have just to get some practice on the client side.

Do you have other ideas about what can be done with the framework? Don't be afraid to share! But then again, if you want you can just borrow the platform, the ideas, the code, and the love, and build some insanely great thing, make your millions, and wave down to me from such great heights. Just remember to honor the licenses. And, if you really felt generous, I suppose I'd like a shiny new motorcycle.

[Update 2005-06-22: I added the bit about groups and message boards. But more importantly, I forgot to mention what might be the most exciting part. If any or all of these applications are built, and in my head it is just a matter of will before they are, then they interoperate seamlessly right out of the gate. That means you could be out downtown one night, hear a band you like, send a text message to the server to remind yourself about them, sit down at your computer the next day, find a new sticky note about the band, search your friend's notes to see if anyone has heard them before, jot down a new note in your "todo" to buy the album, type in a review about how great the band is, automatically syndicate the rewiew via RSS, watch as people add comments in reply to the review, allow friends to make edits and updates to the review, form an ad-hoc community in a user group around the band, etc., etc.

So why would I have the hubris suggest that all of this could really be built? Simply because I don't believe I won't be building it all by myself (even if I could). By opening up a set of APIs that allow anyone anywhere to extend the functionality of the system by building on top of the simple concepts of users, groups, and notes, the new project aims to be nothing more than a starting place for better things to come. Could someone else do all of this? Certainly! And more power to them -- if and when they do it will be easily to tie the systems together. For instance, I was thinking about how to use Orchard as a photo-sharing application then realized it may a lot more sense to simply integrate Orchard notes with Flickr data. Or with del.icio.us bookmarks. Or with any other Web 2.0 application. So that's why I get so excited about the new project...]