Content
Continuing where we left off yesterday in part 2 on this series about building a new web software application, I wanted to take a moment to discuss part of the data model. In particular, I wanted to cover the user model, and how it will impact this application.
Roughly, we should know the following about each user:
- Username — a unique ID by which we can refer to an individual account
- Email address — we need a way of contacting the individual
- Password — to validate each request
- Friends — so we can share various notes
- Preferences — some way of storing per-user configurations
- Level — or priviledges, or administrative rights, etc., for select individuals
- Status — to indicate whether or not the account is active, disabled, deleted, etc.
One option was to use the email address as a unique ID. This is what systems like Gmail do, but those systems are obviously set up to keep email addresses unique. Flickr uses your email address as the primary login key as well, but that forced them to create an awkward contrivance of having the user choose a public identifier for their account. Granted, logging in with an email address is probably easier for the user, but since we are going to request a different public identifier anyway (so the user does not need to reveal their address publicly to share notes) we may as well ask for that up front. Also, this makes it easier when the user needs to change their email address.
We can probably get away with just storing a hash of the password. In other words, when a user changes their password we will simply keep a MD5 or SHA1 hash of the string and discard the original. If the database is compromised the intruder will not gain a list of all the plain text passwords. This also means that we will never be able to tell the user what their password is, but that is a small price to pay for the additional security.
Authentication on each API call will be done by passing in both the username and the password on each request. There are more secure ways, of course. One could use a two-token authentication model or HTTP AUTH. However, a two-token system would require the client to maintain state in a more involved way, and HTTP AUTH could potentially result in a awkward user experience when dialog boxes popped up in the browser. Moreover, both schemes would make it harder for scripts and whatnot to use the backend web services. As a result, I’m leaning toward following Flickr’s lead and simply requesting the username and password on each (stateless) request.
While I don’t expect that this application will need a full friend-of-a-friend framework, I do think it would be useful to say that certain notes are one of (a) private, (b) visible to friends, (c) visible to everyone. In fact, there should also be a distinction between read access and write access for any given note. At this time, however, my inclination is to not bother with a UNIX-style group system, and hope that a friends model will suffice.
The preferences data can probably be stored as simple key/value string pairs. But this will need to be examined in more detail as we see what type of information is required.
The priviledge system can be arbitrarily complex. In reality, it will probably only need to be sophisticated enough to say whether or not a user is an administrator. WordPress uses a notion of “level” to express this with an integer from 1 through 10, though I’m not sure if any features make use of this degree of granularity.
Also, I have decided to make the initial version of this application “invite only,” similar to Gmail. There are a number of reasons for this. First, and most obvious, it helps control the growth of the service.
But more important, this is a good way of combating abuse. For instance, if you see that a user is abusing the system you can close down their account. But if they can simply sign up for another one than you are constantly playing a game of cat and mouse against the abuser. However, if you know where the user got their invite then you can cut them off at the head. But by always requiring invitations you have an explicit tree of responsibility — there is always someone that can be held accountable, going all the way back to the first account. (BTW, I believe this is why Gmail is still, and will always be, inivitation only. This is partly evidenced by the fact they they effectively offer you unlimited invites, but still keep the model alive.)
Certain other bits of data — such as how a user wants his or her name displayed, may fit into the preferences, or may belong as part of the user itself.
Off the top of my head, these are some of the API calls that we will need for users:
/users/user/[username/invite/[email_address]?username=[...]&password=[...](GET)/users/user/[username]?invite_id=[...](PUT)/users/user/[username]?username=[...]&password=[...](GET, POST, DELETE)/users/user/[username]/reset(GET)/users/user/[username]/preferences?username=[...]&password=[...](GET, PUT, POST, DELETE)/users/user/[username]/friends?username=[...]&password=[...](GET, PUT, POST, DELETE)
I tried to twist my mind around enough to make API calls that look like real REST. Most “REST” APIs, such as Amazon and Flickr, aren’t really that REST-ful. However, since we have the luxury of starting from scratch, it is at least worth considering the right ways to do things.
Most of the API calls take parameters. Each of those parameters is technically optional — however, without those optional parameters (such as username and password), the system may potentially behave in different ways (such as denying the caller access). For example, calling /users/user/dewitt/preferences won’t return anything unless you also pass a parameter of my (or an adminstrator’s) username and password. In other words, the optional parameters of “username” and “password” are orthogonal to the path of the API call, but they will influence what results it returns.
So here’s a concrete example of how it all works:
Say I’m DeWitt, and I have a username of “dewitt” and a password of “untonet”. I would like to invite my friend Kristin, at kristin@unto.net (not a real address) to join the site. I would then make an API call of:
/users/user/dewitt/invite/kristin@unto.net?username=dewitt&password=untonet(GET)
The system will then check to see if the username and password I supplied (dewitt and untonet) have permissions to make an API call of /users/user/dewitt/invite. Since they match, and dewitt has some invitations left, the system then generates a unique token (say, 123456) and stores it in the database. The system then sends an email off to kristin@unto.net saying that she’s been invited and that she can sign up at a particular HTML page (part of the client GUI).
Kristin checks her email and sees the invitation. She then navigates to the GUI which asks her to fill in some information consisting of her desired email address, desired username, password, etc. For example, perhaps she picks “kristin” as a username and “secret” as a password. The GUI now makes an API call of:
/users/user/kristin?invite_id=123456(PUT)
And the body of the PUT is an XML (or similar) document containing all of the user information that Kristin supplied. If the invite id is valid, the new username is available, and all of the required data is present, then a new account will be created, the database update to reflect the state of the invitation, and a confirmation email will be sent out to both DeWitt and Kristin.
Kristin gets her confirmation email and can now make API calls of her own. For instance, to retrieve her new user data, she can call:
/users/user/kristin?username=kristin&password=secret(GET)
This will return an XML document containing all of the user information for “kristin” that she is allowed to see. Or, if she wants to see her list of friends, she can call:
/users/user/kristin/friends?username=kristin&password=secret(GET)
And since I am automatically added to Kristin’s list of friends (since I invited her), part of the result set will be an element containing "/users/user/dewitt".
This is just a high-level synopsis of the user model and the corresponding web services API. I am sure that many things will change during the implementation, but I am hoping that the foundation is sound. If you see any areas that need to be addressed before I continue, please let me know.
Much more to come…
May 14th, 2005 at 4:44 pm
It might be worthwhile to think about supporting different groups of friends. For example, the notes that I would share with my personal friends are likely to be different than the notes that I would share with my colleagues. Notes that I share with my girlfriend are notes that I might not want to share with my parents.
Private notes and admin rights could also be implemented through groups. A note would be private if it were attached to a group consisting of just me. Admin rights could be implemented by going the other way — all notes would be associated with an admin group, of which only a few are members.
May 15th, 2005 at 12:50 pm
Michael (good to hear from you!) — I love the idea of groups. I hadn’t considered the scenario you suggest, but now that you mention it, I completely agree that I should be able to select exactly the group of people that I want to share notes with.
So what is a good interface? The nice things about “friends” is that it is easy. Both to manage and to use.
Groups are a little tricker. One could see a scenario in which each note has a “read groups”, in which any member could view the note, and “write groups”, in which members could edit them. Thus there are two many-to-many relationships between notes and groups. And a many-to-many relationship between users and groups.
This works rather cleanly. However, I think the challenge will be in building groups. I suppose an API could be:
/group/[group]?username=[username]&password=[password]/group/[group]/join?username=[username]&password=[password]/group/[group]/inivite/[username]?username=[username]&password=[password]And the groups that a note belongs to will be managed by the notes interface itself. Obviously I need to think this through a lot more. In particular, there is the whole element of a friends group that I think should continue to exist... I want to look at Yahoo Groups and Google Groups as an example of what needs to be supported.
Oh, and as an aside, I realized that I can simplify the users interface to be:
/user/[username]/...Either way, thanks so much for the suggestion. If I can make this easy to use, then I’d like to get it in the backend for version 1.
May 15th, 2005 at 5:50 pm
I was under the impression that REST-ful API look like
/some/path/to/resource?parameter1=value1¶meter2=value2
To follow that model, I was expecting something like
/invite?email=[email_address]&username=[...]&password=[...]
I’m probably missing something obvious, but why do we need the redundant path of /user/users/blah?username=[...]
I’ve written the string “user” three time redundantly. Why?
May 15th, 2005 at 6:49 pm
Good question — I think it is a subtle distinction.
Recall that the
?username=[username]&password=[password]parameters are all optional, but that they establish the credentials of the caller. They should be considered orthogonal to the action being invoked.In the case of invitations, we need to specify which user the invitation should be sent from. Yet it is not always the same as the person making the call however — i.e., a admistrator could be sending an invite from a user.
Thus we can reuse the same API conventions as before. (And, as I realized, we can drop the first
/usersfrom each call.)Thus invites become:
/user/[usernamedoingtheinviting]/invite/[emailaddress]? username=[username]&password=[password]This feels REST-ful to me, yet it does not contain redundant information. And as a real plus, it is entirely consistent with the other
/usercalls.What do you think?
May 16th, 2005 at 6:20 am
For the groups API, how about something like this:
/user/[username]/groups/friends/?username=[username]&password=[password]/user/[username]/groups/friends/join?username=[username]&password=[password]/user/[username]/groups/friends/invite/[username]?username=[username]&password=[password]Afterall, groups are organized per user (i.e. My friends are different than your friends)
If I wanted to programatically add a friend with a groups API that looks like this:
/group/[group]/invite/[username]?username=[username]&password=[password]I would have to (1) make a call to find all of my groups, (2) iterate through the returned list to find the PK of my friends group, and (3) plug that PK into
[group]in the above.May 16th, 2005 at 7:10 am
Michael — that’s an interesting way of looking at it. If I understand correctly, you are proposing that the name of a group is not globally unique like the UNIX model (and the way I was proposing), but it was relative to each user. So there could exist a “dewitt/friends” and a “michael/friends”, each with completely different users.
So when the user Michael is invited to join DeWitt’s friends, the call would look like:
/user/dewitt/groups/friends/invite/michael?username=dewitt&password=dewittI certainly like the consistency there, insofar as getting the list of my groups should be
/user/dewitt/groups. Or getting the members of my friends group/user/dewitt/group/friends.So if I look at the groups that have read access for a note, it would be returned as a path like
/user/dewitt/groups/friends. That’s pretty slick. When fully qualified, it’s a URI — meaning, it’s unique and stable.I’m a little worried about what happens when a group is orphaned. I.e., if people are getting their access to a note because they belong to my friends group, what happens when my account is deleted? What’s a reasonable default behavior? I suppose that one can handle that by never invalidating the group itself, even if the user is in the “deleted” state.
Also, should groups ever have multiple owners? Probably yes, but I’m not sure whether we’d want to solve that via aliases (i.e., my version of group/college points to the same data as yours), or if we want to introduce the notion of global groups and a more robust ownership model there.
In any case, personal groups — with one owner — seem to be more than enough to give this application the initial functionality it needs. Thanks for the suggestion and the clarification, Michael.
May 17th, 2005 at 2:19 pm
A quiet plea from a chronic password-misplacer — especially for new sites that are interesting beta’s but only prove their usefulness later — please make it possible for me to retrieve or reset my password.