Content
Growth is good. Particularly when you are talking about the number of people that are reading your website. In the case of unto net, that number has been growing consistently over the past year or two. Occasionally there are surges when big sites pick up on stories, such as the one about the end of the music industry, or services like the Amazon OpenSearch RSS feeds. But more and more people are just tuning in to read each day — something I find extremely exciting and motivating.
Of course, when you suddenly have a huge increase in traffic you may find that you start to outpace your ability to scale with your current infrastructure. That is exactly the case with unto.net right now — the hardware that is hosting the site simply can’t keep up with the load. More specifically, unto.net is hosted on a virtual machine over a Quantact, which is running User Mode Linux to partition real hardware into many virtual linux boxes. Currently I pay only $25 a month for plenty of bandwidth, but only 96 MB of RAM. Now UML is an amazing bit of code, but it is very limited in its ability to dynamically allocate RAM, so each virtual machine is limited to just a fraction of what may be available on the host.
This is all well and good, and would probably suffice for a simple site with modest traffic like unto.net, but when you start adding in richer application services, such as the AWS search engine or the new project, then it’s easy to go beyond the available RAM constraints. And when you start hitting your swap partition to serve up pages your site is going to be quickly and ruthlessly brought to its knees. To keep things running I needed to drastically limit the number of httpd processes that can run at one time, which means that some requests are being turned away. Not at all ideal.
So while I have absolutely nothing bad to say about Quantact — in fact, I have nothing but great things to say about them and Tim, the guy who runs it — it is probably time to move on to a long term solution. I am going to need it when the new project launches anyway, so it is better to get the site migrated sooner, rather than later.
Here are my new requirements:
- Dedicated hardware.
- Virtual serial console access.
- The ability to install a Linux distro of my choice.
- No control panels like CDesk or Plesk installed.
- Some sort of external firewall (though iptables may suffice).
- Approximately a 2 CPU, 2 Ghz configuration.
- Approximately 1 GB RAM.
- Plenty of disk space.
- Enough bandwidth (it’s not really that much).
- Some sort of backup option would be nice.
- Redundant pipes to the internet.
What I’d ideally find is a hosting provider that would lease me the hardware and sell me the bandwidth and support services. I’m not sure what I’d need to pay for all of this. Unto.net is currently (barely) self-sufficient due to the AWS search engine, but I highly doubt that this will last long (unless you all start shopping on Amazon via unto.net, but I’m not going to bet on it).
Running a personal site is a odd hobby. It’s not really a business, so it is hard to justify making business-like expenditures on it. But it’s also not something I want to give up on just because it costs money and time. I enjoy writing for unto.net, and I very much want to keep building applications here. Running advertisements on unto.net would make me feel dirty and I certainly don’t feel like I have earned the right to ask for donations. (Unless someone wants to donate rackspace or hardware or help hosting the site. Actually, along those lines, if someone wants to learn a bit about how to admin a site, I will need help once the new project launches.)
Anyway, if anyone has any ideas about how to move forward, I’d love to hear it. Web searches for hosting providers are essentially useless, so advice is certainly appreciated. Please comment on this post or email me with your thoughts. As always, many thanks!
[Thanks to Wikipedia for the Mac bomb screenshot.]
May 21st, 2005 at 1:38 am
hi DeWitt, couple of suggestions:
1and1 - 55 a month gets you your own box, with a variety of installs. we’ve been very pleased with the host thus far (and no, i have no other relationship with them to disclose).
ServerPronto - cheapest dedicated host i’ve found (30/month, albeit w/ substantial setup fees) and allows for a variety of distros.
alternately, i may have some space soon. drop me a line if you’re interested. either way, happy hunting ;)
May 21st, 2005 at 7:25 am
Stephen — thanks! At your recommendation I looked into both options. I like a lot of what I saw on 1and1 — they have a configuration called Root Server II that is almost exactly what I need. (It’s $89/mo, rather than $55/mo, though.) However, they also install Plesk on top of the stock RedHat FC2 distro. I’ve had bad experiences with Plesk before, and much prefer a clean install. (Far easier to manage and upgrade.) I sent them an email — we’ll see what they recommend.
Note that this is all so frustrating. I priced the box I needed from Dell, and it’s about $800. Hell, I could put together the machine from spare parts lying around my apartment for free. I’d be happy to put down that money if I could just get 1U of rack space and modest bandwidth from a colo provider. But the overhead of going through a hosting provider is exorbitant. With all the setup fees and monthly charges — plus incremental upgrades on the order of $15/mo for one more IP address or a few more MB of RAM — I quickly start to feel royally ripped off.
Anyone out there have their own cage and want to let me drop a 1U box in there? (Oh! I just reread the last line of your comment, Stephen. Thanks!)
May 21st, 2005 at 9:36 am
ServerMatrix/ThePlanet and EV1Servers are both worth a look, although base prices seem to have jumped a little recently to ~$100 or more. My recent low end ServerMatrix boxes come with a serial console that I can ssh into through some other server.
The other place I use is johncompanies.com, their main business is virtual machines (but using Virtuozzo rather than UML) but they also do BYO 1 RU hosting for $105/month.
I’ve had good experiences with them all, johncompanies is a bit more personalised. ServerMatrix/ThePlanet and EV1 both have forums that might be worth having a dig around in just to see what’s going on there.
May 21st, 2005 at 10:08 am
It may feel like a tangent to you, but I’m really glad you raised this point because most developers find themselves in your situation somewhare along the way. About a year ago I thought I had similar needs, spent a lot of time researching hosts/options, vetted the list… and then along came TextDrive’s initial VC opportunity. TextDrive isn’t geared toward what you have in mind, but you might want to see what they can do for you because part of their business plan includes supplying resources to open source developers. If TextDrive hadn’t come along, I’d have gone with one of the following:
Bytemark - Mark Pilgrim’s host. ‘Nuff said.
JohnCompanies - Already mentioned, but there’s also this: “Special Offer: We are offering our standard FreeBSD or Linux colocation package for $45 / month to Open Source Software contributors - please email info@johncompanies.com for information on qualifying for this discounted price.”
Linode - again, not what you have in mind, and you already have a UML host you like, but they seem to be so well respected and customer-focued that I thought they deserved a mention.
May 21st, 2005 at 1:37 pm
Thank you everybody for the good advice. Please keep it coming — I’ll be looking into this over the next few weeks.
I bought myself a little more time by porting the CGI version of AWS OpenSearch over to mod_perl. Coincidentally mod_perl2 was release this weekend, so things worked out well. Mod_perl is still a real programmer’s tool, not a toy — it’s a good thing I remembered my way around the code.
I’ll keep people posted.
May 22nd, 2005 at 10:34 am
Oops. Went down yet again. I added another 32MB of RAM in hopes that I can get through the continuing growth spurt alive.
May 23rd, 2005 at 3:39 pm
I just contacted John Companies. I was impressed by their virtual server offering (I know, I know) and decided to go with them, at least for a little while. I’m getting pretty good at the setup now, so I suspect that I can do the migration over a few hours here and there.
Again, I want to emphasize that I liked Quantact and highly recommend contacting them if you need a smaller scale solution and want to run your own box. The price is great for what you get and the customer service is fantastic. They are up to some interesting things and I wouldn’t be surprised to hear more about them in the future.
I’ll write more about this post migration. Thanks to everyone for all the tips — I wouldn’t have discovered all this if not for you!
May 24th, 2005 at 2:20 pm
If all goes well, you will be able to read this comment as soon as the new DNS records reach you…