Growth is good. Particularly when you are talking about the number of people that are reading your website. In the case of unto net, that number has been growing consistently over the past year or two. Occasionally there are surges when big sites pick up on stories, such as the one about the end of the music industry, or services like the Amazon OpenSearch RSS feeds. But more and more people are just tuning in to read each day -- something I find extremely exciting and motivating.
Of course, when you suddenly have a huge increase in traffic you may find that you start to outpace your ability to scale with your current infrastructure. That is exactly the case with unto.net right now -- the hardware that is hosting the site simply can't keep up with the load. More specifically, unto.net is hosted on a virtual machine over a Quantact, which is running User Mode Linux to partition real hardware into many virtual linux boxes. Currently I pay only $25 a month for plenty of bandwidth, but only 96 MB of RAM. Now UML is an amazing bit of code, but it is very limited in its ability to dynamically allocate RAM, so each virtual machine is limited to just a fraction of what may be available on the host.
This is all well and good, and would probably suffice for a simple site with modest traffic like unto.net, but when you start adding in richer application services, such as the AWS search engine or the new project, then it's easy to go beyond the available RAM constraints. And when you start hitting your swap partition to serve up pages your site is going to be quickly and ruthlessly brought to its knees. To keep things running I needed to drastically limit the number of httpd processes that can run at one time, which means that some requests are being turned away. Not at all ideal.
So while I have absolutely nothing bad to say about Quantact -- in fact, I have nothing but great things to say about them and Tim, the guy who runs it -- it is probably time to move on to a long term solution. I am going to need it when the new project launches anyway, so it is better to get the site migrated sooner, rather than later.
Here are my new requirements:
- Dedicated hardware.
- Virtual serial console access.
- The ability to install a Linux distro of my choice.
- No control panels like CDesk or Plesk installed.
- Some sort of external firewall (though iptables may suffice).
- Approximately a 2 CPU, 2 Ghz configuration.
- Approximately 1 GB RAM.
- Plenty of disk space.
- Enough bandwidth (it's not really that much).
- Some sort of backup option would be nice.
- Redundant pipes to the internet.
What I'd ideally find is a hosting provider that would lease me the hardware and sell me the bandwidth and support services. I'm not sure what I'd need to pay for all of this. Unto.net is currently (barely) self-sufficient due to the AWS search engine, but I highly doubt that this will last long (unless you all start shopping on Amazon via unto.net, but I'm not going to bet on it).
Running a personal site is a odd hobby. It's not really a business, so it is hard to justify making business-like expenditures on it. But it's also not something I want to give up on just because it costs money and time. I enjoy writing for unto.net, and I very much want to keep building applications here. Running advertisements on unto.net would make me feel dirty and I certainly don't feel like I have earned the right to ask for donations. (Unless someone wants to donate rackspace or hardware or help hosting the site. Actually, along those lines, if someone wants to learn a bit about how to admin a site, I will need help once the new project launches.)
Anyway, if anyone has any ideas about how to move forward, I'd love to hear it. Web searches for hosting providers are essentially useless, so advice is certainly appreciated. Please comment on this post or email me with your thoughts. As always, many thanks!
[Thanks to Wikipedia for the Mac bomb screenshot.]