I started this post nearly two years ago, and it has been sitting in my “Drafts” folder since then, untouched.
I recently came up against a need for which Amazon’s cloud was once against the hand’s down most cost effective way to meet my needs, and I figured it was worth documenting the use case. If you don’t know already, I play minecraft. I have a realm. I’ve hosted my own server in the past, and have done a whole mess of stuff with it over the years, including some pixelart (which I will link once I’m not at work anymore). Realms don’t support any kind of mods or plugins, or maps, etc. And so I’ve had to do some manual work to get a map of our realm available online. When I was using fragnet (had great experience with them, btw … our use case just changed is all), they had basically one click install options for using dynmap, a popular dynamic mapping system for minecraft, and it basically auto-magically maintained the map for me, no maintenance or anything necessary. It would update reasonably quickly after you explored more of your world, and all was well in the world.
No mods… need to come up with another solution. For the moment, I download the “latest map” periodically from the realm, run it through a locally installed instance of Overviewer (another map generation tool), and then I was going to upload that to my hostmonster hosted website.
Only one problem with this. I have a hard file count limit on my account which is something like 200,000 files.
You might think… hey, that is not an unreasonable limit. That is a lot of damn files to be hosting for a website, no?
Well, yes, you’re right, under normal circumstances. But the problem is that these types of maps are based on the google maps API, and that uses “tiles” to represent parts of the world. The more you allow the user to drill down into the map to “zoom in” and the bigger your world, the larger the number of tiles. My map, with three views, is currently over 500,000 files. And yes, they’re all small, amounting to about 5G of space or so. It isn’t the size that matters (har), its the quantity. I did talk to hostmonster about how to solve this, but their options essentially amounted to upgrading to (much) higher cost hosting packages. Not ideal. I only pay them something like $7/month or something, and the minimum additional cost would have more than doubled that.
So what next? Well, I vaguely recalled that S3 buckets (AWS’ core storage product) can allow you to host static content and turn the bucket into a web server if you set it up correctly. There are no file limits and as long as the aggregate usage stays below reasonable thresholds, you shouldn’t incur significant cost. So I decided to experiment, add a new bucket onto my existing account, upload all the files, and see what kinds of costs it would add in. And so far, I have to say this is a resounding success. Although I certainly need to see more usage data than just the week I’ve had it up so far, so far it looks like this will cost only pennies per day. Lets call it $1.80/month to throw a round number at it. More than reasonable.
What I find most interesting is that Amazon has managed to find several of these “niche” spots in the overall Internet hosting market where they are just the hand’s down market owners, without any real competition to speak of. I doubt Azure or Google’s hosting solutions could meet this need in quite the same way either (though, in fairness, I’ve not investigated it). This is kind of a “near-bare-metal” solution that doesn’t require a server to support the functionality. Good on them. The only thing I’ve heard that approaches this is github’s ability to host content in a repo. Yes, there is ultimately a server behind the goop that holds that together, but it is an adjacent function to the already existing repository/SCM function, with a couple of “bolt-on” widgets to support static content hosting. Not sure if there is any cost on that model, but I also don’t think they would host 500K files for me, either.
Anyway, cool stuff. Go Team AWS!
(Edits to follow for adding in links once I get home and get time to plug them in)