Archive for July, 2012

Growing Pains of Going Pro

Since the initial release of the Hopville 2.0 beta will have a stripped-down feature set – and I mean, a really stripped down feature set – I thought it might be good to share some of the details of what 2.0 represents under the covers.

Hopville grew to 100,000 recipes as one of several side projects, without having serious focus given to it.  With Hopville 2.0 I’m beginning to give it real focus, getting it ready to grow to its next milestone of 1,000,000 recipes. First step was to overhaul its infrastructure.  Here’s what happened this week:

  • New web host: Since its inception, Hopville has run on an affordable hosting account through Site5.  While they’ve served Hopville well, the nature of that hosting service (cheap hosting on shared servers without system-level access) is too limited for a site operating at professional capacity. The tallest hurdle jumped this week was the migration from their service to cloud servers hosted by Rackspace, with whom I’ve had great experience in a few startups where I’ve worked.
  • New web framework version: Hopville was most recently running Rails 2.3, but the version released this week represents an upgrade to Rails 3.2.
  • New codebase: Due to some delicious tech tweaks introduced into Rails as of its 3.1 version, plus a completely new approach to how Beer Calculus works, I took the opportunity to overhaul the codebase, rewriting nearly everything from scratch.  (Might’ve bit off more than I can chew, frankly, as this project ran far longer than my original estimates.)
  • New database: The site still runs on the open source MySQL database, but for 2.0 it has been restructured and expanded.  Where it took about 20 tables to run the old Hopville, it’s taking about 30 (so far) to run the new service.  One side effect is that folks may have to reset their passwords as the new server and framework use stronger encryption for passwords.  I’ve also isolated the database onto its own server which will help the site to scale and eventually facilitate things like a public API.  You’ll start seeing the reasoning behind the restructuring (in the form of new features) soon…
  • New image service: One of the embarrassing historical aspects of the old site was how images were handled.  I thought to myself when Hopville launched, “since nobody is using this site, I don’t have to optimize its images in any way – it doesn’t matter how huge and slow the pages are with zero traffic” Then I proceeded to never upgrade the service after the site grew to have traffic.  I let the browser resize images into thumbnails, etc – if you’re a web developer you know that this is an absolute no-no.  Hopville 2 will have a lot more image functionality (eventually).  Accordingly, with this launch images are finally processed and handled correctly, as well as being stored on Amazon’s cloud, which should speed up some pages quite a bit.

July will be a bit bumpy since Hopville has no QA department and does testing in the wild.  I’ve also lucked into an unfortunately-timed but also all-expenses-paid vacation to Alaska, so as of Friday I’ll be mostly offline for a whole week.  Sorry folks!  Y’all will notice a distinct lack of features during the initial beta testing, as I removed lots of features to focus on the systems migration.  But you’ll just have to trust me that I can see far into the future that was enabled by the Great Migration of Hopville 2.0.  And what I can see out there on the horizon is damn exciting.  The summer of 2012 is the awkward puberty that Hopville has to transition through on its way to becoming an adult.  Stay tuned…

Final recipe count for Hopville 1.0: 118,222


Leave a Comment

Hopville 2.0 Migration (Week of July 2, 2012)

The 2.0 project was a major undertaking, involving a complete rewrite of the application and a restructuring of the database powering the site.  (Don’t worry, all your data will be preserved.)  It’s also the time when the site graduates from a cheap hosting provider to a professional-grade, not-so-cheap one.

So, the migration is quite a juggling act.  With so many technical hurdles to worry about, the website feature set has been cut back dramatically to launch a pared-down, more stable beta site.  The 2.0 beta will roll live in a minimalist state and then features will begin to reappear (along with the inevitable bug fixes) over the next few days/weeks as things stabilize and get tested in the wild.  Hopville will continue to be 100% free until all the major features planned for the 2.0 release are live, after which it will transition to the “freemium” model as announced previously.  At this point the full feature set is at least a few weeks out, maybe a few months.

Meanwhile, Hopville has to move house.  Starting tonight I’ll post updates about outages and major bugs below for those who’d like to follow along.

  • [Latest] July 4 @ 5:30 PM: The stripped-down site is finally live!
  • July 4 @ 5 PM: Setting up DB replication, which helps the site stay fast and ensures that there are multiple copies of the data in case of catastrophic failure. If I waited until after the site launched, I’d have to disable it again to start the replication process, so I’m taking care of it now.
  • July 4 @ 3 PM: Looking good to switch on the minimalist initial version of the new site later today.  Currently working to migrate all of the brewer profile images to the new service.
  • July 3 @ 9:30 PM: Okay, I think I have ironed out everything with the data migration, my suite of test recipes is looking solid, so I’m now prepping to release the initial minimalist 2.0 site tomorrow. I have the whole day off tomorrow…but I’m being extra careful about the migration part of this process.  Since the data model is changing in a few ways, there is no turning back once the new site goes live – that’s why I’ve kept it dark during the transition.  For safekeeping, I will continue to run the “final state” of old website under a private domain so that I can use it to QA any issues and run incremental data fixes as necessary.
  • July 2 @ 9 PM: At this point I’m going to run another complete import to help ensure a clean, finalized database before releasing anything to the brewing public.  I’ll need to test that tomorrow once the fresh build is complete, but I won’t have the whole day free as I did today. Sorry folks, but  it’s looking like the site might remain in the dark through much if not all of Tuesday July 3.
  • July 2 @ 6 PM: Working out some kinks in the migration script for recipes, making sure all customizations saved under the old codebase are preserved in the new one.
  • July 2 @ 4 PM: Data migration looks okay but I’ll continue the QA process to make sure recipes transfer successfully. Also migrating brewer profile images, etc.  A few hours of work still remain.
  • July 2 @ 7:30 AM: All brewer information has been fully migrated. Next up: recipes!  That’s the juicy one – it might take several hours to migrate all the recipe data and QA it before switching the site back on.
  • July 2 @ 6 AM: The migration script caught a couple data validation problems overnight. That’s good, because it’s enforcing stronger data integrity for version 2.0.  But I’ll have to spend this morning running the final migration incrementally so I can babysit each step to make sure everything gets carried over correctly and completely.
  • July 2 @ 1:15 AM: Migrating DB from 1.0 to 2.0. This is a long-running process so I’ll let it complete overnight and then start work again in the morning. No further updates this evening, folks!
  • July 2 @ 12:30 AM: Creating final, redundant backups of Hopville’s 1.0 database.
  • July 2 @ Midnight: has been disabled.  Off we go…
  • July 1 @ 10 PM: Site migration scheduled to begin at midnight PDT tonight.

Comments (6)