Third Prestige We make the web work.

How 7 Lines of Ruby Will Speed Up Your Heroku Deploys 10x

Nathaniel Jones · June 14, 2013

We each know the pain of forgetting to check in one file at 3am on a Tuesday. We deploy to staging so we can email the client a progress report, only to bring down the app because of one oversight. We quickly rollback, but now the fun begins. We git push heroku master once again, only to wait, and wait, and wait, while rake assets:precompile generates an identical asset manifest to the one just generated 10 minutes ago.

What if we could reliably speed up our deploys by a factor of 10? If we deploy ten times a week and save 3 minutes per deploy, we've added 2 hours a month of productivity.

Solution

How would we go about this? By attacking the largest bottleneck in deploying Rails apps on Heroku: precompiling assets.

Often our front-end code is unchanged, and we just want to tweak a rake task or fix a minor bug. This pull request sidesteps the precompile task on such deploys by seamlessly loading in the precompiled assets from Heroku's cache directory.

We have been reliably using this technique on several production apps for about 8 months now, including at Huckberry.com, with about a dozen deploys a week. We've dropped junior developers into apps with this buildpack and it has just worked, no explanation required. And it's saved us many hours, our clients a lot of money, and made our Heroku experience much more enjoyable overall.

heroku config:add BUILDPACK_URL=https://github.com/nthj/heroku-buildpack-ruby.git#precompile-optimizations

How it Works

Heroku offers us a cache folder that lives across multiple deploys, so actually caching the compiled assets is fairly straightforward.     

cache_store(“public/assets”)
Then, to pull it in again later:

cache_load(“public/assets”)

Cache Invalidation Is A Hard Problem

 There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

So now we have to decide when our cache is invalidated, when we’ve changed our assets and we need to start from scratch.   This all happens in the appropriately named precompiled_assets_are_cached?.  Let’s break it down.

uncompiled_cache_directories.all? { |directory| run("diff #{directory} #{cache_base + directory} --recursive").split("\n").length.zero? }

So this makes sense, right?  We have an list of directories where our uncompiled assets live in, and we recursively compare them with the cached versions of those directories.  If any files have changed, we know our cache is invalid.

File.exist?("#{cache_base}/public/assets/.version") &&    File.read("#{cache_base}/public/assets/.version") == asset_configuration_hash &&

This is less intuitive and handles more of an outlier case.  Rails allows you to specify some configuration options for the asset pipeline.  Examples include:

config.assets.precompile += %w(ie.css)
or

    config.assets.paths += %w(app/views/themes/*/assets)

So if these settings change, we need to kick off the precompile task again and make sure we bring them over into our public/assets folder. right, so Let’s see what we can do about that:

@asset_configuration_hash ||= run("grep -r 'config.assets' . | grep -v './#{cache_base}' | grep -v './tmp' | grep -v './vendor/bundler' | sed -e 's/ *//g;' | shasum")[0...-2].strip

What? Alright, let’s work through the crazy pipe nonsense here.  We’re looking for any lines of code that reference `config.assets`.

  • We don’t want to match anything in our cache directory (from previous deploys), so we strip that out.
  • We don’t want to match anything in the tmp/ folder.
  • We don’t want to match anything in vendor since our Gemfile.lock check should catch changes there.
  • Also, let’s use sed to strip out whitespace, since whitespace doesn’t affect the results.
  • Finally, let’s pipe all this in to shasum and get a nice identifier that’s easy to store.

Caching assets for next time

Of course, we’ll want to actually cache the compiled assets (public/assets).  But we also need to store the original files, so we can compare them with the new files on the next deploy.  And of course, we need that configuration hash, which we throw into a “.version” file.

Faster deploys are greater than the sum of the time saved

“Give me six hours to chop down a tree and I will spend the first four sharpening the ax.” 

— Abraham Lincoln

As developers, we all know the magic of getting into the zone.  We layer in feature after feature, crush bug after bug, and everything is smooth sailing—like a fairy tale.  But, like in all fairy tales, the spell is easily broken: one ringing phone, one “where is the Brigham report?”-type context switch, and the magic is lost.  Lengthy deploys are no help. We have a 5-minute wait, so we go browse Hacker News. Then, we’re sucked in for 20 minutes and forget where we were.

Staying in the zone is worth far more than just the number of minutes saved.

I hope this helps you stay in the zone. I’d love to hear your feedback, how you’re using it, ideas for improvement, or just a “hello!”—drop me an email anytime to nj@thirdprestige.com.

Do you like this?

Help us spread the word: Tweet about this or +1 our Heroku pull request on GitHub.



Authored_nathaniel-32

Sometimes called a “Unicorn” in the Austin tech community for his broad expertise, Nathaniel is equally at home identifying key business goals, optimizing a website's speed tenfold, or polishing the final UI details that make software shine.

Also he may or may not have flown a 2-seater plane without a pilot's license while down under, so, there's that.