Agile Animal Inc.

A Great Idea

Continuous Integration (CI) is a really great idea.

For a guy like me it means that one week before a major product deadline I can leave early on Friday (as opposed to freaking out and working a 75 hour week.)

But, it doesn’t mean what it used to. Less that two years ago I remember talking about Continuous Integration as “Making sure someone doesn’t break the build.” Reading that statement makes me laugh now. What a ridiculously low bar.

Back then just having a server check out your code and compile it was something. Having it run unit tests the package your software for deployment was amazing.

Times have changed.

What I am now looking for in a Continuous Integration server it is completely different. I want it to grab the code and run Unit, Functional and Acceptance tests on it. Then I want a code coverage report for those tests. On top of that I want a report on standard code metrics that will tell me things like how “DRY” the code is.

To be considered exceptional all of these features need to be tightly integrated and status notifications about updates should be done over Twitter and with a Growl notifier.

Better yet the tests, coverage and metrics should be run pre-commit and anything less than 100% passing and 100% coverage should not be allowed in the repository.

Continuous Analysis & Integration

Really what we are talking about here is not Continuous Integration anymore, it is Continuous Analysis and Integration. Because it really doesn’t matter if the code integrates unless the code is also DRY and thoroughly tested.

My Wishlist

So here is my wish-list for a CI server to use on upcoming projects:

• Integration with GitHub including post commit hooks.
• The ability to run tests, coverage and metrics pre-commit via a gem (think like that Heroku magic for deploying).
• A project template that includes tabs for artifacts from test results, coverage reports and metrics
• Twitter integration.
• A growl notifier.
• Aware of Ruby/Rails for the collection of artifacts necessary for deployment.
• Integrated ability to run capistrano deployment scripts to stage a site if everything passes.
• Tasks run as worker process with customizable credentials and not as the Apache user! (I shouldn’t need to mention this one.)
• All of this out of the box without hours of configuration.

A Final Word or Two

The idea of convention over configuration is beautiful and should be applied to Continuous Integration.

A lot of systems can do anything but everything is hard. That is just dumb.

Good practices should be simple. Customization should be possible.

Post a comment if you have something that you think fits the bill. I think a lot of people would be really interested in hearing about it.

Nick Quaranto has posted a nice article talking about their Continuous Integration setup over at Thoughtbot.

And while he did a great job of explaining what he did, the details of how exactly he did it were left as an exercise to the reader. Here are the details that you are going to need to actually get this thing running.

There are two parts to this adventure, Integrity and Metric_fu.

Integrity

First we need to get Integrity up and running. Install the following gems first:

• Rack (v 0.9.1) sudo gem install rack -v0.9.1
• Sinatra (v 0.9.1) gem install sinatra -v0.9.1
• Thin (v 1.0.0) gem install thin -v 1.0.0

If you are running on OSX then you will need to get require ‘forwardable’ into your environment.rb because it is not there. This is needed for Thin 1.0.0.

Turns out that Sinatra needs to be version 0.9.1 or the notifiers will explode.

The you can then install integrity. Got to the Integrity Website for the details on this. USE THIN. If you don’t you are going to be compiling code as www-data. Create a user account on your server that can access the console and start up thin with this account. Start and stop thin like this:

thin -C thin.yml -R config.ru start
thin -C thin.yml -R config.ru stop

You likely want to have this start with a LaunchDaemon on OSX or something else on Linux. And you likely want to toss this into your Apache config file with a little something like this

############ INTEGRITY PROXY ###########

<VirtualHost *:80>
        ServerName integrity.yourserver.xxx
        ProxyRequests Off
        ProxyPreserveHost On
        <Proxy *>
                Order deny,allow
                Allow from all
        </Proxy>


        <Proxy balancer://mongrelcluster>
                BalancerMember http://127.0.0.1:8910
                BalancerMember http://127.0.0.1:8911
        </Proxy>

        ProxyPass / balancer://mongrelcluster/
        ProxyPassReverse / balancer://mongrelcluster/
        ProxyPreserveHost on

        <Location />
                Order allow,deny
                Allow from all
        </Location>
</VirtualHost>

That should get Integrity going. But, who cares unless it tells that things are breaking. Add a little Twitter notification with by adding the following to your config.ru file:

require "notifier/integrity_twitter"
Integrity::Notifier.register(Integrity::Notifier::IntegrityTwitter)

And then get an integrity-twitter notifier gem installed. I use this one:

git://github.com/hchoroomi/integrity-twitter.git

Great. Now you can get your tests running.

What I’d suggest is to manually check your stuff out and try to have it run on the server. This will help you get all the gems you need for your project installed and let you debug issues.

If things are just refusing to run, ask yourself this question “Who is thin running as and can they run on the shell?”

metric_fu

Nothing worth doing is simple.

Okay with that out of the way we need to get the gem for metric_fu installed and working. First grab qrush’s version: git clone git://github.com/qrush/metric_fu.git

And this is where the fun starts again. Rcov have moved but no one knows yet. So grab it from here: git clone git://github.com/relevance/rcov.git

And then install it by hand:

gem build rcov.gemspec
sudo gem install rcov-0.8.3.4.gem

Before this point you might want to delete the following lines from tasks/metric_fu.rake to stop it from opening the tests once complete:

if MetricFu.report.open_in_browser?
  MetricFu.report.show_in_browser(MetricFu.output_directory)
end

Then you can install the qrush fork of metric_fu:

gem build metric_fu.gemspec 
sudo gem install metric_fu-1.0.3.gem

Bringing it Together

Once this is done you can do something like: rake test:all rake metrics:all

And you should have some tests that run and metrics created. The question remains, how do I get this on the web?

I don’t have the answer to that yet. But I will let you know once I am operational.