Integrity | Continuous Integration server

As soon as you push your commits, Integrity builds your code, run your tests and makes sure everything works fine. It then reports the build status using various notifiers back to you and your team so everyone is on the same page and problems can be fixed right away.

Check out Integrity watching itself:

Screenshot

Read more about about Continuous Integration on Martin Fowler’s website and Wikipedia.

Installation

Make sure your system meet these prerequisites:

Ruby >= 1.8.7 (1.8.7, 1.9.2, 1.9.3 and 2.0.0 should work)
RubyGems >= 1.3.5
git >= 1.6

Then grab Integrity via git, install its dependencies and create the database:

$ gem install bundler
$ git clone git://github.com/integrity/integrity
$ cd integrity
$ git checkout -b deploy v26
$ bundle install
$ bundle exec rake db

To give it a quick try before going any further, run bundle exec rackup and navigate to http://0.0.0.0:9292.

Upgrade:

$ git fetch origin
$ git merge v26

Configuration

Integrity is configured via the init.rb file using Ruby.

Integrity.configure do |c|
  c.database                    = "sqlite3:integrity.db"
  c.directory                   = "builds"
  c.base_url                    = "http://ci.example.org"
  c.log                         = "integrity.log"
  c.builder                     = :threaded, 5
  c.build_all                   = true
  c.project_default_build_count = 10
end

Basic settings

database

Any valid database URI supported by DataMapper.

sqlite3:integrity.db
mysql://user:password@localhost/integrity
postgres://user:password@localhost/integrity

Important

You need to install the appropriate data_objects adapter as well. The SQLite3 adapter (do_sqlite3) is installed by default.

directory: This is where your projects' code will be checked out. Make sure it is writable by the user who runs Integrity.
base_url: Absolute URL to your Integrity instance, without a trailing slash. This setting is optional but is required to have proper links in notifications.
log: Path to the Integrity log file; mostly useful for troubleshooting
build_all!: Tells Integrity to build every single commit it is notified about. Only builds the newest otherwise.
auto_branch!: Say the project Integrity is tracking the master branch, and I push my build-duration topic branch to GitHub, Integrity will create and build a new project named Integrity (build-duration) using the same build command and notifiers.
trim_branches: When Integrity is set to auto branch, it is possible for a large number of stale branches to stick around which no longer exist. Turning this on will automatically detect when a branch is deleted via the GitHub post-receive hook and delete the build history for the branch.
project_default_build_count: How many builds to initially show on project pages. nil, which is the default, means show all builds.
HTTP authentication: If both username and password settings are set then only the logged-in users can administer the projects and see the private ones.

c.username = "admin"
c.password = "password"

To protect the whole Integrity instance, set ADMIN_USERNAME and ADMIN_PASSWORD environment variables before starting Integrity:

export ADMIN_USERNAME=admin ADMIN_PASSWORD=secret

On Heroku:

heroku config:add ADMIN_USERNAME=admin ADMIN_PASSWORD=secret

checkout_proc: Set to a Proc instance to override default checkout mechanism. For example, to use git-cachecow to cache repositories locally:

c.checkout_proc = Proc.new do |runner, repo_uri, branch, sha1, target_directory|
  runner.run! "git scclone #{repo_uri} #{target_directory} #{sha1}"
end

Building private repositories

Integrity will use keys configured in its environment for repository access.

The easiest option is to switch to the Unix user that Integrity runs under and create an SSH key for this user:

ssh-keygen

Save the key to the default location. Add the public key to the list of allowed keys in the repository you wish to build. Clone the repository manually to check that it works and trust the server’s host key if necessary.

An alternative is to configure GIT_SSH environment variable. Integrity ships with a sample git_ssh file in doc directory. Path to this file should be placed in the GIT_SSH environment variable. You can do this on Heroku by editing init.rb to include the following:

ENV['GIT_SSH'] = File.join(File.dirname(__FILE__), 'doc/git_ssh')

Then, specify a private key that is allowed by repository:

heroku config:add GIT_PRIVATE_KEY="----contents of private key
second line of private key
make sure to use double quotes
to allow multi-line config var value
end of private key-----------"

Warning

Storing secret data such as private keys in environment is inherently insecure. If you do this, restrict access to your Integrity installation to trusted users only.

Automating the builds

Integrity provides two HTTP endpoints to trigger new builds:

POST /:project/builds: Causes Integrity to fetch the HEAD of the remote repository and build it. Note that HTTP authentication applies to it if set. This endpoint can’t be disabled.
POST /github/:token: Supports GitHub’s Post-Receive Hook. Enable it:

c.github_token = "TOKEN"

Choosing a builder

Integrity ships with three ways of building your codes in the background.

Threaded

The threaded builder pushes the build job to an in-memory queue and processes them as soon as possible. It relies on Ruby’s thread so it doesn’t have any dependency nor requires to run other processes.

c.builder = :threaded, 5

The second argument is the size of the thread pool.

Note	The threaded builder cannot be used with Passenger.

Delayed::Job

The dj builder queues up the builds into an SQL database using Delayed::Job. To use it, install its dependencies by uncommenting the relevant lines in the Gemfile and run bundle lock && bundle install.

c.builder = :dj, { :adapter => "sqlite3", :database => "dj.db" }

The second argument must be a valid ActiveRecord connection hash.

Run a worker with rake jobs:work. See Delayed::Job’s documentation for more details.

Resque

The resque builder, as the name implies, uses resque to store jobs.

c.builder = :resque

Notification

After a build status is available, you want to know it immediately. Integrity supports a number of notification mechanisms:

AMQP
Campfire
Flowdock
Co-op
Email
HTTP
Amazon SES
Shell
TCP

To enable and use a notifier with your projects:

Open the Gemfile, uncomment the relevant lines and run bundle lock && bundle install to install the notifier’s dependencies.

Edit the init.rb file to require it. Example:

require "integrity"
# You need to add this line:
require "integrity/notifier/email"

Restart Integrity and go to a project settings screen where you can enable and configure the notifier.

HTTP Notifier

This notifier sends an HTTP POST request to the specified URL. The payload is sent as HTML form fields, with application/x-www-form-urlencoded content type. The fields are as follows:

name: project name
status: build status string (success/failed)
url: url to build page in Integrity
repo: URI specified for repository in project configuration
branch: branch specified in project configuration
commit: SHA1 of the commit that was built
author: Commit’s author name (without email address)
message: Commit message, prefixed with branch name

Example request body:

name=My+Test+Project&status=success&url=http%3A%2F%2Fwww.example.com%2Fmy-test-project%2Fbuilds%2F1&repo=%2Ftmp%2Fi%2Ftmp%2Fmy_test_project&branch=master&commit=d669c99c26419806c80326e1b09eda378e65491d&author=John+Doe&message=master%3A+This+commit+will+work

Deployment

Phusion Passenger

Please see Passenger’s documentation for Apache and Nginx.

Note	Currently ThreadedBuilder does not work with Passenger (issue #156). Please use DelayedBuilder or ResqueBuilder.

Thin

Uncomment the thin line in the Gemfile
Run bundle install to install Thin

$ $EDITOR doc/thin.yml
$ thin -C doc/thin.yml start

Unicorn

Uncomment the unicorn line in the Gemfile
Run bundle install to install Unicorn

$ $EDITOR doc/unicorn.rb
$ unicorn -c doc/unicorn.rb -D

Reverse Proxy

You will want to run a reverse proxy such as Nginx or Apache in front of Thin or Unicorn servers.

Nginx

Nginx can be configured as follows:

http {
  upstream builder-integrityapp-com {
  # thin or unicorn - first instance
  server 127.0.0.1:8910;
  # thin only - specify all remaining instances
  server 127.0.0.1:8911;
}

server {
  server_name builder.integrityapp.com;
  location / {
    proxy_pass http://builder-integrityapp-com;
  }
}

Apache

Apache can be configured as follows:

<VirtualHost *:80>
        ServerName ci.example.org
        ServerAlias ci

        ProxyRequests Off
        ProxyPreserveHost On

        <Proxy *>
                Order allow,deny
                Allow from all
        </Proxy>

        ProxyPass / http://127.0.0.1:8910/
        ProxyPassReverse / http://127.0.0.1:8910/

        ProxyPassReverse / http://127.0.0.1:8911/
</VirtualHost>

If you wish Apache to serve static files:

<VirtualHost *:80>
        ServerName ci.example.org
        ServerAlias ci

        ProxyRequests Off
        ProxyPreserveHost On

        <Proxy *>
                Order allow,deny
                Allow from all
        </Proxy>

        DocumentRoot /path/to/integrity/lib/app/public
        RewriteEngine On
        RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
        RewriteRule ^/(.*)$ http://localhost:8910/$1 [P]
</VirtualHost>

<Directory /path/to/integrity/lib/app/public>
        Order allow,deny
        Allow from all
        Options None
        AllowOverride None

        # Apache 2.4+
        AuthType None
        Require all granted
</Directory>

FAQ

Does it support Subversion or any other SCM?

Integrity only works with git. However, git can mirror other SCMs, for example Subversion via git-svn, and Integrity will work with such mirrored repositories. It is up to you to keep the mirrored repositories up to date with your primary repositories.

But does it work with <insert tech here>?

Absolutely! As long as your build process can be run from an UNIX-y environment and that it returns a zero status code for success and non-zero for failure, then Integrity works for you.

Support / Contributing

You can get in touch via IRC at #integrity on freenode. If no one happens to be around, you can ask our mailing list at integrity@librelist.com. (Archives)

If you find a bug, or want to give us a feature request, log it into our bug tracker.

To start hacking, grab the code from our git repository at git://github.com/integrity/integrity.git and setup the dependencies with bundle install && bundle lock. Finally, hack and bundle exec rake as usual ;-)

Once you’re done, make sure your changes are rebased on on top of the master branch and open a new ticket in our bug tracker to let us know where we can pull from.

Documentation

To build HTML documentation, run rake html. This is what goes on the Integrity website (http://integrity.github.com).

Dependencies for HTML documentation generation: