Rube Goldberg contraptions with RabbitMQ

I’ve just finished building and deploying this website. It uses jekyll to render the content, and the content author uses git to push the changes up to the hosting machine. From there, a nice little chain of programs arranges for the site to be rebuilt on the server and made live:

  • A git post-receive hook uses curl to HTTP POST an empty message into a RabbitMQ exchange via the RabbitHub plugin.

  • D. J. Bernstein’s daemontools supervises an instance of amqp-consume, which connects to a queue bound to the exchange the post-receive hook delivers into, and whenever a message is received, invokes a shell-script. The command-line for invoking amqp-consume is roughly

    amqp-consume \
        -s localhost \
        --username=... --password=... \
        -e exchangename \
        -A \
  • The shell-script invoked for every message from RabbitMQ checks out a fresh copy of the website, compiles it, and deploys the resulting static HTML into the correct location on the file system for Apache to pick up.

  • I’m also monitoring the RabbitMQ exchange using the rabbitmq-xmpp plugin talking to my desktop XMPP client, Adium, so whenever anyone does a git push, I get a message appearing in my IM client from exchangename@my.rabbitmq.hostname letting me know a new version of the site has just gone live.

Comments (closed)
alexis richardson 05:40, 25 May 2010

This is indeed Neat.