I recently felt the need to automate a project's updating without using Git on the production server. The project is hosted on GitHub, so I had GitHub's WebHooks available to me.

Since the Node community is unbelievably awesome at producing libraries for anything you can think of, and because of how easy is makes setting up a "web server", I first checked out what Node projects were available for receiving Github WebHooks.

I chose gith, which is a simple package for responding to WebHooks.

So, how will this work?

When a commit is pushed to GitHub, a POST request will be sent to a URL of our choosing. That URL is set in the "settings" page of any repository. This POST request will include a "payload" variable with information about the repository and the latest commit(s).

Our code will then take action on this - in this case, if the push was to the master branch, it will run a shell script to download the latest zip file of the repo, unzip it and move it to where it needs to be on the server. This avoids using git directly on the server.

Node will create the web server to listen for the WebHook. It executes the shell script which does the heavy lifting.

Let's begin.

Node.js

Assuming Node and NPM are installed (via sudo apt-get install -y nodejs npm on an Ubuntu server), we do the following:

$ cd /path/to/node/app
$ npm install gith

Gith is now installed at /path/to/node/app, so let's write our node script using it. The node script:

$ vim /path/to/node/app/hook.js

// Listen on port 9001
var gith = require('gith').create( 9001 );
// Import execFile, to run our bash script
var execFile = require('child_process').execFile;

gith({
    repo: 'fideloper/example'
}).on( 'all', function( payload ) {
    if( payload.branch === 'master' )
    {
            // Exec a shell script
            execFile('/path/to/hook.sh', function(error, stdout, stderr) {
                    // Log success in some manner
                    console.log( 'exec complete' );
            });
    }
});

Note: This will run the file as the user that starts/owns the Node process. You'll want the Node process to be a user with permission to run these operations. Alternativey, use exec rather than execFile and run the script as another user using su.

Buffer Size

If your shell script outputs a lot of data to stdout, then you may max out Node's "maxBuffer" setting. If this is reached, then the child process is killed! In the example above, this means that the hook.sh script will stop mid-process.

In order to increase the default buffer size limit, you can pass in some options to the execFile function:

// Increase maxBuffer from 200*1024 to 1024*1024
var execOptions = {
	 maxBuffer: 1024 * 1024 // 1mb
}

// Pass execOptions
execFile('/path/to/hook.sh', execOptions, function(error, stdout, stderr) { ... }

Shell Script

We use a shell script to get the files from master and replace the latest files with them.

Note, install unzip if you don't already have it. On Ubuntu, you can run:

$ sudo apt-get install unzip

Now, create the shell script:

$ vim /path/to/hook.sh

#!/bin/bash

# First, get the zip file
cd /path/to/put/zip/file && wget -O projectmaster.zip -q https://github.com/fideloper/example/archive/master.zip

# Second, unzip it, if the zip file exists
if [ -f /path/to/put/zip/file/projectmaster.zip ]; then
    # Unzip the zip file
    unzip -q /path/to/put/zip/file/projectmaster.zip
    
    # Delete zip file
    rm /path/to/put/zip/file/projectmaster.zip

    # Rename project directory to desired name
    mv Project-master somesite.com
    
    # Delete current directory
    rm -rf /var/www/somesite.com
    
    # Replace with new files
    mv somesite.com /var/www/
    
	# Perhaps call any other scripts you need to rebuild assets here
	# or set owner/permissions
	# or confirm that the old site was replaced correctly
fi

Putting it together

So, we have a GitHub Webhook sending POST data to http://somesite.com:9001, as set in GitHub project settings and in our Node script. When that hook is received, we check if it's the master branch. If so, we run the shell script hook.sh.

Lastly, We need to keep the Node script running. If it stops running without us knowing about it, then GitHub WebHook's will do nothing and we'll be running out-of-date code. This is where forever comes in - It will watch a Node process and turn it back on if the Node app errors out or otherwise stops running.

# To install globally, run as a priviledged user (use sudo)
$ sudo npm install -g forever

# Start our Node app … FOREVER!
$ forever start /path/to/node/app/hook.js

Firewall

If you're using a firewall such as iptables (you are using a firewall, right?), you will likely need to open your chosen port to receive web traffic. Here's how you can do it:

# Need to be a priviledged user
$ sudo iptables -I INPUT 4 -p tcp --dport 9001 -j ACCEPT # (I)nserts this rule after the 4th iptables firewall rule

Note that I use -I to insert a new rule in with existing ones. This will add it after the 4th rule. The order is important in iptables rules, since the firewall will stop and apply at the first rule that matches the incoming request.

Adding a new rule instead of inserting one can be added in this manner:

$ iptables -A INPUT -p tcp --dport 9001 -j ACCEPT

Last Note

If you wish to create an Upstart script so you can do fancy things like:

$ sudo service nodehook start

You can create an upstart script such as this example one.

PS: Don't copy and paste that upstart script blindly. Some file path changes will need to be applied. Also, the location of "forever", if installed globally, may not be ./forever as in that example.

Here's a blog post with an example of setting up a simple Upstart for Node.

PPS: Hookshot is an alternative package for listening to GitHub WebHooks.