How to deploy a Node.js app to a private server using Git

This project is about figuring out how to deploy a Node.js web application to a remote server. It is inspired by James Shore: Let's Code: Test Driven Javascript lessons (Chapter 6, Deployment). I choose an alternative way of deployment: the destination is not Heroku but a private server. I choose Git for deployment. I know that Git is not a deployment tool, but this is an experiment: how can be used Git for this purpose.

We have a simple app that - in its present state - can start a Node.js localhost server, and it can serve a homepage and a custom 404 page: (weewikipaint.js). Our aim is to be able to

  • reach the homepage through the web,
  • deployment (that is modify remotely the components of the app)
  • test the deployment process

Reach the homepage through the web

We can run this app on the remote server, but how could we reach the page it serves? The idea is that we can create a proxy server, that redirects the request to the localhost server, and serves its response, so the proxy acts as an intermediary between the client and the localhost server. So we have to

  • configure the proxy server (at subdomain wee.tompascall.com)
  • create custom error page in case the app doesn't run

Configure a basic proxy server with custom error page

I used Nginx to configure the proxy. If you are new at Nginx, you can find a nice guide about configuration here.

When the proxy destination is unavailable (for example you haven't launched the app yet), Nginx gives back a default error message. If you want to give custom message for your visitors, you have to configure a custom error page. There is an entry in Nginx documentation about error page. This article was also very useful when I worked on the config. Here is the configuration in question:

server {  
  listen 80;
  server_name wee.tompascall.com;

  error_page 500 502 503 504 /502.html;

  location = /502.html {
    root /home/tompascall/www/wee/src/server/content; // to-be path of the 502.html page
  }

  location / {
    proxy_pass http://localhost:8080/;
    proxy_set_header Host $host;
    proxy_buffering off;
  }
}

Form the server side environment of the deployment

Before deploying we have to form the server side environment of the deployment. It means, that we have to

  • install Node.js
  • install Git
  • set up Git server
  • launch the app as a daemon

Install Node.js

We have to install Node.js on the remote server to able to run our app. The installation is quite straightforward. You can download the source code or the binaries from here.

Install Git

Git is an open source distributed version control system. It will help us to manage deployment. We have to install it on the server. As regards installation, checkout this guide. If you haven't use Git yet, here is a post about some Git basics.

Set up Git server

How could we want to use Git for deployment? The idea is that

  1. develop our app with Git locally
  2. set up Git server
    • clone our local repository locally as a bare repository
    • copy the bare repository to the remote server
    • set the bare repository as a remote repo
  3. clone a repository from the bare repo to the server (so as to have a repo with working directory)
  4. update the bare repository by pushing the locally committed changes.

It is not a simple solution, there may be better ways to use Git for deployment. I tried to use the bare repository alone, and set the work-tree variable in the git config file to an other directory, but it didn't work: after the first checkout, when I updated the bare repo and tried to check out the changes, it seemed that everything ok but nothing happened in the work-tree directory. Then I decided to clone the bare repository, so the cloned repo has a working directory, and it can be updated by a git pull (this pull the changes from the bare repo as its origin remote repository).

Update: I found a simpler solution of setting up the Git server. We don't need to clone the bare repository on the server. It is enough to set the value of GIT_WORK_TREE to another directory than the repository, and you have to check out using the -f option to update the working directory correctly (see the link above).

The big advantage of this solution is that you can fix the code on the server too, commit these changes, update the bare repo, and then you can simply update the local repository. Another advantage is that you can clone the project from the bare repo, and more participants can take part in the development if they have access to the server.

So, let's set up Git server. We have a great online source, the Git Pro book (written by Scott Chacon), that helps us. We are now interested in chapter 4.2, Getting Git on the server. It says that in order to initially set up any Git server, you have to export an existing repository into a new bare repository - a repository that does not contain a working directory:

$ git clone --bare path_to_local_project_directory wee.git

It makes a copy of the Git directory data in wee.git directory. Now you have to copy this directory to the server (assuming that you have SSH access to your server):

$ scp -r wee.git username@your_domain:/path_for_wee.git

Now you can push the locally committed changes into that repository by adding it as a remote (called here ghost-wee) and pushing up a branch:

$ git remote add ghost-wee username@your_domain:path_of_wee.git  
$ git push ghost-wee

We have to clone the bare repo on the server:

git clone wee.git /path_for_working_directory

And that's all, we have deployed our app to the server.

Launch the app as a daemon

As the app runs a localhost server, it is worth considering that we should launch it as a daemon via an init script. In this way the app is run automatically when the system is booting up, and we can stop and restart it whenever we want to. To do this we have to

  • create the init srcipt
  • copy the script to /etc/init.d/
  • create the necessary links with update-rc.d
  • run the script

I created the script by modifying this script for Ghost blogging platform. You can see my script here. After copying the script to /etc/init.d/ (you need root for this), we can launch the app like this:

$ service manage_wee start

To run automatically the script when booting, we used update-rc.d:

$ sudo update-rc.d manage_wee defaults 98 02

Modify remotely the components of the app

The process could look like this:

  • test code locally
  • commit changes
  • git push to the bare repository
  • stop app
  • update the working directory
  • start app

Git push to the remote bare repository

After testing the changes locally and committing, you have to update the bare repository:

$ git push ghost-wee

Start and stop the app remotely

With the daemon above it is very easy to manage the app remotely, if you have ssh access to the server. For example we can stop the daemon like this:

$ ssh tompascall@tompascall.com "service manage_wee stop"

And we can start the service remotely in the same vein with the start argument.

Automate deployment

We can create a small script which does the deployment process shown above instead of us:

#!/usr/bin/env bash

# This script (deploy.sh) manages deployment of WeeWikiPaint project
# You have to run it from the working directory after the code had been tested locally and changes had been committed

# Its work is
# - git push to the remote bare repository
# - stop app on the server
# - update the working directory on the server
# - start app on the server

git push ghost-wee deploy && ssh tompascall@tompascall.com "bash -s" < ./script/update_repo.sh

And the helper script looks like the following:

#!/usr/bin/env bash

# This script (update_repo.sh) is a helping script for deploy.sh
# It 
# - stops app on the server
# - updates the working directory on the server
# - starts app on the server 

service manage_wee stop && cd /home/tompascall/www/wee/ && \  
git pull && service manage_wee start

Ok, so we have the server side environment for the deployment, and we can start and stop the app remotely. After testing the code locally, and committing the changes, a simple script does the deployment. Now we would like to test the deployment with Nodeunit.

Test the deployment

We will test the deployment by by seeking a marker in the homepage (and the 404 page) that the app serves:

(function(){

// test remote server
// get a page
// confirm we got a marker

var http = require("http");

var TEST_PORT = "8080";  
var BASE_URL = "http://wee.tompascall.com";

exports.test_canGetHomePage = function(test){  
  var homePageMarker = "WeeWikiPaint home page on remote server";
  checkMarker(BASE_URL, homePageMarker, function(foundMarker){
    test.ok(foundMarker, "homepage should have contained " + homePageMarker);
    test.done();      
  });
};

exports.test_canGet404Page = function(test){  
  var notFoundMarker = "WeeWikiPaint 404 page on remote server";
  checkMarker(BASE_URL + "/nonexistant.html", notFoundMarker, function(foundMarker){
    test.ok(foundMarker, "404 page should have contained " + notFoundMarker);
    test.done();      
  });
};


function checkMarker(url, marker, callback) {  
    var request = http.get(url);
    request.on("response", function(response) {
      var responseData = "";
      response.setEncoding("utf8");

      response.on("data", function(chunk) {
        responseData += chunk;
      });
      response.on("end", function() {
        var foundMarker = responseData.indexOf(marker) !== -1;
        callback(foundMarker);        
      });
    });
  }
}());

Presently the page has not contain the marker ("WeeWikiPaint home page on remote server"), so our test will fail:

$ grunt  
Running "jshint:files" (jshint) task  
>> 8 files lint free.

Running "nodeunit:files" (nodeunit) task

_deploy_test.js  
✖ test_canGetHomePage

Assertion Message: homepage should have contained WeeWikiPaint home page on remote server  
AssertionError: homepage should have contained WeeWikiPaint home page on remote server  
    at Object.ok (d:\dev\letscodejavascript\node_modules\grunt-contrib-nodeunit\node_modules\nodeuni
t\lib\types.js:83:39)  
    at d:\dev\letscodejavascript\src\_deploy_test.js:16:8
    at IncomingMessage. anonymous (d:\dev\letscodejavascript\src\_deploy_test.js:41:5)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

✖ test_canGet404Page

Assertion Message: 404 page should have contained WeeWikiPaint 404 page on remote server  
AssertionError: 404 page should have contained WeeWikiPaint 404 page on remote server  
    at Object.ok (d:\dev\letscodejavascript\node_modules\grunt-contrib-nodeunit\node_modules\nodeuni
t\lib\types.js:83:39)  
    at d:\dev\letscodejavascript\src\_deploy_test.js:24:8
    at IncomingMessage. anonymous (d:\dev\letscodejavascript\src\_deploy_test.js:41:5)
    at IncomingMessage.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)

We put the marker to the homepage (and the 404 page) as a html comment, commit the changes, and run deploy.sh. Now let's run the test again:

$ grunt  
Running "jshint:files" (jshint) task  
>> 8 files lint free.

Running "nodeunit:files" (nodeunit) task

_deploy_test.js  
✔ test_canGetHomePage
✔ test_canGet404Page

The test has passed now. It means that our deployment method works, and we are happy :)

Summary

Our aim was to give a way of deployment of a Node.js app, using a remote Git server. Because our app works a localhost server, the first task was to make reachable the homepage served by this localhost server. We solved it by configuring a proxy server. The second task was to create the server side environment of deploying. We set up a Git server, so we can update the project components on the server by updating the repository, using the power of Git. We had to figure out, how can be launched the app along the boot process, and how can we start and stop it remotely so as the updating does not confuse the working of the app. The solution was to launch a daemon as an init script. Then we made a script, that automates the deployment process. Finally, we tested the deployment using Grunt and Nodeunit by writing a test, that seeks a marker in the source of the homepage. We put this marker in the source, and ran the deploying script. After the deployment the test has passed.



comments powered by Disqus