Test a Gruntfile using Node.js child_process module

This project is based on James Shore: Let's Code: Test Driven Javascript lessons (Chapter 5, Smoke Test). In these lessons you can learn about smoke test, and how to launch a child process in your test. We are not going to make a smoke test in this post, just focus our attention on Node.js child_process module to test a Gruntfile.

The structure of the project

We will use Grunt for build automation, and Nodeunit plugin for the test. If you haven't used Grunt or Nodeunit before, I suggest checking out this blog post about basic Nodeunit tests. We also use Jshint plugin to check our code for javascript syntax errors. So we have a Gruntfile that first provides for Jshint-ing our javascript code, then runs our Nodeunit test. In the test we are to launch Grunt as a child process in a test directory, and we test the Grunfile located in this directory. So we want to tell Node.js to run Grunt as a child process with optional parameters, in a given directory, and we want to pipe the output stream of the child process into the parent process to check that.

Launching a child process

There are more possibilities in the Node.js child_process module to launch child processes:

How can communicate child process with its parent process? As the Node.js documentation says

child processes always have three streams associated with them: child.stdin, child.stdout, and child.stderr. These may be shared with the stdio streams of the parent process, or they may be separate stream objects which can be piped to and from.

The main difference between spawn()/fork() and exec()/execFile() is that spawn() uses streams, but exec() uses buffers of a given size (default: 200k) to communicate with the parent process. An other difference is that spawn() manages only executable files, exec() can manage .bat files too. As we will see later it is quite a remarkable difference if we want to care about portability.

The initial test functions

In this project we would like to work with streams, so the child_process.spawn(command, [args], [options]) method or child_process.fork(modulePath, [args], [options]) may come into play.

As you can see above, the spawn() method requires a command parameter namely the process you want to launch (this will be grunt in our project), you can give arguments for the process, and an options object. In this object we set the cwd property (current working directory of the child process). cwd allows you to specify the working directory from which the process is spawned. We also set stdio property: the pipe value creates a pipe between the child process and the parent process. For more details please check out the Node.js documentation.

In the test we have a launchGruntWithParameter() function, which sets the the cwd and stdio property and launches grunt as a child process with our parameters. There is also a helper function called streamReader(), that gets the output stream from the child process, and later it will provide the possibility to check this stream. So our initial helper functions are something like this:

// child_process_test.js  
var child_process = require("child_process");

function launchGruntWithParameter(gruntParam, callback){  
    var options = {
        stdio: "pipe",
        cwd: "src/test_project/"
    };
    var spawnFunction = child_process.spawn("grunt", gruntParam, options);
    streamReader(spawnFunction, function(){
        callback();
    });
}

function streamReader(child, callback){  
    var processLog = "";    
    var streamStart = "\n\n:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::\n";        
    var streamEnd = "------------------------ CHILD-PROCESS-STREAM-END -----------------------\n\n";        

    child.stdout.setEncoding("utf8");       

    child.stdout.on("data", function(chunk){
        processLog += chunk;
    });

    child.stdout.on("end", function(){
        console.log(streamStart + processLog + streamEnd);
        callback();
    });
}

Check the default task

As regards the test-project, at the beginning the Gruntfile is quite empty:

// child process Gruntfile.js  
module.exports = function(grunt) {

};

Our first test is to check the default task:

// child_process_test.js  
exports.test_defaultTask = function(test){  
    launchGruntWithParameter([], function(){
        test.done();
    });
};

It effects that the child process launch without parameter (it means for Grunt that you want to run the default task). As the Gruntfile is empty (so that it has no default task yet), we assume that child process grunt fails, and it does:

:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::  
Warning: Task "default" not found. Use --force to continue.

Aborted due to warnings.  
------------------------ CHILD-PROCESS-STREAM-END -----------------------

Let's write a simple default task:

// child process Gruntfile.js  
module.exports = function(grunt) {

    grunt.registerTask("emptyTask", function(){

    });

    grunt.registerTask("default", ["emptyTask"]);
};

So we have a default "task", grunt runs without error. Here is the output of the child process:

:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::  
Running "emptyTask" task

Done, without errors.  
------------------------ CHILD-PROCESS-STREAM-END -----------------------

And we are happy. But wait. I tried the same code in Windows environment, and it just crashed. What the heck is going on? I figured out, that the launchGruntWithParameter() function works with child_process.exec():

// child_process_test.js  
function launchGruntWithParameter(gruntParam, callback){  
    var options = {
        stdio: "pipe",
        cwd: "src/test_project/"
    };

    var execFunction = child_process.exec("grunt " + gruntParam, options);

    streamReader(execFunction, function(){
        callback();
    });
}

But why child_process.spawn() doesn't work? After some searching I realized the following:

In Windows environment, grunt is a batch file, it launches 'node' with the grunt.js module as a parameter. In Mac/Linux environment grunt is an executable file (that also launches node with grunt.js). child_process.spawn() doesn't handle batch files, this is why our attempts failed. I tweaked the code, and tried the following:

// child_process_test.js  
exports.test_defaultTask = function(test){  
    launchGruntWithParameter(["../../node_modules/grunt-cli/bin/grunt"], function(){
        test.done();
    });
};

function launchGruntWithParameter(gruntParam, callback){  
    var options = {
        stdio: "pipe",
        cwd: "src/test_project/"
    };
    var spawnFunction = child_process.spawn("node", gruntParam, options);
    streamReader(spawnFunction, function(){
        callback();
    }); 
}

The path of grunt is quite weird. We need the ../../ at the beginning of the path, because the value of cwd (the current working directory) of options object is added to the path, and we have to get out of it to find node_modules directory.

It works, fine. But if the grunt is a node module, we should rather use child_process.fork() method, because it is designed for launch node modules. So the final form of the helper functions and the first test:

// child_process_test.js  
var child_process = require("child_process");

exports.test_defaultTask = function(test){  
    launchGruntWithParameter([], function(){
        test.done();
    });
};

function launchGruntWithParameter(gruntParam, callback){  
    var GRUNT_PATH = "../../node_modules/grunt-cli/bin/grunt";
    var options = {
        silent: true, //stdin, stdout, and stderr of the child will be piped to the parent
        cwd: "src/test_project/"
    };
    var forkFunction = child_process.fork(GRUNT_PATH, gruntParam, options);

    streamReader(forkFunction, function(){
        callback();
    });
}

function streamReader(child, callback){  
    var processLog = "";    
    var errLog = "";
    var streamStart = "\n\n:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::\n";        
    var streamEnd = "------------------------ CHILD-PROCESS-STREAM-END -----------------------\n\n";        

    child.stdout.on("data", function(chunk){
        processLog += chunk;
    });

    child.stderr.on("data", function(chunk){
        errLog += chunk;
    });

    child.stdout.on("end", function(){
        console.log(streamStart + processLog + streamEnd);
        console.log(errLog);
        callback();
    });
}

But how could our test know that the given task failed or it didn't?

Getting feedback from the child process

One solution is that we make the child process send a message to the console, and then we can check the output stream of the child process. For example if the Grunt task runs without error, the Grunt task sends a "task is ok" message to the console, and in the test we check if the output stream contains this test-message.

First we need a helper function in the test that can check the output stream if it contains a given string:

// child_process_test.js  
function streamChecker(processLog, taskOkMessage){  
    taskOk = processLog.indexOf(taskOkMessage) != -1;
    return taskOk;
}

Then we create a new test_emptyTask() function, that checks 'emptyTask':

// child_process_test.js  
exports.test_emptyTask = function(test){  
    launchGruntWithParameter(["emptyTask"], function(processLog){
        var testMessage = "emptyTask is ok";
        test.ok(streamChecker(processLog, testMessage), "Cannot find test message, the task must have failed");
        test.done();
    });
};

This function will get the output stream from the streamReader() function via a callback function (so we had to amend the end of streamReader() function to make a callback). Now the test fails, because the Gruntfile does not log the test message (I commented out test_defaultTask() function):

$ grunt  
Running "jshint:files" (jshint) task  
>> 3 files lint free.

Running "nodeunit:files" (nodeunit) task

child\_process\_test.js


:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::
Running "emptyTask" task

Done, without errors.  
------------------------ CHILD-PROCESS-STREAM-END -----------------------


✖ test_emptyTask

Assertion Message: Cannot find test message, the task may have failed  
AssertionError: Cannot find test message, the task may have failed  
    at Object.ok (.../node_modules/grunt-contrib-nodeunit/node_modules/nodeunit/lib/types.js:83:39)
    at .../src/child_process_test.js:14:9
    at Socket. anonymous (.../src/child_process_test.js:47:4)
    at Socket.EventEmitter.emit (events.js:117:20)
    at _stream_readable.js:920:16
    at process._tickCallback (node.js:415:13)


FAILURES: 1/1 assertions failed (306ms)  
Warning: We have got test failures. Use --force to continue.

Aborted due to warnings.

Ok, let's make the Gruntfile log the test message:

// child process Gruntfile.js  
grunt.registerTask("emptyTask", function(){  
    console.log('emptyTask is ok');
});

And the test passes:

$ grunt  
Running "jshint:files" (jshint) task  
>> 3 files lint free.

Running "nodeunit:files" (nodeunit) task

child\_process\_test.js


:::::::::::::::::::::: CHILD-PROCESS-STREAM-START :::::::::::::::::::::::
Running "emptyTask" task  
emptyTask is ok

Done, without errors.  
------------------------ CHILD-PROCESS-STREAM-END -----------------------


✔ test_emptyTask

OK: 1 assertions (305ms)

Done, without errors.

Really nice, isn't it? Unfortunately there are tasks, which are asynchronous. In this case it may occur, that your task logs the test message, and later the task fails. It is quite a problem, because our test gets back the test message, and the task seems to be ok though it failed. Maybe there are more elegant ways to solve this, but I figured out the following:

With the help of grunt.task.requires(taskName) tasks can be dependent on the successful execution of other tasks. So We have to create a helping task, that is dependent on our tested task, and the helping task will log the test message. If the tested task fails, the helping task won't log the message. The only thing is, that we have to call the helping task with the tested task, too.

// child process Gruntfile.js  
grunt.registerTask("emptyTask", function(){  
    //return false;
});

grunt.registerTask("checkEmptyTask", function(){  
    grunt.task.requires("emptyTask");
    console.log('emptyTask is ok');
});

And the amended test:

// child_process_test.js  
exports.test_emptyTask = function(test){  
    launchGruntWithParameter(["emptyTask", "checkEmptyTask"], function(processLog){
        var testMessage = "emptyTask is ok";
        test.ok(streamChecker(processLog, testMessage), "Cannot find test message, the task may have failed");
        test.done();
    });
};

If emptyTask returns false, checkEmptyTask doesn't log the test message, and the test fails, otherwise the test passes.

Summary

We decided to make a test, which can test Grunt. It was quite a challenge because we also use Grunt for our test (more precisely: Nodeunit plugin for Grunt), so we have to launch Grunt again in a child process (using Nodejs child_process module). First we tried child_process.spawn(), but it crashed in Windows environment. Then we tried child_process.exec(), and it worked, but we wanted to use streams to communicate with the parent process, not a fixed size buffer. We realized, that grunt is a batch file in Windows, and child_process.spawn() cannot work with it. When we launch node with the grunt.js module, it worked, but finally we changed child_process.spawn() to child_process.fork(), because the latter is specially designed for launching Nodejs modules.

After being able to launch Grunt, we would have liked to check the tasks of Gruntfile. Our concept was that we examine the stream of the child process, and somehow find if a given task works. We made the task log a test-message, and later we checked if the stream contained the message. Because some task are asynchronous, it is not a suitable solution in every situation. We created a helping task for the tested task that only runs if the tested test was successfully executed [using grunt.task.requires()]. We logged the test message by the helping task, and checked if the output stream of the child process contains it.

If you want to check out the code, you can clone the project from here.



comments powered by Disqus