For many of the assignments in this class, you will need to compare your program’s output to some known correct output. This is fairly easy to do for a few cases, but it can get quite annoying if you have to do it repeatedly. While there are many heavy-weight systems for testing programs, we can rely on some basic functionality in the shell and a standard command line utility to build a simple testing system.
Imagine we have a simple program called
sorter that is supposed to read in values and print them out in sorted order.
The first step is to write a simple shell script that can run our program on some basic test cases.
We’ll call this file testcases.sh, with the contents below:
#!/bin/bash echo 1 2 3 4 5 | ./sorter echo 5 4 3 2 1 | ./sorter echo 9 3 0 5 3 | ./sorter
The first line of this file tells Linux that it should run this code with the
bash program, which also happens to be the default shell.
Before you can run the program you have to make it executable.
You can do this from your shell:
$ chmod u+x testcases.sh
This grants the user (you) permission to execute the file
Now we have a script that can run our program on a few simple inputs. The next step is to make a file that holds the expected output for each run. We’ll call this correct-outputs.txt:
1 2 3 4 5 1 2 3 4 5 0 3 3 5 9
Each line corresponds to the output from a single run of the program.
Given these correct outputs, now we can run
testcases.sh to get the real output from our program.
We don’t necessarily want to see every line of output if we have many test cases, so we’ll instead capture the output in a file.
Then, we can use the
diff command to compare
It would be convenient if we could use
make to run our tests, so we’ll actually add the code to do this to
At the end of
Makefile, we’ll add the following lines:
test:: sorter ./testcases.sh > output.txt diff output.txt correct-output.txt
Make sure your indented lines are intented with Tab, not spaces. Makefiles are picky about whitespace, and you could get a cryptic error if you indent incorrectly here.
Now to run our tests, we just run
If everything matches,
make test will complete successfully.
If it doesn’t match,
make test will exit with an error and print out the lines that differ in the two output files.
You could split up your test cases to make it easier to check output values if you have many long test cases, but this approach should work well for small numbers of tests that don’t produce too much output.