A project of mine called for a major DB re-org. The result was going to correct a number of longstanding system issues. It was also going to break the entire application. In theory, once the library code that powers the app was updated to work with the new database structure, the software would be fixed. To insure I addressed all the issues I decided to write a suite of unit tests and watch as they slowly went from all failing to all succeeding.
Given that the application is written in PHP, the obvious choice for a unit test framework was PHPUnit. After reading the docs, however, I couldn't resist putting together my own framework. It had three guiding principles.
First, I wanted tests to be easy to write. The convention I arrived was to have a collection of files nested under a top level tests directory. Each file would contain a variable named $tests that was an array of functions. Each function represents a single test. If the function executes without throwing an exception it is considered passing. That description sounds complex; in pracice the code is pithy:
$ cd tests/utils/date-and-time
$ cat time.php
<?php
$tests = [
function() {
$t1 = strtotime("tomorrow 3:00pm");
$t2 = strottime("+1 day 3:00pm");
assert($t1 == $t2);
},
function() {
$t1 = time() + (60*60*24*3) + (60*60*3);
$t2 = strototime("+3 days 3am");
assert($t1 == $t2);
}
];
?>
assert is a standard php function and can be configured to throw an exception. Though any code that throws an exception can be used to fail a test. By catching exceptions, it's possible to confirm that negative paths are working too. For example:
$tests = [
function() {
$name = gen_unique_username();
$u1 = new_user(['username' => $name]);
try {
$u2 = new_user(['username' => $name]);
assert(false, "Uh, should have failed with duplicate user ex");
} catch(DuplicateUserException $ex) {
assert(true, "Hurray, we got a dup user ex");
}
}
];
Beyond picking the path for the .php file, nothing else is named. That's by design, as it makes composing tests that much simpler.
The next principle I was after was to make the test suite easy to run. Because the framework is so lightweight, it's straightforward to create top level programs that run the tests. For example, here's a test driver that runs from the command line:
<?php
require_once(__DIR__ . '/../lib/siteconfig.php');
(php_sapi_name() == 'cli') || die("I'm a command line tool, thanks.");
function show_outcome($file, $index, $error = false) {
$path = preg_replace('|^.*?/tests/|', '', $file);
echo "$path:$index:" . ($error ? $error : "pass") . "\n";
}
$stats = testing_run_all(['fail' => 'show_outcome']);
echo "\nPass: {$stats['pass']}, Fail: {$stats['fail']}\n";
And here's a similar version that runs from within the web app itself:
<?php
require_once(__DIR__ . '/../lib/siteconfig.php');
header("Content-Type: text/plain");
(g($_GET,'key') == 'a8a49399f8073e7f26d87a12771b954f') || die("Access Denied");
function show_outcome($file, $index, $error = false) {
$path = preg_replace('|^.*?/tests/|', '', $file);
echo "$path:$index:" . ($error ? $error : "pass") . "\n";
}
$stats = testing_run_all(['fail' => 'show_outcome']);
echo "\nPass: {$stats['pass']}, Fail: {$stats['fail']}\n";
And here's a version that's intended to run from cron and doesn't output the details of the tests. Instead, it logs the outcomes to AWS's CloudWatch, which make the data trivial to include in a system-wide dashboard.
<?php
require_once(__DIR__ . '/../lib/siteconfig.php');
(php_sapi_name() == 'cli') || die("I'm a command line tool, thanks.");
function show_outcome($file, $index, $error = false) {
$cloudwatch = get_cloudwatch_instance();
$cloudwatch->putMetricData([
'Namespace' => 'app-unit-tests',
'MetricData' => [
[ 'MetricName' => $error ? "fail" : "pass",
'Timestamp' => time(),
'Value' => 1,
'Unit' => 'Count' ]
]
]);
}
testing_run_all(['fail' => 'show_outcome']);
Finally, I wanted make the framework easy to embed into other projects. By keeping the framework lean, I was able to hit this goal. Recently I got to test this out by packaging unit tests with a custom WordPress plugin. The tests directory was out of the way, and a WordPress friendly test running program was easy to write. Here's the test runner's code. Note how I'm using the prefix 'foo', it assumes I'm authoring with 'foo' plugin.
<?p<?php
/*
* A PHP file for implementing a trivial unit test driver
*/
require_once(__DIR__ . "/../../../../wp-config.php");
header("Cache-Control: no-cache");
header("Expires: 0");
(foo_g($_GET, 'key') == '81b05f7ad603f5d7ae925e44d08ae14b') || wp_die("Permission Denied");
?>
<html>
<head>
<style>
table {
border-collapse: collapse;
border: 1px solid #222;
width: 100%;
}
table td {
border-top: 1px solid #666;
vertical-align: top;
}
td, th {
padding: 1em;
}
</style>
</head>
<body>
<table>
<tr>
<th>Test</th>
<th>Outcome</th>
</tr>
<? $stats = foo_testing_run_all(['fail' => 'show_outcome', 'pass' => 'show_outcome']); ?>
</table>
<h2>Outcomes</h2>
<p>
<b>Pass:</b> <?= $stats['pass'] ?>
</p>
<p>
<b>Fail:</b> <?= $stats['fail'] ?>
</p>
</body>
</html>
<?
function show_outcome($file, $index, $error = false) {
$path = preg_replace('|^.*?/tests/|', '', $file);
echo "<tr>";
echo "<td>$path:$index</td>";
echo "<td><pre>" . ($error ? $error : "pass") . "</pre></td>";
echo "</tr>";
}
Below is the code for the test framework. Feel free to grab and use it. Remember: tests get easier and more addictive the more you write. Like, say, version control once you've setup your environment and overcome the learning curve you'll wonder how you ever lived without this tool.
<?php
// shared/lib/testing.php -- test framework
function testing_run_all($options = []) {
return directory_deep_fold(__DIR__ . '/../../tests',
function($file, $carry) use($options) {
$stats = testing_run($file, $options);
return [
'pass' => $stats['pass'] + $carry['pass'],
'fail' => $stats['fail'] + $carry['fail'],
];
}, ['pass' => 0, 'fail' => 0]);
}
function testing_run($file, $options) {
set_error_handler('exception_error_handler');
ini_set('zend.assertions', true);
ini_set('assert.exception', true);
$pass_handler = g($options, 'pass', function($file, $index) {});
$fail_handler = g($options, 'fail', function($file, $index, $cause) {});
require_once($file);
$stats = ['pass' => 0, 'fail' => 0];
$ctx = [];
if(isset($tests)) {
foreach($tests as $i => $t) {
try {
$ctx = $t($ctx);
$pass_handler($file, $i);
$stats['pass']++;
} catch(Throwable $ex) {
$stats['fail']++;
$fail_handler($file, $i, $ex);
} catch(Exception $ex) {
$stats['fail']++;
$fail_handler($file, $i, $ex);
}
}
} else {
$fail_handler($file, 0, new Exception("No variables \$tests defined"));
$stats['fail']++;
}
return $stats;
}
// Utility Functions
function g($array, $key, $default = false) {
return array_key_exists($key, $array) ? $array[$key] : $default;
}
function directory_deep_fold($root, $fn, $carry) {
$dh = opendir($root);
while($file = readdir($dh)) {
if($file == '.' || $file == '..') {
continue;
} else if(is_dir("$root/$file")) {
$carry = directory_deep_fold("$root/$file", $fn, $carry);
} else {
$carry = call_func($fn, "$root/$file", $carry);
}
}
return $carry;
}
// Borrowed from: https://www.php.net/errorexception
function exception_error_handler($severity, $message, $file, $line) {
if (!(error_reporting() & $severity)) {
return;
}
throw new ErrorException($message, 0, $severity, $file, $line);
}
No comments:
Post a Comment