Andrew Kesterson
437da2960b
Every wrapper outside the list and tree code was untested. Six new test
files close that, following the plan already written in TODO.md 1.2-1.6:
test_stream.c fopen/fread/fwrite/fclose -- happy paths, the round
trip, AKERR_EOF on a short read, AKERR_IO on a stream
opened in the wrong mode, ENOENT, and the NULL guards
test_format.c printf/fprintf/sprintf -- text *and* count asserted
(stdout is pointed at a temp file to check aksl_printf),
all eight NULL guards, EBADF on a read-only stream, and
512 variadic calls in a loop as sanitizer cover for the
missing va_end
test_convert.c ato{i,l,ll,f} happy paths, negatives, leading
whitespace, NULL guards
test_path.c realpath on a file and on a symlink, both compared
against realpath(3) since TMPDIR may itself be a link;
ENOENT, ENOTDIR, NULL path
test_strhash.c djb2 known-answer vectors, len == 0, embedded NUL,
stability, NULL guards
test_convert_strict.c
known-failing (2.1.5): the AKERR_VALUE / ERANGE
contract the ato* family cannot express today
test_tree.c gains the BFS AKERR_NOT_IMPLEMENTED contract, NULL arguments,
and a callback error that is not AKERR_ITERATOR_BREAK propagating out.
Tests deliberately say nothing about behaviour TODO.md records as
defective -- unchecked ptr/mode/resolved_path, short transfers reported as
success, *count left at -1, the djb2 sign extension -- so the eventual fix
does not have to come with a test rewrite. Each failure case in
test_path.c passes a zeroed buffer, because the wrapper's own error path
formats resolved_path with %s (2.1.6).
aksl_capture.h gains aksl_temp_file() with an atexit unlink backstop.
Without it every test that fails before its own unlink leaves temp files
behind -- which is the normal case for a known-failing test, and happens
173 times over in a mutation run.
Coverage on src/stdlib.c: 52.0% -> 99.0% of lines (200/202), 23.6% ->
51.0% of branches, 8/21 -> 21/21 functions. The two uncovered lines are
both `} HANDLE(e, AKERR_ITERATOR_BREAK) {`, where the macro starts with
the `break;` of PROCESS's `case 0:` arm -- reachable only via a non-NULL
error context whose status is zero, the pathology 2.2.1 exists to remove.
Mutation score on src/stdlib.c: 46.8% -> 89.6% (155/173 killed). CI, the
pre-push hook and the docs ratchet from 40 to 80 accordingly, and the 18
survivors are grouped by cause in TODO.md and README.md. A new CI
coverage job gates at 90% lines / 45% branches.
Verified:
ctest --test-dir build # 12/12
ctest --test-dir build-asan # 12/12 under ASan + UBSan
ctest --test-dir build-coverage # 14/14, report attached
ctest --test-dir build -j8 --repeat until-fail:3
gcc -Wall -Wextra -c on all nine test files # no warnings
python3 scripts/mutation_test.py --target src/stdlib.c # 89.6%
No temp files left in /tmp after any of the above.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
101 lines
3.1 KiB
Bash
Executable File
101 lines
3.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# pre-push: build libakstdlib and run the test harnesses before anything leaves
|
|
# this machine.
|
|
#
|
|
# Install (once per clone):
|
|
#
|
|
# git config core.hooksPath .githooks
|
|
#
|
|
# Runs by default, a few seconds all in:
|
|
#
|
|
# * default build + ctest
|
|
# * ASan/UBSan build + ctest
|
|
#
|
|
# Opt in to the slow harness (~25 minutes -- it rebuilds and re-runs the whole
|
|
# suite once per mutant):
|
|
#
|
|
# AKSL_HOOK_MUTATION=1 git push
|
|
#
|
|
# Bypass everything (git's own escape hatch):
|
|
#
|
|
# git push --no-verify
|
|
#
|
|
# Builds go under .git/aksl-prepush so the hook never disturbs whatever is in
|
|
# your own build/ directory. Override with AKSL_HOOK_BUILD_DIR if you want them
|
|
# somewhere else.
|
|
|
|
set -u
|
|
|
|
# Keep in sync with the --threshold in .gitea/workflows/ci.yaml, so a push that
|
|
# would fail CI fails here first.
|
|
MUTATION_THRESHOLD="${AKSL_MUTATION_THRESHOLD:-80}"
|
|
|
|
ZERO_SHA=0000000000000000000000000000000000000000
|
|
|
|
root=$(git rev-parse --show-toplevel) || exit 1
|
|
cd "$root" || exit 1
|
|
|
|
# git feeds us one line per ref being pushed. A push that only *deletes* refs
|
|
# has no commits to test, so there is nothing to do. An empty stdin (nothing to
|
|
# push) lands here too, which is equally fine to skip.
|
|
has_updates=0
|
|
while read -r _local_ref local_sha _remote_ref _remote_sha; do
|
|
if [ "$local_sha" != "$ZERO_SHA" ]; then
|
|
has_updates=1
|
|
fi
|
|
done
|
|
if [ "$has_updates" -eq 0 ]; then
|
|
exit 0
|
|
fi
|
|
|
|
if [ ! -f deps/libakerror/CMakeLists.txt ]; then
|
|
echo "pre-push: deps/libakerror is empty. Run:" >&2
|
|
echo " git submodule update --init --recursive" >&2
|
|
exit 1
|
|
fi
|
|
|
|
builddir="${AKSL_HOOK_BUILD_DIR:-$(git rev-parse --git-dir)/aksl-prepush}"
|
|
mkdir -p "$builddir" || exit 1
|
|
logfile="$builddir/last.log"
|
|
|
|
# Run a step quietly; on failure, dump what it said and abort the push.
|
|
run() {
|
|
if ! "$@" > "$logfile" 2>&1; then
|
|
echo >&2
|
|
echo "pre-push: FAILED: $*" >&2
|
|
echo "---------------------------------------------------------------" >&2
|
|
cat "$logfile" >&2
|
|
echo "---------------------------------------------------------------" >&2
|
|
echo "pre-push: push aborted. Use 'git push --no-verify' to override." >&2
|
|
exit 1
|
|
fi
|
|
}
|
|
|
|
echo "pre-push: default build + ctest"
|
|
run cmake -S . -B "$builddir/default"
|
|
run cmake --build "$builddir/default"
|
|
run ctest --test-dir "$builddir/default" --output-on-failure
|
|
|
|
echo "pre-push: sanitizer build + ctest"
|
|
run cmake -S . -B "$builddir/asan" -DAKSL_SANITIZE=ON
|
|
run cmake --build "$builddir/asan"
|
|
run ctest --test-dir "$builddir/asan" --output-on-failure
|
|
|
|
if [ "${AKSL_HOOK_MUTATION:-0}" = "1" ]; then
|
|
echo "pre-push: mutation testing, threshold ${MUTATION_THRESHOLD}% (this takes a while)"
|
|
# Deliberately not wrapped in run(): this one is slow enough that you want
|
|
# to watch it make progress.
|
|
if ! python3 scripts/mutation_test.py \
|
|
--target src/stdlib.c \
|
|
--threshold "$MUTATION_THRESHOLD"; then
|
|
echo >&2
|
|
echo "pre-push: mutation score below ${MUTATION_THRESHOLD}%. Push aborted." >&2
|
|
echo "pre-push: use 'git push --no-verify' to override." >&2
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
echo "pre-push: OK"
|
|
exit 0
|