Andrew Kesterson
f8425b8729
All checks were successful
The threshold had been 80 against src/stdlib.c alone. There are three more sources now and nobody had measured them, so that number was a guess carried forward. Measured: 72.3%, 188 of a 260-mutant sample from the 1701 the four sources generate. Gate set to 65 -- a ratchet with headroom for the runner and for the sample shifting as sources change, not a target. A sample rather than the whole set, because 1701 rebuilds and test runs is hours. --max-mutants samples by even index rather than at random, so the same 260 run every time and the gate stays reproducible; sampling all four files beats exhausting one of them, which is what this job did before. 72.3% against the 89.6% reported at 0.1.0 is a change in denominator, not a regression in the tests. That figure covered one 561-line file; this covers four totalling 1716 lines, and most of the added surface is argument validation whose mutants are frequently *equivalent* -- 12 of the 72 survivors are `errno = 0` deleted from a wrapper whose libc call always sets errno, which no test that could be written would catch. The README breaks all 72 down and says which are worth acting on; TODO.md 2.4 carries the three clusters that are. Two of them were real and are fixed here and in the previous commit: the right child's `depth + 1` in the depth-first walk, and aksl_tree_remove on an empty tree, which without its guard dereferences NULL. Neither had a test; both do now. That is what the harness is for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
124 lines
4.4 KiB
Bash
Executable File
124 lines
4.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# pre-push: build libakstdlib and run the test harnesses before anything leaves
|
|
# this machine.
|
|
#
|
|
# Install (once per clone):
|
|
#
|
|
# git config core.hooksPath .githooks
|
|
#
|
|
# Runs by default, a few seconds all in:
|
|
#
|
|
# * default build + ctest (with -Werror, as CI has it)
|
|
# * ASan/UBSan build + ctest
|
|
# * doxygen -- fails on an undocumented public function
|
|
#
|
|
# Opt in to the slow harness (~25 minutes -- it rebuilds and re-runs the whole
|
|
# suite once per mutant):
|
|
#
|
|
# AKSL_HOOK_MUTATION=1 git push
|
|
#
|
|
# Bypass everything (git's own escape hatch):
|
|
#
|
|
# git push --no-verify
|
|
#
|
|
# Builds go under .git/aksl-prepush so the hook never disturbs whatever is in
|
|
# your own build/ directory. Override with AKSL_HOOK_BUILD_DIR if you want them
|
|
# somewhere else.
|
|
|
|
set -u
|
|
|
|
# Keep in sync with the --threshold in .gitea/workflows/ci.yaml, so a push that
|
|
# would fail CI fails here first.
|
|
MUTATION_THRESHOLD="${AKSL_MUTATION_THRESHOLD:-65}"
|
|
|
|
ZERO_SHA=0000000000000000000000000000000000000000
|
|
|
|
root=$(git rev-parse --show-toplevel) || exit 1
|
|
cd "$root" || exit 1
|
|
|
|
# git feeds us one line per ref being pushed. A push that only *deletes* refs
|
|
# has no commits to test, so there is nothing to do. An empty stdin (nothing to
|
|
# push) lands here too, which is equally fine to skip.
|
|
has_updates=0
|
|
while read -r _local_ref local_sha _remote_ref _remote_sha; do
|
|
if [ "$local_sha" != "$ZERO_SHA" ]; then
|
|
has_updates=1
|
|
fi
|
|
done
|
|
if [ "$has_updates" -eq 0 ]; then
|
|
exit 0
|
|
fi
|
|
|
|
if [ ! -f deps/libakerror/CMakeLists.txt ]; then
|
|
echo "pre-push: deps/libakerror is empty. Run:" >&2
|
|
echo " git submodule update --init --recursive" >&2
|
|
exit 1
|
|
fi
|
|
|
|
builddir="${AKSL_HOOK_BUILD_DIR:-$(git rev-parse --git-dir)/aksl-prepush}"
|
|
mkdir -p "$builddir" || exit 1
|
|
logfile="$builddir/last.log"
|
|
|
|
# Run a step quietly; on failure, dump what it said and abort the push.
|
|
run() {
|
|
if ! "$@" > "$logfile" 2>&1; then
|
|
echo >&2
|
|
echo "pre-push: FAILED: $*" >&2
|
|
echo "---------------------------------------------------------------" >&2
|
|
cat "$logfile" >&2
|
|
echo "---------------------------------------------------------------" >&2
|
|
echo "pre-push: push aborted. Use 'git push --no-verify' to override." >&2
|
|
exit 1
|
|
fi
|
|
}
|
|
|
|
# -DAKSL_WERROR=ON here and not in a plain local build: a warning that stops you
|
|
# mid-thought teaches people to turn warnings off, but a warning that reaches
|
|
# main is one nobody will look at again. This is the last point where it is
|
|
# still cheap to fix.
|
|
echo "pre-push: default build + ctest"
|
|
run cmake -S . -B "$builddir/default" -DAKSL_WERROR=ON
|
|
run cmake --build "$builddir/default"
|
|
run ctest --test-dir "$builddir/default" --output-on-failure
|
|
|
|
echo "pre-push: sanitizer build + ctest"
|
|
run cmake -S . -B "$builddir/asan" -DAKSL_SANITIZE=ON -DAKSL_WERROR=ON
|
|
run cmake --build "$builddir/asan"
|
|
run ctest --test-dir "$builddir/asan" --output-on-failure
|
|
|
|
# Doxygen runs with WARN_IF_UNDOCUMENTED and WARN_NO_PARAMDOC on and EXTRACT_ALL
|
|
# off, so the target fails when a public function, parameter or return value has
|
|
# nobody describing it. Skipped where doxygen is not installed rather than
|
|
# failing: it is a documentation check, not a build dependency.
|
|
if command -v doxygen > /dev/null 2>&1; then
|
|
echo "pre-push: doxygen"
|
|
run cmake --build "$builddir/default" --target docs
|
|
else
|
|
echo "pre-push: doxygen not installed, skipping the documentation check"
|
|
fi
|
|
|
|
if [ "${AKSL_HOOK_MUTATION:-0}" = "1" ]; then
|
|
echo "pre-push: mutation testing, threshold ${MUTATION_THRESHOLD}% (this takes a while)"
|
|
# Deliberately not wrapped in run(): this one is slow enough that you want
|
|
# to watch it make progress.
|
|
# Sampled, like CI: the full set is 1701 mutants and each is a rebuild plus
|
|
# a full test run. --max-mutants samples by even index rather than at
|
|
# random, so this is the same 260 CI gates on.
|
|
if ! python3 scripts/mutation_test.py \
|
|
--target src/stdlib.c \
|
|
--target src/string.c \
|
|
--target src/stream.c \
|
|
--target src/collections.c \
|
|
--max-mutants "${AKSL_HOOK_MUTANTS:-260}" \
|
|
--threshold "$MUTATION_THRESHOLD"; then
|
|
echo >&2
|
|
echo "pre-push: mutation score below ${MUTATION_THRESHOLD}%. Push aborted." >&2
|
|
echo "pre-push: use 'git push --no-verify' to override." >&2
|
|
exit 1
|
|
fi
|
|
fi
|
|
|
|
echo "pre-push: OK"
|
|
exit 0
|