Gate mutation testing on a measured score, not an inherited one
All checks were successful
libakstdlib CI Build / cmake_build (push) Successful in 2m53s
libakstdlib CI Build / sanitizers (push) Successful in 2m51s
libakstdlib CI Build / coverage (push) Successful in 2m43s
libakstdlib CI Build / mutation_test (push) Successful in 12m37s

The threshold had been 80 against src/stdlib.c alone. There are three
more sources now and nobody had measured them, so that number was a
guess carried forward.

Measured: 72.3%, 188 of a 260-mutant sample from the 1701 the four
sources generate. Gate set to 65 -- a ratchet with headroom for the
runner and for the sample shifting as sources change, not a target.

A sample rather than the whole set, because 1701 rebuilds and test runs
is hours. --max-mutants samples by even index rather than at random, so
the same 260 run every time and the gate stays reproducible; sampling
all four files beats exhausting one of them, which is what this job did
before.

72.3% against the 89.6% reported at 0.1.0 is a change in denominator,
not a regression in the tests. That figure covered one 561-line file;
this covers four totalling 1716 lines, and most of the added surface is
argument validation whose mutants are frequently *equivalent* -- 12 of
the 72 survivors are `errno = 0` deleted from a wrapper whose libc call
always sets errno, which no test that could be written would catch. The
README breaks all 72 down and says which are worth acting on; TODO.md
2.4 carries the three clusters that are.

Two of them were real and are fixed here and in the previous commit: the
right child's `depth + 1` in the depth-first walk, and aksl_tree_remove
on an empty tree, which without its guard dereferences NULL. Neither had
a test; both do now. That is what the harness is for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 08:15:08 -04:00
parent cc5e7899bb
commit f8425b8729
5 changed files with 120 additions and 26 deletions

View File

@@ -30,7 +30,7 @@ set -u
# Keep in sync with the --threshold in .gitea/workflows/ci.yaml, so a push that
# would fail CI fails here first.
MUTATION_THRESHOLD="${AKSL_MUTATION_THRESHOLD:-80}"
MUTATION_THRESHOLD="${AKSL_MUTATION_THRESHOLD:-65}"
ZERO_SHA=0000000000000000000000000000000000000000
@@ -102,11 +102,15 @@ if [ "${AKSL_HOOK_MUTATION:-0}" = "1" ]; then
echo "pre-push: mutation testing, threshold ${MUTATION_THRESHOLD}% (this takes a while)"
# Deliberately not wrapped in run(): this one is slow enough that you want
# to watch it make progress.
# Sampled, like CI: the full set is 1701 mutants and each is a rebuild plus
# a full test run. --max-mutants samples by even index rather than at
# random, so this is the same 260 CI gates on.
if ! python3 scripts/mutation_test.py \
--target src/stdlib.c \
--target src/string.c \
--target src/stream.c \
--target src/collections.c \
--max-mutants "${AKSL_HOOK_MUTANTS:-260}" \
--threshold "$MUTATION_THRESHOLD"; then
echo >&2
echo "pre-push: mutation score below ${MUTATION_THRESHOLD}%. Push aborted." >&2