Gate mutation testing on a measured score, not an inherited one
All checks were successful
libakstdlib CI Build / cmake_build (push) Successful in 2m53s
libakstdlib CI Build / sanitizers (push) Successful in 2m51s
libakstdlib CI Build / coverage (push) Successful in 2m43s
libakstdlib CI Build / mutation_test (push) Successful in 12m37s

The threshold had been 80 against src/stdlib.c alone. There are three
more sources now and nobody had measured them, so that number was a
guess carried forward.

Measured: 72.3%, 188 of a 260-mutant sample from the 1701 the four
sources generate. Gate set to 65 -- a ratchet with headroom for the
runner and for the sample shifting as sources change, not a target.

A sample rather than the whole set, because 1701 rebuilds and test runs
is hours. --max-mutants samples by even index rather than at random, so
the same 260 run every time and the gate stays reproducible; sampling
all four files beats exhausting one of them, which is what this job did
before.

72.3% against the 89.6% reported at 0.1.0 is a change in denominator,
not a regression in the tests. That figure covered one 561-line file;
this covers four totalling 1716 lines, and most of the added surface is
argument validation whose mutants are frequently *equivalent* -- 12 of
the 72 survivors are `errno = 0` deleted from a wrapper whose libc call
always sets errno, which no test that could be written would catch. The
README breaks all 72 down and says which are worth acting on; TODO.md
2.4 carries the three clusters that are.

Two of them were real and are fixed here and in the previous commit: the
right child's `depth + 1` in the depth-first walk, and aksl_tree_remove
on an empty tree, which without its guard dereferences NULL. Neither had
a test; both do now. That is what the harness is for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 08:15:08 -04:00
parent cc5e7899bb
commit f8425b8729
5 changed files with 120 additions and 26 deletions

30
TODO.md
View File

@@ -16,6 +16,7 @@ second, what is missing last.
| Line coverage | 99.5% (1708/1716) |
| Function coverage | 100% (154/154) |
| Doxygen | 100% of 154, gated — `cmake --build build --target docs` fails on an undocumented function, parameter or return |
| Mutation score | 72.3% (188/260 sampled from 1701), gated at 65 |
The six confirmed defects that used to head this file are fixed and
`AKSL_KNOWN_FAILING_TESTS` is empty. What they were, and what changed as a
@@ -139,7 +140,34 @@ message can name the caller's full version.
to participate, that is the line to change, and the `#if` in
`tests/test_version.c` is the test that encodes the rule.
### 2.4 Four uncovered `HANDLE(e, AKERR_ITERATOR_BREAK)` lines
### 2.4 Surviving mutants worth turning into assertions
The mutation harness samples 260 of 1701 mutants and kills 72.3% of them. Most of
the 72 survivors are equivalent mutants rather than missing tests — `README.md`
has the full breakdown — but three clusters are real work:
- **`FINISH(e, true)``FINISH(e, false)`, 3 survivors.** An error swallowed
instead of propagated out of an `ATTEMPT` block, and nothing notices. Each one
is a call whose failure path is exercised but whose *propagation* is not: the
test asserts the status the callee raised without checking it came from the
callee rather than being re-raised locally. `tests/test_pool.c`'s origin
assertions are the shape of the fix.
- **`SUCCEED_RETURN` deleted, 7 survivors.** The function falls off the end and
returns whatever is in the return register, which is NULL often enough to pass.
These need an assertion on the *side effect* — the buffer that was filled, the
node that was linked — rather than on the returned status.
- **`FAIL_*` guards deleted, ~6 of the 14 in that group.** Each is an argument
check nothing drives. The other 8 in the group are constant shifts that no test
can catch, because the test names the same constant symbolically and moves with
it.
Two survivors in this class were real and are fixed: the right child's
`depth + 1` in the depth-first walk, and `aksl_tree_remove` on an empty tree.
**What closing them would touch.** Only `tests/`. Raise the `--threshold` in
`.gitea/workflows/ci.yaml` and `.githooks/pre-push` in step, as a ratchet.
### 2.5 Four uncovered `HANDLE(e, AKERR_ITERATOR_BREAK)` lines
Macro artifacts rather than gaps. In libakerror that macro begins with the
`break;` belonging to `PROCESS`'s `case 0:` arm, reachable only when a callback