Gate mutation testing on a measured score, not an inherited one
All checks were successful
All checks were successful
The threshold had been 80 against src/stdlib.c alone. There are three more sources now and nobody had measured them, so that number was a guess carried forward. Measured: 72.3%, 188 of a 260-mutant sample from the 1701 the four sources generate. Gate set to 65 -- a ratchet with headroom for the runner and for the sample shifting as sources change, not a target. A sample rather than the whole set, because 1701 rebuilds and test runs is hours. --max-mutants samples by even index rather than at random, so the same 260 run every time and the gate stays reproducible; sampling all four files beats exhausting one of them, which is what this job did before. 72.3% against the 89.6% reported at 0.1.0 is a change in denominator, not a regression in the tests. That figure covered one 561-line file; this covers four totalling 1716 lines, and most of the added surface is argument validation whose mutants are frequently *equivalent* -- 12 of the 72 survivors are `errno = 0` deleted from a wrapper whose libc call always sets errno, which no test that could be written would catch. The README breaks all 72 down and says which are worth acting on; TODO.md 2.4 carries the three clusters that are. Two of them were real and are fixed here and in the previous commit: the right child's `depth + 1` in the depth-first walk, and aksl_tree_remove on an empty tree, which without its guard dereferences NULL. Neither had a test; both do now. That is what the harness is for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
30
TODO.md
30
TODO.md
@@ -16,6 +16,7 @@ second, what is missing last.
|
||||
| Line coverage | 99.5% (1708/1716) |
|
||||
| Function coverage | 100% (154/154) |
|
||||
| Doxygen | 100% of 154, gated — `cmake --build build --target docs` fails on an undocumented function, parameter or return |
|
||||
| Mutation score | 72.3% (188/260 sampled from 1701), gated at 65 |
|
||||
|
||||
The six confirmed defects that used to head this file are fixed and
|
||||
`AKSL_KNOWN_FAILING_TESTS` is empty. What they were, and what changed as a
|
||||
@@ -139,7 +140,34 @@ message can name the caller's full version.
|
||||
to participate, that is the line to change, and the `#if` in
|
||||
`tests/test_version.c` is the test that encodes the rule.
|
||||
|
||||
### 2.4 Four uncovered `HANDLE(e, AKERR_ITERATOR_BREAK)` lines
|
||||
### 2.4 Surviving mutants worth turning into assertions
|
||||
|
||||
The mutation harness samples 260 of 1701 mutants and kills 72.3% of them. Most of
|
||||
the 72 survivors are equivalent mutants rather than missing tests — `README.md`
|
||||
has the full breakdown — but three clusters are real work:
|
||||
|
||||
- **`FINISH(e, true)` → `FINISH(e, false)`, 3 survivors.** An error swallowed
|
||||
instead of propagated out of an `ATTEMPT` block, and nothing notices. Each one
|
||||
is a call whose failure path is exercised but whose *propagation* is not: the
|
||||
test asserts the status the callee raised without checking it came from the
|
||||
callee rather than being re-raised locally. `tests/test_pool.c`'s origin
|
||||
assertions are the shape of the fix.
|
||||
- **`SUCCEED_RETURN` deleted, 7 survivors.** The function falls off the end and
|
||||
returns whatever is in the return register, which is NULL often enough to pass.
|
||||
These need an assertion on the *side effect* — the buffer that was filled, the
|
||||
node that was linked — rather than on the returned status.
|
||||
- **`FAIL_*` guards deleted, ~6 of the 14 in that group.** Each is an argument
|
||||
check nothing drives. The other 8 in the group are constant shifts that no test
|
||||
can catch, because the test names the same constant symbolically and moves with
|
||||
it.
|
||||
|
||||
Two survivors in this class were real and are fixed: the right child's
|
||||
`depth + 1` in the depth-first walk, and `aksl_tree_remove` on an empty tree.
|
||||
|
||||
**What closing them would touch.** Only `tests/`. Raise the `--threshold` in
|
||||
`.gitea/workflows/ci.yaml` and `.githooks/pre-push` in step, as a ratchet.
|
||||
|
||||
### 2.5 Four uncovered `HANDLE(e, AKERR_ITERATOR_BREAK)` lines
|
||||
|
||||
Macro artifacts rather than gaps. In libakerror that macro begins with the
|
||||
`break;` belonging to `PROCESS`'s `case 0:` arm, reachable only when a callback
|
||||
|
||||
Reference in New Issue
Block a user