Record the mutation score and the concurrency mutants it misses
Some checks failed
libakerror CI Build / cmake_build (push) Successful in 2m48s
libakerror CI Build / coverage (push) Successful in 2m47s
libakerror CI Build / thread_sanitizer (push) Failing after 2m49s
libakerror CI Build / mutation_test (push) Successful in 38m22s

src/error.c now scores 81.2%: 238 of 293 mutants killed, 204 by a
failing test, 24 by failing to compile, and 10 by hanging the suite --
deleting akerr_mutex_init() or the akerr_initializing re-entry guard
deadlocks the first test, which is the right answer for a broken lock.

Lock deletions are the one survivor category where surviving does not
mean harmless, so measure it rather than assume: rebuilt, the surviving
"delete the pool lock" mutant fails tests/err_threads_pool.c in 4 of 10
runs and fails under scripts/thread_test.sh in 5 of 5, with no false
positive on the unmutated library. The property assertions alone are a
coin flip on a missing lock; the sanitizer run is what holds that line.
The harness builds mutants with default CMake options and so never sees
it -- TODO item 8.

Also warn that a sanitized test binary run by hand does not inherit the
halt_on_error CTest gives it, and will print a race and still exit 0.
That is how the 5-of-5 above first read as 2 of 5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 08:45:20 -04:00
parent be24f80022
commit 756933c600
3 changed files with 51 additions and 5 deletions

15
TODO.md
View File

@@ -97,6 +97,21 @@ fail carries about twenty-five. Validating more inputs therefore lowers the
ratio by construction. Before adding defensive checks, expect to add a test that
drives them, as `tests/err_copy_string.c` does.
## 8. Mutation testing judges concurrency mutants without a sanitizer
`scripts/mutation_test.py` configures each mutant build with the default CMake
options, so a mutant that only breaks under concurrency is judged by a suite
running without ThreadSanitizer. Deleting the pool's `akerr_mutex_lock()` call
survives the run even though it is a real race: rebuilt and run directly, that
mutant fails `tests/err_threads_pool.c` in 4 of 10 runs, and fails under
`scripts/thread_test.sh` in 5 of 5. So 81.2% is a floor for that category, not a
verdict.
Closing it means a `--cmake-arg` passthrough on the harness so the mutant build
can be configured with `-DAKERR_SANITIZE=thread`. The whole run then costs a
TSan-instrumented suite per mutant (roughly 6s instead of 0.4s), so it belongs
behind a flag rather than in the default target or in CI.
## Unrelated pre-existing issues
- The `AKERR_USE_STDLIB=OFF` build does not compile at all: `bool`, `PATH_MAX`