Record the mutation score and the concurrency mutants it misses
src/error.c now scores 81.2%: 238 of 293 mutants killed, 204 by a failing test, 24 by failing to compile, and 10 by hanging the suite -- deleting akerr_mutex_init() or the akerr_initializing re-entry guard deadlocks the first test, which is the right answer for a broken lock. Lock deletions are the one survivor category where surviving does not mean harmless, so measure it rather than assume: rebuilt, the surviving "delete the pool lock" mutant fails tests/err_threads_pool.c in 4 of 10 runs and fails under scripts/thread_test.sh in 5 of 5, with no false positive on the unmutated library. The property assertions alone are a coin flip on a missing lock; the sanitizer run is what holds that line. The harness builds mutants with default CMake options and so never sees it -- TODO item 8. Also warn that a sanitized test binary run by hand does not inherit the halt_on_error CTest gives it, and will print a race and still exit 0. That is how the 5-of-5 above first read as 2 of 5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
15
TODO.md
15
TODO.md
@@ -97,6 +97,21 @@ fail carries about twenty-five. Validating more inputs therefore lowers the
|
||||
ratio by construction. Before adding defensive checks, expect to add a test that
|
||||
drives them, as `tests/err_copy_string.c` does.
|
||||
|
||||
## 8. Mutation testing judges concurrency mutants without a sanitizer
|
||||
|
||||
`scripts/mutation_test.py` configures each mutant build with the default CMake
|
||||
options, so a mutant that only breaks under concurrency is judged by a suite
|
||||
running without ThreadSanitizer. Deleting the pool's `akerr_mutex_lock()` call
|
||||
survives the run even though it is a real race: rebuilt and run directly, that
|
||||
mutant fails `tests/err_threads_pool.c` in 4 of 10 runs, and fails under
|
||||
`scripts/thread_test.sh` in 5 of 5. So 81.2% is a floor for that category, not a
|
||||
verdict.
|
||||
|
||||
Closing it means a `--cmake-arg` passthrough on the harness so the mutant build
|
||||
can be configured with `-DAKERR_SANITIZE=thread`. The whole run then costs a
|
||||
TSan-instrumented suite per mutant (roughly 6s instead of 0.4s), so it belongs
|
||||
behind a flag rather than in the default target or in CI.
|
||||
|
||||
## Unrelated pre-existing issues
|
||||
|
||||
- The `AKERR_USE_STDLIB=OFF` build does not compile at all: `bool`, `PATH_MAX`
|
||||
|
||||
Reference in New Issue
Block a user