Add mutation testing to validate the test suite
Introduce a self-contained mutation testing harness that verifies the unit tests actually catch bugs: it makes small deliberate breakages to the library (flip comparisons, delete statements, swap true/false, etc.), rebuilds, and runs the whole CTest suite against each mutant. Tests that still pass reveal a gap; tests that fail "kill" the mutant. - scripts/mutation_test.py: the engine (stdlib only, no LLVM/clang deps). Operators ROR/LCR/BCR/AOR/ICR/SDL over src/error.c and the macro header. Mutates a scratch copy, never the working tree. Supports --target, --list, --max-mutants sampling, --threshold gating, --timeout. - CMakeLists.txt: 'mutation' custom target (cmake --build build --target mutation). - .gitea/workflows/ci.yaml: gated mutation job on src/error.c (threshold 65%). - tests/MUTATION.md: how to run, interpret survivors, and known equivalents. Close the real gaps the harness found in src/error.c (score 53% -> 71%): - err_error_names: the AKERR_* codes have their names registered by akerr_init - err_release_clears: releasing a context wipes it before reuse - err_pool_exhaust: akerr_next_error returns NULL when the pool is full and always hands back the lowest free slot Also surfaced (documented, not fixed): AKERR_MAX_ERR_VALUE (+15) is below AKERR_NOT_IMPLEMENTED (+16) and AKERR_BADEXC (+17), so those codes can never have a name registered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
44
tests/err_release_clears.c
Normal file
44
tests/err_release_clears.c
Normal file
@@ -0,0 +1,44 @@
|
||||
#include "akerror.h"
|
||||
#include "err_capture.h"
|
||||
#include <string.h>
|
||||
|
||||
/*
|
||||
* Releasing an error context back to the pool must wipe it, so the next caller
|
||||
* that checks it out never sees stale status/message/stacktrace from a previous
|
||||
* error. Mutation testing showed the clearing memset in akerr_release_error
|
||||
* could be deleted without any test noticing.
|
||||
*/
|
||||
|
||||
akerr_ErrorContext *boom(void)
|
||||
{
|
||||
PREPARE_ERROR(e);
|
||||
FAIL_RETURN(e, AKERR_VALUE, "stale dirty message that must not survive");
|
||||
}
|
||||
|
||||
int main(void)
|
||||
{
|
||||
akerr_capture_install();
|
||||
akerr_init();
|
||||
|
||||
/* Raise and fully handle an error; FINISH_NORETURN releases it to the pool. */
|
||||
PREPARE_ERROR(e);
|
||||
ATTEMPT {
|
||||
CATCH(e, boom());
|
||||
} CLEANUP {
|
||||
} PROCESS(e) {
|
||||
} HANDLE(e, AKERR_VALUE) {
|
||||
} FINISH_NORETURN(e);
|
||||
|
||||
AKERR_CHECK(e == NULL);
|
||||
|
||||
/* The next context handed out is the slot we just released: it must be clean. */
|
||||
akerr_ErrorContext *slot = akerr_next_error();
|
||||
AKERR_CHECK(slot != NULL);
|
||||
AKERR_CHECK(slot->status == 0);
|
||||
AKERR_CHECK(slot->message[0] == '\0');
|
||||
AKERR_CHECK(slot->stacktracebuf[0] == '\0');
|
||||
AKERR_CHECK(strstr(slot->message, "stale dirty message") == NULL);
|
||||
|
||||
fprintf(stderr, "err_release_clears ok\n");
|
||||
return 0;
|
||||
}
|
||||
Reference in New Issue
Block a user