scripts/coverage.py configures an instrumented build tree
(-DAKERR_COVERAGE=ON), runs the CTest suite in it, and reports merged
gcov line/branch/function coverage per library source. Like the mutation
harness it has no third-party dependencies and supports --threshold and
--junit; thresholds gate each file as well as the total so the generated
status-name table cannot mask a regression in src/error.c.
Only the library is instrumented. The public header's macros cannot be
measured this way -- GCC attributes an expanded macro to its call site,
so header logic would report as lines of the test that used it -- which
is what mutation testing against include/akerror.tmpl.h is for.
Coverage flags are applied per target rather than globally, so they do
not leak into the exported/installed target interface.
Current numbers for src/error.c are 94.0% line and 59.5% branch; the CI
gate is set to 90/50 to keep headroom, matching the convention used for
the mutation score threshold.
Tests run: ctest (23/23), cmake --build build --target coverage,
threshold gate verified failing at --threshold 99, cmake --install
checked for flag leakage, out-of-tree --build-dir checked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Produce machine-readable results and surface them in the Gitea pipeline:
- ctest: run with --output-junit to write ctest-junit.xml. The path must be
absolute ("$(pwd)/...") because --output-junit otherwise resolves relative to
the --test-dir build directory.
- mutation_test.py: new --junit FILE option writes a JUnit report where each
mutant is a test case and a surviving mutant is a <failure> (so gaps show up
as failing tests).
- .gitea/workflows/ci.yaml: both jobs generate their XML and feed it to
mikepenz/action-junit-report with `if: always()`, so results publish even
when a gate fails. Mutation publishing is display-only (fail_on_failure:
false); the --threshold flag remains the gate.
- .gitignore: ignore the generated *-junit.xml artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce a self-contained mutation testing harness that verifies the unit
tests actually catch bugs: it makes small deliberate breakages to the library
(flip comparisons, delete statements, swap true/false, etc.), rebuilds, and
runs the whole CTest suite against each mutant. Tests that still pass reveal a
gap; tests that fail "kill" the mutant.
- scripts/mutation_test.py: the engine (stdlib only, no LLVM/clang deps).
Operators ROR/LCR/BCR/AOR/ICR/SDL over src/error.c and the macro header.
Mutates a scratch copy, never the working tree. Supports --target, --list,
--max-mutants sampling, --threshold gating, --timeout.
- CMakeLists.txt: 'mutation' custom target (cmake --build build --target mutation).
- .gitea/workflows/ci.yaml: gated mutation job on src/error.c (threshold 65%).
- tests/MUTATION.md: how to run, interpret survivors, and known equivalents.
Close the real gaps the harness found in src/error.c (score 53% -> 71%):
- err_error_names: the AKERR_* codes have their names registered by akerr_init
- err_release_clears: releasing a context wipes it before reuse
- err_pool_exhaust: akerr_next_error returns NULL when the pool is full and
always hands back the lowest free slot
Also surfaced (documented, not fixed): AKERR_MAX_ERR_VALUE (+15) is below
AKERR_NOT_IMPLEMENTED (+16) and AKERR_BADEXC (+17), so those codes can never
have a name registered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
