Andrew Kesterson
43f46dca64
Introduce a self-contained mutation testing harness that verifies the unit tests actually catch bugs: it makes small deliberate breakages to the library (flip comparisons, delete statements, swap true/false, etc.), rebuilds, and runs the whole CTest suite against each mutant. Tests that still pass reveal a gap; tests that fail "kill" the mutant. - scripts/mutation_test.py: the engine (stdlib only, no LLVM/clang deps). Operators ROR/LCR/BCR/AOR/ICR/SDL over src/error.c and the macro header. Mutates a scratch copy, never the working tree. Supports --target, --list, --max-mutants sampling, --threshold gating, --timeout. - CMakeLists.txt: 'mutation' custom target (cmake --build build --target mutation). - .gitea/workflows/ci.yaml: gated mutation job on src/error.c (threshold 65%). - tests/MUTATION.md: how to run, interpret survivors, and known equivalents. Close the real gaps the harness found in src/error.c (score 53% -> 71%): - err_error_names: the AKERR_* codes have their names registered by akerr_init - err_release_clears: releasing a context wipes it before reuse - err_pool_exhaust: akerr_next_error returns NULL when the pool is full and always hands back the lowest free slot Also surfaced (documented, not fixed): AKERR_MAX_ERR_VALUE (+15) is below AKERR_NOT_IMPLEMENTED (+16) and AKERR_BADEXC (+17), so those codes can never have a name registered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
43 lines
1.6 KiB
YAML
43 lines
1.6 KiB
YAML
name: libakerror CI Build
|
|
run-name: ${{ gitea.actor }} libakerror test
|
|
on: [push]
|
|
|
|
jobs:
|
|
cmake_build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- run: echo "Triggered by ${{ gitea.event_name }} from ${{ gitea.repository }}@${{ gitea.ref }}. Building on ${{ runner.os }}."
|
|
- name: Check out repository code
|
|
uses: actions/checkout@v4
|
|
- name: dependencies
|
|
run: |
|
|
sudo apt-get update -y
|
|
sudo apt-get install -y cmake gcc moreutils
|
|
- name: build and test
|
|
run: |
|
|
mkdir installdir
|
|
cmake -S . -B build -DCMAKE_INSTALL_PREFIX=installdir
|
|
cmake --build build
|
|
cmake --install build
|
|
cmake --build build --target test
|
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|
|
|
|
mutation_test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Check out repository code
|
|
uses: actions/checkout@v4
|
|
- name: dependencies
|
|
run: |
|
|
sudo apt-get update -y
|
|
sudo apt-get install -y cmake gcc moreutils python3
|
|
# Verify the tests actually catch bugs: break the library many ways and
|
|
# confirm the suite fails. Gated on src/error.c (fast, deterministic).
|
|
# The threshold keeps headroom below the current score for equivalent
|
|
# mutants; see tests/MUTATION.md. Run the full default target locally for
|
|
# deeper (slower) coverage including the macro header.
|
|
- name: mutation testing
|
|
run: |
|
|
python3 scripts/mutation_test.py --target src/error.c --threshold 65
|
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|