New AKSL_COVERAGE option instruments the library and its tests with
--coverage -O0 and wires the report into the suite itself, so a plain
ctest --test-dir build-coverage both runs the tests and produces coverage.
Two CTest entries do the work, held in order by a CTest fixture rather
than by declaration order so they also hold under ctest -j:
coverage_reset (FIXTURES_SETUP) clears the .gcda counters before any test,
since gcov counts are cumulative and would otherwise fold in earlier runs;
coverage_report (FIXTURES_CLEANUP) aggregates gcov output afterwards.
AKSL_COVERAGE_THRESHOLD / AKSL_COVERAGE_BRANCH_THRESHOLD gate the report,
the same regression-ratchet idea as the mutation score. The `coverage`
target builds, runs and prints in one step.
scripts/coverage.py parses gcov's JSON output, aggregates line, branch and
function counts across translation units, and lists every uncovered line
and never-called function -- the actionable half, as with surviving
mutants. Python stdlib plus gcc's own gcov only: no lcov, gcovr or
genhtml. It also writes coverage-summary.txt (CTest hides the output of a
passing test) and a Cobertura coverage.xml for CI publishers.
Instrumentation is per target, so deps/libakerror stays out of the report.
The mutation harness now ignores build*/ and gcov artifacts when copying
the tree, so a coverage build does not slow it down.
Baseline on src/stdlib.c: 52.0% of lines, 23.6% of branches, 8 of 21
functions. The uncovered functions are the untested wrappers the mutation
survivors already point at (printf, ato*, stream, realpath, strhash).
Verified:
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON
cmake --build build-coverage --target coverage # 8/8, report printed
ctest --test-dir build-coverage -j8 # fixture order holds
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON -DAKSL_COVERAGE_THRESHOLD=60
ctest --test-dir build-coverage --output-on-failure # gate fails as expected
ctest --test-dir build --output-on-failure # 6/6, no .gcda emitted
ctest --test-dir build-asan --output-on-failure # 6/6
scripts/mutation_test.py --target src/stdlib.c --list # 173 mutants, unchanged
Totals match gcov itself: 51.98% of 202 lines, 23.60% of 661 branches.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cover deterministic TODO 1.1 memory-wrapper behavior with a new CTest target. Exercise malloc/free round trips, NULL argument handling, memset fill/no-op semantics, and memcpy copy/no-op/null-pointer paths.
Harden AKSL_CHECK_OK so success requires no returned error context, catching wrappers that accidentally raise status-0 errors from stale errno.
Co-authored-by: Codex GPT-5.5 Default <codex-gpt-5.5-default@openai.com>
A separate mutation_test job runs scripts/mutation_test.py over src/stdlib.c
with a --threshold gate and publishes a JUnit report, mirroring libakerror's
CI. Checkout needs submodules: recursive -- the harness copies the repo and
builds the copy top-level, so deps/libakerror has to be there.
Measured score on the section 1.0 suite is 46.8% (81/173 killed). The gate is
set at 40: a regression ratchet, not a quality bar. Survivors cluster in the
wrappers that have no tests yet (printf family, ato* family, realpath, the
memory and stream wrappers); the list and tree code that section 1.0 does
cover leaves only 5 survivors between them.
Also caps every test with TIMEOUT 30. Measuring the score turned this up: a
mutant deleting `fast = fast->next->next` makes aksl_list_iterate spin
forever, so ctest never returned, the harness killed ctest at its own 120s
timeout, and the orphaned test binary kept burning a core while holding the
pipe open -- the run wedged and needed killing by hand. With a per-test
timeout ctest reaps its own child, those mutants are killed in 30s, and
nothing is left behind. It guards the real suite against the same shape of
bug too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite could not fail and did not run. test_linkedlist.c asserted nothing
at all -- it printed node names and returned 0 -- so every confirmed list
defect passed it. test_tree.c was red on every run because parms.steps was
never reset between its three searches, and four libakerror tests registered
but never built, reporting Not Run. CI, meanwhile, was not fetching the
submodule, so configure failed before reaching any of it.
- tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to
run an akerror-returning call, assert its status and release the context,
a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver
that fails any test leaking an error-pool slot.
- test_linkedlist.c: rewritten as 15 assertion cases over the append,
iterate and pop behaviour that is correct today.
- test_tree.c: each search builds its own tree and params.
- Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by
design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new
tests asserting correct behaviour for the confirmed defects in TODO.md
2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed",
which is the cue to promote it into AKSL_TESTS.
- add_test/set_tests_properties are shadowed across the libakerror
add_subdirectory call: CMake cannot un-register a test and
set_tests_properties cannot cross directory scopes. The dependency has
its own CI.
- AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan.
- scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to
a manual `mutation` target, not a CI gate until 1.1-1.9 exist.
- CI: checkout with submodules: recursive, and ctest --output-on-failure.
ctest is now 5/5 green in both the default and sanitizer builds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
