Commit Graph

3 Commits

Author SHA1 Message Date
82c47ed773 Add code coverage to the CTest suite
New AKSL_COVERAGE option instruments the library and its tests with
--coverage -O0 and wires the report into the suite itself, so a plain
ctest --test-dir build-coverage both runs the tests and produces coverage.

Two CTest entries do the work, held in order by a CTest fixture rather
than by declaration order so they also hold under ctest -j:
coverage_reset (FIXTURES_SETUP) clears the .gcda counters before any test,
since gcov counts are cumulative and would otherwise fold in earlier runs;
coverage_report (FIXTURES_CLEANUP) aggregates gcov output afterwards.
AKSL_COVERAGE_THRESHOLD / AKSL_COVERAGE_BRANCH_THRESHOLD gate the report,
the same regression-ratchet idea as the mutation score. The `coverage`
target builds, runs and prints in one step.

scripts/coverage.py parses gcov's JSON output, aggregates line, branch and
function counts across translation units, and lists every uncovered line
and never-called function -- the actionable half, as with surviving
mutants. Python stdlib plus gcc's own gcov only: no lcov, gcovr or
genhtml. It also writes coverage-summary.txt (CTest hides the output of a
passing test) and a Cobertura coverage.xml for CI publishers.

Instrumentation is per target, so deps/libakerror stays out of the report.
The mutation harness now ignores build*/ and gcov artifacts when copying
the tree, so a coverage build does not slow it down.

Baseline on src/stdlib.c: 52.0% of lines, 23.6% of branches, 8 of 21
functions. The uncovered functions are the untested wrappers the mutation
survivors already point at (printf, ato*, stream, realpath, strhash).

Verified:
  cmake -S . -B build-coverage -DAKSL_COVERAGE=ON
  cmake --build build-coverage --target coverage      # 8/8, report printed
  ctest --test-dir build-coverage -j8                 # fixture order holds
  cmake -S . -B build-coverage -DAKSL_COVERAGE=ON -DAKSL_COVERAGE_THRESHOLD=60
  ctest --test-dir build-coverage --output-on-failure  # gate fails as expected
  ctest --test-dir build --output-on-failure           # 6/6, no .gcda emitted
  ctest --test-dir build-asan --output-on-failure      # 6/6
  scripts/mutation_test.py --target src/stdlib.c --list # 173 mutants, unchanged
Totals match gcov itself: 51.98% of 202 lines, 23.60% of 661 branches.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 02:07:35 -04:00
54fb44cd2b Document the test harnesses and add a pre-push hook
All checks were successful
libakstdlib CI Build / cmake_build (push) Successful in 2m46s
libakstdlib CI Build / mutation_test (push) Successful in 7m48s
README.md was a build badge and nothing else. It now covers what the library
is, how to build it, and how to run each of the three harnesses locally: the
ctest suite, the AKSL_SANITIZE ASan/UBSan build, and mutation testing.

The section on reading ctest output matters most: two of the three test lists
are marked WILL_FAIL, so an all-green run does not mean the library is
defect-free -- it means the known-good tests passed and the known-bad ones are
still failing in the documented way.

.githooks/pre-push runs the two fast harnesses (default build + ctest, then
sanitizer build + ctest) before a push leaves the machine. It skips when there
is nothing to test (branch deletions, empty pushes), fails the push with the
captured output on any error, and builds under .git/aksl-prepush so it never
disturbs your own build/. The slow mutation gate is opt-in via
AKSL_HOOK_MUTATION=1, and git push --no-verify bypasses everything.

Enable with: git config core.hooksPath .githooks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 11:51:28 -04:00
3e3ea41dc8 Add CI build
Some checks failed
libakstdlib CI Build / cmake_build (push) Failing after 2m41s
2026-06-27 10:40:06 -04:00