Files
libakstdlib/.gitea/workflows/ci.yaml
Andrew Kesterson 437da2960b Test the libc wrappers: 52% -> 99% line coverage
Every wrapper outside the list and tree code was untested. Six new test
files close that, following the plan already written in TODO.md 1.2-1.6:

  test_stream.c   fopen/fread/fwrite/fclose -- happy paths, the round
                  trip, AKERR_EOF on a short read, AKERR_IO on a stream
                  opened in the wrong mode, ENOENT, and the NULL guards
  test_format.c   printf/fprintf/sprintf -- text *and* count asserted
                  (stdout is pointed at a temp file to check aksl_printf),
                  all eight NULL guards, EBADF on a read-only stream, and
                  512 variadic calls in a loop as sanitizer cover for the
                  missing va_end
  test_convert.c  ato{i,l,ll,f} happy paths, negatives, leading
                  whitespace, NULL guards
  test_path.c     realpath on a file and on a symlink, both compared
                  against realpath(3) since TMPDIR may itself be a link;
                  ENOENT, ENOTDIR, NULL path
  test_strhash.c  djb2 known-answer vectors, len == 0, embedded NUL,
                  stability, NULL guards
  test_convert_strict.c
                  known-failing (2.1.5): the AKERR_VALUE / ERANGE
                  contract the ato* family cannot express today

test_tree.c gains the BFS AKERR_NOT_IMPLEMENTED contract, NULL arguments,
and a callback error that is not AKERR_ITERATOR_BREAK propagating out.

Tests deliberately say nothing about behaviour TODO.md records as
defective -- unchecked ptr/mode/resolved_path, short transfers reported as
success, *count left at -1, the djb2 sign extension -- so the eventual fix
does not have to come with a test rewrite. Each failure case in
test_path.c passes a zeroed buffer, because the wrapper's own error path
formats resolved_path with %s (2.1.6).

aksl_capture.h gains aksl_temp_file() with an atexit unlink backstop.
Without it every test that fails before its own unlink leaves temp files
behind -- which is the normal case for a known-failing test, and happens
173 times over in a mutation run.

Coverage on src/stdlib.c: 52.0% -> 99.0% of lines (200/202), 23.6% ->
51.0% of branches, 8/21 -> 21/21 functions. The two uncovered lines are
both `} HANDLE(e, AKERR_ITERATOR_BREAK) {`, where the macro starts with
the `break;` of PROCESS's `case 0:` arm -- reachable only via a non-NULL
error context whose status is zero, the pathology 2.2.1 exists to remove.

Mutation score on src/stdlib.c: 46.8% -> 89.6% (155/173 killed). CI, the
pre-push hook and the docs ratchet from 40 to 80 accordingly, and the 18
survivors are grouped by cause in TODO.md and README.md. A new CI
coverage job gates at 90% lines / 45% branches.

Verified:
  ctest --test-dir build            # 12/12
  ctest --test-dir build-asan       # 12/12 under ASan + UBSan
  ctest --test-dir build-coverage   # 14/14, report attached
  ctest --test-dir build -j8 --repeat until-fail:3
  gcc -Wall -Wextra -c on all nine test files  # no warnings
  python3 scripts/mutation_test.py --target src/stdlib.c  # 89.6%
No temp files left in /tmp after any of the above.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 02:07:36 -04:00

106 lines
4.2 KiB
YAML

name: libakstdlib CI Build
run-name: ${{ gitea.actor }} libakstdlib test
on: [push]
jobs:
cmake_build:
runs-on: ubuntu-latest
steps:
- run: echo "Triggered by ${{ gitea.event_name }} from ${{ gitea.repository }}@${{ gitea.ref }}. Building on ${{ runner.os }}."
- name: Check out repository code
uses: actions/checkout@v4
with:
# A top-level build uses deps/libakerror via add_subdirectory, so the
# submodule has to be present or the configure step fails outright.
submodules: recursive
- name: dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y cmake gcc moreutils
# Depends on libakerror@main
git clone https://source.starfort.tech/andrew/libakerror.git
cd libakerror
cmake -S . -B build
cmake --build build
cmake --install build
- name: build and test
run: |
cmake -S . -B build
cmake --build build
sudo cmake --install build
ctest --test-dir build --output-on-failure
- run: echo "🍏 This job's status is ${{ job.status }}."
coverage:
runs-on: ubuntu-latest
steps:
- name: Check out repository code
uses: actions/checkout@v4
with:
submodules: recursive
- name: dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y cmake gcc python3
# scripts/coverage.py needs nothing but python3 and gcc's own gcov, so
# there is no lcov/gcovr to install here.
#
# The gate is a ratchet, not a target: src/stdlib.c is at 99.0% of lines
# and 51.0% of branches, so 90/45 fails on a real regression (a test
# deleted, or new untested code added) without tripping over rounding.
# The report is printed either way -- the uncovered lines it lists are the
# missing tests.
- name: coverage
run: |
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON \
-DAKSL_COVERAGE_THRESHOLD=90 \
-DAKSL_COVERAGE_BRANCH_THRESHOLD=45
cmake --build build-coverage
ctest --test-dir build-coverage --output-on-failure
cat build-coverage/coverage-summary.txt
- run: echo "🍏 This job's status is ${{ job.status }}."
mutation_test:
runs-on: ubuntu-latest
steps:
- name: Check out repository code
uses: actions/checkout@v4
with:
# The harness copies the repo and builds the copy top-level, so it
# needs deps/libakerror present just like the main job does.
submodules: recursive
- name: dependencies
run: |
sudo apt-get update -y
sudo apt-get install -y cmake gcc moreutils python3
# Verify the tests actually catch bugs: break the library many ways and
# confirm the suite fails. Gated on src/stdlib.c (fast, deterministic);
# run the full default target locally for the macro header as well.
#
# The threshold is a ratchet, not a quality bar. The score is 89.6%
# (155/173 killed) now that TODO.md sections 1.2-1.6 have tests; it was
# 46.8% when only the list and tree functions were covered. 80 leaves
# headroom for the runner while still failing on a real regression (tests
# deleted, or new untested code added). The 18 survivors are listed in the
# published report -- each one is a missing assertion.
- name: mutation testing
run: |
python3 scripts/mutation_test.py \
--target src/stdlib.c \
--junit mutation-junit.xml \
--threshold 80
# Publish even when the threshold gate fails, so survivors are visible --
# each one is a missing test. Display-only (fail_on_failure: false); the
# --threshold above is the gate. annotate_only avoids the Checks API 404
# on Gitea (mikepenz/action-junit-report#23).
- name: publish mutation results
if: always()
uses: mikepenz/action-junit-report@v4
with:
report_paths: 'mutation-junit.xml'
annotate_only: true
detailed_summary: true
include_passed: true
fail_on_failure: 'false'
- run: echo "🍏 This job's status is ${{ job.status }}."