Every wrapper outside the list and tree code was untested. Six new test
files close that, following the plan already written in TODO.md 1.2-1.6:
test_stream.c fopen/fread/fwrite/fclose -- happy paths, the round
trip, AKERR_EOF on a short read, AKERR_IO on a stream
opened in the wrong mode, ENOENT, and the NULL guards
test_format.c printf/fprintf/sprintf -- text *and* count asserted
(stdout is pointed at a temp file to check aksl_printf),
all eight NULL guards, EBADF on a read-only stream, and
512 variadic calls in a loop as sanitizer cover for the
missing va_end
test_convert.c ato{i,l,ll,f} happy paths, negatives, leading
whitespace, NULL guards
test_path.c realpath on a file and on a symlink, both compared
against realpath(3) since TMPDIR may itself be a link;
ENOENT, ENOTDIR, NULL path
test_strhash.c djb2 known-answer vectors, len == 0, embedded NUL,
stability, NULL guards
test_convert_strict.c
known-failing (2.1.5): the AKERR_VALUE / ERANGE
contract the ato* family cannot express today
test_tree.c gains the BFS AKERR_NOT_IMPLEMENTED contract, NULL arguments,
and a callback error that is not AKERR_ITERATOR_BREAK propagating out.
Tests deliberately say nothing about behaviour TODO.md records as
defective -- unchecked ptr/mode/resolved_path, short transfers reported as
success, *count left at -1, the djb2 sign extension -- so the eventual fix
does not have to come with a test rewrite. Each failure case in
test_path.c passes a zeroed buffer, because the wrapper's own error path
formats resolved_path with %s (2.1.6).
aksl_capture.h gains aksl_temp_file() with an atexit unlink backstop.
Without it every test that fails before its own unlink leaves temp files
behind -- which is the normal case for a known-failing test, and happens
173 times over in a mutation run.
Coverage on src/stdlib.c: 52.0% -> 99.0% of lines (200/202), 23.6% ->
51.0% of branches, 8/21 -> 21/21 functions. The two uncovered lines are
both `} HANDLE(e, AKERR_ITERATOR_BREAK) {`, where the macro starts with
the `break;` of PROCESS's `case 0:` arm -- reachable only via a non-NULL
error context whose status is zero, the pathology 2.2.1 exists to remove.
Mutation score on src/stdlib.c: 46.8% -> 89.6% (155/173 killed). CI, the
pre-push hook and the docs ratchet from 40 to 80 accordingly, and the 18
survivors are grouped by cause in TODO.md and README.md. A new CI
coverage job gates at 90% lines / 45% branches.
Verified:
ctest --test-dir build # 12/12
ctest --test-dir build-asan # 12/12 under ASan + UBSan
ctest --test-dir build-coverage # 14/14, report attached
ctest --test-dir build -j8 --repeat until-fail:3
gcc -Wall -Wextra -c on all nine test files # no warnings
python3 scripts/mutation_test.py --target src/stdlib.c # 89.6%
No temp files left in /tmp after any of the above.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
106 lines
4.2 KiB
YAML
106 lines
4.2 KiB
YAML
name: libakstdlib CI Build
|
|
run-name: ${{ gitea.actor }} libakstdlib test
|
|
on: [push]
|
|
|
|
jobs:
|
|
cmake_build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- run: echo "Triggered by ${{ gitea.event_name }} from ${{ gitea.repository }}@${{ gitea.ref }}. Building on ${{ runner.os }}."
|
|
- name: Check out repository code
|
|
uses: actions/checkout@v4
|
|
with:
|
|
# A top-level build uses deps/libakerror via add_subdirectory, so the
|
|
# submodule has to be present or the configure step fails outright.
|
|
submodules: recursive
|
|
- name: dependencies
|
|
run: |
|
|
sudo apt-get update -y
|
|
sudo apt-get install -y cmake gcc moreutils
|
|
# Depends on libakerror@main
|
|
git clone https://source.starfort.tech/andrew/libakerror.git
|
|
cd libakerror
|
|
cmake -S . -B build
|
|
cmake --build build
|
|
cmake --install build
|
|
- name: build and test
|
|
run: |
|
|
cmake -S . -B build
|
|
cmake --build build
|
|
sudo cmake --install build
|
|
ctest --test-dir build --output-on-failure
|
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|
|
|
|
coverage:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Check out repository code
|
|
uses: actions/checkout@v4
|
|
with:
|
|
submodules: recursive
|
|
- name: dependencies
|
|
run: |
|
|
sudo apt-get update -y
|
|
sudo apt-get install -y cmake gcc python3
|
|
# scripts/coverage.py needs nothing but python3 and gcc's own gcov, so
|
|
# there is no lcov/gcovr to install here.
|
|
#
|
|
# The gate is a ratchet, not a target: src/stdlib.c is at 99.0% of lines
|
|
# and 51.0% of branches, so 90/45 fails on a real regression (a test
|
|
# deleted, or new untested code added) without tripping over rounding.
|
|
# The report is printed either way -- the uncovered lines it lists are the
|
|
# missing tests.
|
|
- name: coverage
|
|
run: |
|
|
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON \
|
|
-DAKSL_COVERAGE_THRESHOLD=90 \
|
|
-DAKSL_COVERAGE_BRANCH_THRESHOLD=45
|
|
cmake --build build-coverage
|
|
ctest --test-dir build-coverage --output-on-failure
|
|
cat build-coverage/coverage-summary.txt
|
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|
|
|
|
mutation_test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Check out repository code
|
|
uses: actions/checkout@v4
|
|
with:
|
|
# The harness copies the repo and builds the copy top-level, so it
|
|
# needs deps/libakerror present just like the main job does.
|
|
submodules: recursive
|
|
- name: dependencies
|
|
run: |
|
|
sudo apt-get update -y
|
|
sudo apt-get install -y cmake gcc moreutils python3
|
|
# Verify the tests actually catch bugs: break the library many ways and
|
|
# confirm the suite fails. Gated on src/stdlib.c (fast, deterministic);
|
|
# run the full default target locally for the macro header as well.
|
|
#
|
|
# The threshold is a ratchet, not a quality bar. The score is 89.6%
|
|
# (155/173 killed) now that TODO.md sections 1.2-1.6 have tests; it was
|
|
# 46.8% when only the list and tree functions were covered. 80 leaves
|
|
# headroom for the runner while still failing on a real regression (tests
|
|
# deleted, or new untested code added). The 18 survivors are listed in the
|
|
# published report -- each one is a missing assertion.
|
|
- name: mutation testing
|
|
run: |
|
|
python3 scripts/mutation_test.py \
|
|
--target src/stdlib.c \
|
|
--junit mutation-junit.xml \
|
|
--threshold 80
|
|
# Publish even when the threshold gate fails, so survivors are visible --
|
|
# each one is a missing test. Display-only (fail_on_failure: false); the
|
|
# --threshold above is the gate. annotate_only avoids the Checks API 404
|
|
# on Gitea (mikepenz/action-junit-report#23).
|
|
- name: publish mutation results
|
|
if: always()
|
|
uses: mikepenz/action-junit-report@v4
|
|
with:
|
|
report_paths: 'mutation-junit.xml'
|
|
annotate_only: true
|
|
detailed_summary: true
|
|
include_passed: true
|
|
fail_on_failure: 'false'
|
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|