Document the test harnesses and add a pre-push hook
All checks were successful
libakstdlib CI Build / cmake_build (push) Successful in 2m46s
libakstdlib CI Build / mutation_test (push) Successful in 7m48s

README.md was a build badge and nothing else. It now covers what the library
is, how to build it, and how to run each of the three harnesses locally: the
ctest suite, the AKSL_SANITIZE ASan/UBSan build, and mutation testing.

The section on reading ctest output matters most: two of the three test lists
are marked WILL_FAIL, so an all-green run does not mean the library is
defect-free -- it means the known-good tests passed and the known-bad ones are
still failing in the documented way.

.githooks/pre-push runs the two fast harnesses (default build + ctest, then
sanitizer build + ctest) before a push leaves the machine. It skips when there
is nothing to test (branch deletions, empty pushes), fails the push with the
captured output on any error, and builds under .git/aksl-prepush so it never
disturbs your own build/. The slow mutation gate is opt-in via
AKSL_HOOK_MUTATION=1, and git push --no-verify bypasses everything.

Enable with: git config core.hooksPath .githooks

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-29 11:51:28 -04:00
parent 68009ea0e3
commit 54fb44cd2b
2 changed files with 229 additions and 0 deletions

129
README.md
View File

@@ -1,3 +1,132 @@
# README
![build badge](https://source.starfort.tech/andrew/libakstdlib/actions/workflows/ci.yaml/badge.svg?branch=main)
`libakstdlib` wraps C standard library functions so that they report failures
through [libakerror](https://source.starfort.tech/andrew/libakerror)'s
`ATTEMPT { ... } HANDLE { ... }` error contexts instead of through return codes
and `errno`. It also provides a few data structures built on the same
convention (a doubly-linked list and a binary tree).
Every entry point returns `akerr_ErrorContext *` and is marked `AKERR_NOIGNORE`.
See `TODO.md` for the current state of the library: what is covered by tests,
which corner cases are still open, and which libc functions are not yet wrapped.
## Building
```sh
git submodule update --init --recursive # deps/libakerror
cmake -S . -B build
cmake --build build
cmake --install build
```
A top-level build compiles the vendored `deps/libakerror`. When `libakstdlib` is
consumed as a subproject, it uses whatever `akerror::akerror` target or installed
package the parent provides instead.
## Testing
There are three harnesses. The first two take seconds; the third takes about
half an hour.
### 1. The test suite
```sh
cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure
```
Tests live one per file in `tests/test_<name>.c` and share the helpers in
`tests/aksl_capture.h``AKSL_CHECK()` for plain assertions (unlike `assert()`
it survives `-DNDEBUG`), `AKSL_CHECK_STATUS(call, expected)` to run a wrapper and
assert on the status it returns, and an `AKSL_RUN()` driver that additionally
fails any test which leaks a slot from libakerror's error pool.
To add a test, drop `tests/test_mything.c` in place and add `mything` to
`AKSL_TESTS` in `CMakeLists.txt`.
**Reading the results.** `CMakeLists.txt` splits tests into three lists, and two
of them invert the meaning of "Passed":
| List | Meaning |
|---|---|
| `AKSL_TESTS` | Ordinary tests. Must exit 0. |
| `AKSL_WILL_FAIL_TESTS` | Expected to abort by design — an unhandled error reaching `FINISH_NORETURN`, or a deliberate contract violation. Marked `WILL_FAIL`, so a non-zero exit is a pass. |
| `AKSL_KNOWN_FAILING_TESTS` | Assert the *correct* behaviour of a confirmed defect (see `TODO.md` §2.1). Also marked `WILL_FAIL`. |
So `ctest` reporting all green does **not** mean the library is defect-free — it
means the known-good tests passed and the known-bad ones are still failing in the
documented way. When a defect is fixed, its test starts passing, CTest reports it
as failed with *unexpectedly passed*, and that is the cue to move it from
`AKSL_KNOWN_FAILING_TESTS` into `AKSL_TESTS`.
Every test is capped with a 30-second CTest `TIMEOUT`. The list and tree code is
full of loops whose termination hangs on a single condition, so a bug of that
shape hangs the suite rather than failing it.
### 2. Sanitizers
```sh
cmake -S . -B build-asan -DAKSL_SANITIZE=ON
cmake --build build-asan
ctest --test-dir build-asan --output-on-failure
```
Builds the library, the tests and the vendored libakerror with ASan + UBSan and
`-fno-sanitize-recover=all`. Several of the open items in `TODO.md` §2 only
misbehave under instrumentation — the uninitialised `%s` in `aksl_realpath`, the
unbounded `vsprintf` behind `aksl_sprintf`, the missing `va_end` in the `printf`
family — so new tests for those should be run this way.
### 3. Mutation testing
The suite tells you the library works. Mutation testing tells you the *suite*
works: it breaks the library in small ways, one at a time, and checks that the
tests notice.
```sh
cmake --build build --target mutation # src/stdlib.c + include/akstdlib.h
```
or drive the script directly for a faster or narrower run:
```sh
scripts/mutation_test.py --target src/stdlib.c # C source only
scripts/mutation_test.py --target src/stdlib.c --list # enumerate, build nothing
scripts/mutation_test.py --target src/stdlib.c --max-mutants 20
scripts/mutation_test.py --target src/stdlib.c --threshold 40
```
A mutant that makes the tests fail is *killed* (good); one the tests still pass
is a *survivor*, and names a missing test. The score is `killed / total`, and the
run prints every survivor with `file:line` and the exact edit. The harness never
touches your working tree — it copies the repo to a scratch directory and mutates
the copy.
CI runs the `src/stdlib.c` set with `--threshold 40`. That is a regression
ratchet rather than a quality bar: the current score is 46.8%, and the survivors
are concentrated in the wrappers that have no tests yet. Raise the threshold as
coverage lands.
## The pre-push hook
`.githooks/pre-push` runs the fast harnesses — the default build and the
sanitizer build, each followed by `ctest` — before letting a push out. Enable it
once per clone:
```sh
git config core.hooksPath .githooks
```
It only builds when there are commits to push (a branch deletion is a no-op), and
it builds under `.git/aksl-prepush` so it never disturbs your own `build/`.
```sh
AKSL_HOOK_MUTATION=1 git push # also run the mutation gate (slow)
git push --no-verify # skip the hook entirely
```
Other knobs: `AKSL_MUTATION_THRESHOLD` (default 40, keep it in step with
`.gitea/workflows/ci.yaml`) and `AKSL_HOOK_BUILD_DIR`.