Tachikoma
f60787a852
Bump deps/libakerror 22 commits to 5ff8790 (1.0.0), which makes the status-name table private, moves consumer status codes to a band starting at AKERR_FIRST_CONSUMER_STATUS, enforces range ownership rather than treating it as advisory, and gives the library an soname. See deps/libakerror/UPGRADING.md. src/stdlib.c needed no changes. This library defines no status codes of its own -- it raises libakerror's AKERR_* codes and propagates errno, both inside libakerror's reserved 0-255 band -- and it never referenced AKERR_MAX_ERR_VALUE, __AKERR_ERROR_NAMES, AKERR_STATUS_RANGE_OK or AKERR_STATUS_NAME_OK. What moved was everything around the code: A -DAKSL_COVERAGE=ON build stopped configuring at all. libakerror namespaces its `mutation` target when embedded but not its `coverage` target, so it collided with ours. Shadow add_custom_target for the duration of the add_subdirectory() call and rename the dependency's to akerror_coverage, alongside the existing add_test shadow. Fix upstream and delete the workaround; recorded in TODO.md. Pin the 1.0.0 floor three ways, since no single one covers every consumption path: an #error in akstdlib.h feature-testing AKERR_FIRST_CONSUMER_STATUS, because libakerror publishes no version macro; Requires: akerror >= 1.0.0 in akstdlib.pc, which also gets consumers -lakerror transitively; and find_dependency(akerror) in akstdlibConfig.cmake. The last was already broken before this bump -- the template still carried its MyLibraryConfig placeholder with the dependency commented out, so any external find_package(akstdlib) failed with a bare "akerror::akerror not found" out of the generated targets file. Branch coverage of src/stdlib.c fell from 51.0% to 44.3% with no source or test change: the 1.0.0 PREPARE_ERROR/FAIL_* macros expand to more branches at every call site, so 337/661 became 481/1087 -- 144 more branches covered, 426 more counted. Line coverage held at 99.0% (200/202) and function coverage at 100% (21/21). Re-ratchet the CI branch gate 45 -> 40 rather than chase branches that belong to libakerror's own suite. tests/test_status_registry.c pins the contract that made the status-code migration a no-op: libakstdlib reserves no consumer range, so an application may allocate from AKERR_FIRST_CONSUMER_STATUS without coordinating with it, and every status this library raises is inside the reserved band with a name actually registered -- an unnamed one degrades to "Unknown Error" in every later stack trace, which nothing else would notice. It exercises the new ownership enforcement too, so the "reserves nothing" assertion cannot pass vacuously. ctest 13/13, ASan+UBSan 13/13, coverage 15/15 at 90/40, mutation 89.6% (155/173, unchanged). Also verified out of tree: the #error fires as the first diagnostic against a stale akerror.h, pkg-config refuses akerror 0.9.0, and an external find_package(akstdlib) consumer builds and runs against a temp-prefix install. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-Authored-By: Andrew Kesterson <andrew@aklabs.net>
273 lines
13 KiB
Markdown
273 lines
13 KiB
Markdown
# README
|
||
|
||
|
||
|
||
`libakstdlib` wraps C standard library functions so that they report failures
|
||
through [libakerror](https://source.starfort.tech/andrew/libakerror)'s
|
||
`ATTEMPT { ... } HANDLE { ... }` error contexts instead of through return codes
|
||
and `errno`. It also provides a few data structures built on the same
|
||
convention (a doubly-linked list and a binary tree).
|
||
|
||
Every entry point returns `akerr_ErrorContext *` and is marked `AKERR_NOIGNORE`.
|
||
See `TODO.md` for the current state of the library: what is covered by tests,
|
||
which corner cases are still open, and which libc functions are not yet wrapped.
|
||
|
||
## Building
|
||
|
||
```sh
|
||
git submodule update --init --recursive # deps/libakerror
|
||
cmake -S . -B build
|
||
cmake --build build
|
||
cmake --install build
|
||
```
|
||
|
||
A top-level build compiles the vendored `deps/libakerror`. When `libakstdlib` is
|
||
consumed as a subproject, it uses whatever `akerror::akerror` target or installed
|
||
package the parent provides instead.
|
||
|
||
### The libakerror version floor
|
||
|
||
**libakerror 1.0.0 or newer is required.** That release made the status-name
|
||
table private, moved consumer status codes into a band starting at
|
||
`AKERR_FIRST_CONSUMER_STATUS` (256), made range ownership enforced rather than
|
||
advisory, and gave the library an soname — see
|
||
`deps/libakerror/UPGRADING.md`. It is a source *and* ABI break, so pairing this
|
||
header with an older `akerror.h` is not a compile problem you can work around;
|
||
the pairing is simply invalid.
|
||
|
||
Three things enforce the floor, because no single one covers every way the
|
||
library gets consumed:
|
||
|
||
| Mechanism | Where | Catches |
|
||
| --- | --- | --- |
|
||
| `#error` on a missing `AKERR_FIRST_CONSUMER_STATUS` | `include/akstdlib.h` | a stale `akerror.h` earlier on the include path, at the first diagnostic rather than as a pile of errors inside `src/stdlib.c` |
|
||
| `Requires: akerror >= 1.0.0` | `akstdlib.pc.in` | a pkg-config consumer, which also now gets `-lakerror` transitively |
|
||
| `find_dependency(akerror)` | `cmake/akstdlib.cmake.in` | a `find_package(akstdlib)` consumer, which previously failed with a bare *"akerror::akerror not found"* out of the generated targets file |
|
||
|
||
The header guard feature-tests rather than version-tests because libakerror
|
||
publishes no version macro; `AKERR_FIRST_CONSUMER_STATUS` is the symbol 1.0.0
|
||
introduced, so its absence is what "older than 1.0.0" actually looks like. The
|
||
CMake path requests no version for the same kind of reason: libakerror installs
|
||
no `akerrorConfigVersion.cmake`, so `find_dependency(akerror 1.0.0)` would be
|
||
refused for want of a version file no matter which akerror is installed.
|
||
|
||
**libakstdlib defines no status codes of its own.** It raises libakerror's
|
||
`AKERR_*` codes and propagates the host's `errno` values, all of which live in
|
||
libakerror's reserved `0`–`255` band, so it reserves no range and an application
|
||
is free to allocate from `AKERR_FIRST_CONSUMER_STATUS` without coordinating with
|
||
it. `tests/test_status_registry.c` pins that, along with the requirement that
|
||
every status this library raises actually has a name registered — an unnamed one
|
||
degrades to `"Unknown Error"` in every later stack trace, which nothing else
|
||
would notice.
|
||
|
||
## Testing
|
||
|
||
There are four harnesses. The first three take seconds; the fourth takes about
|
||
half an hour.
|
||
|
||
### 1. The test suite
|
||
|
||
```sh
|
||
cmake -S . -B build
|
||
cmake --build build
|
||
ctest --test-dir build --output-on-failure
|
||
```
|
||
|
||
Tests live one per file in `tests/test_<name>.c` and share the helpers in
|
||
`tests/aksl_capture.h` — `AKSL_CHECK()` for plain assertions (unlike `assert()`
|
||
it survives `-DNDEBUG`), `AKSL_CHECK_STATUS(call, expected)` to run a wrapper and
|
||
assert on the status it returns, `aksl_temp_file()` for tests that need a real
|
||
file to work on, and an `AKSL_RUN()` driver that additionally fails any test which
|
||
leaks a slot from libakerror's error pool.
|
||
|
||
One file per area of the API: `convert` (the `ato*` family), `format` (the
|
||
`printf` family), `stream` (`fopen`/`fread`/`fwrite`/`fclose`), `path`
|
||
(`aksl_realpath`), `strhash`, `memory`, `linkedlist`, `tree`, and
|
||
`status_registry` (this library's side of the libakerror status-registry
|
||
contract — see "The libakerror version floor" above).
|
||
|
||
To add a test, drop `tests/test_mything.c` in place and add `mything` to
|
||
`AKSL_TESTS` in `CMakeLists.txt`.
|
||
|
||
**Reading the results.** `CMakeLists.txt` splits tests into three lists, and two
|
||
of them invert the meaning of "Passed":
|
||
|
||
| List | Meaning |
|
||
|---|---|
|
||
| `AKSL_TESTS` | Ordinary tests. Must exit 0. |
|
||
| `AKSL_WILL_FAIL_TESTS` | Expected to abort by design — an unhandled error reaching `FINISH_NORETURN`, or a deliberate contract violation. Marked `WILL_FAIL`, so a non-zero exit is a pass. |
|
||
| `AKSL_KNOWN_FAILING_TESTS` | Assert the *correct* behaviour of a confirmed defect (see `TODO.md` §2.1). Also marked `WILL_FAIL`. |
|
||
|
||
So `ctest` reporting all green does **not** mean the library is defect-free — it
|
||
means the known-good tests passed and the known-bad ones are still failing in the
|
||
documented way. When a defect is fixed, its test starts passing, CTest reports it
|
||
as failed with *unexpectedly passed*, and that is the cue to move it from
|
||
`AKSL_KNOWN_FAILING_TESTS` into `AKSL_TESTS`.
|
||
|
||
Every test is capped with a 30-second CTest `TIMEOUT`. The list and tree code is
|
||
full of loops whose termination hangs on a single condition, so a bug of that
|
||
shape hangs the suite rather than failing it.
|
||
|
||
### 2. Sanitizers
|
||
|
||
```sh
|
||
cmake -S . -B build-asan -DAKSL_SANITIZE=ON
|
||
cmake --build build-asan
|
||
ctest --test-dir build-asan --output-on-failure
|
||
```
|
||
|
||
Builds the library, the tests and the vendored libakerror with ASan + UBSan and
|
||
`-fno-sanitize-recover=all`. Several of the open items in `TODO.md` §2 only
|
||
misbehave under instrumentation — the uninitialised `%s` in `aksl_realpath`, the
|
||
unbounded `vsprintf` behind `aksl_sprintf`, the missing `va_end` in the `printf`
|
||
family — so new tests for those should be run this way.
|
||
|
||
### 3. Code coverage
|
||
|
||
```sh
|
||
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON
|
||
cmake --build build-coverage --target coverage
|
||
```
|
||
|
||
`-DAKSL_COVERAGE=ON` compiles the library and the tests with `--coverage -O0`,
|
||
and wires the report into the suite itself, so a plain
|
||
`ctest --test-dir build-coverage` also produces it. Two extra CTest entries
|
||
appear, held in place by a CTest fixture rather than by declaration order, so
|
||
they work under `ctest -j` too:
|
||
|
||
| Test | When | Does |
|
||
|---|---|---|
|
||
| `coverage_reset` | before every other test | deletes the accumulated `.gcda` counters |
|
||
| `coverage_report` | after every other test | aggregates `gcov` output, prints the summary, applies the threshold gate |
|
||
|
||
The reset matters: gcov counters are cumulative, so without it each report would
|
||
fold in every earlier run and overstate coverage.
|
||
|
||
`coverage` here is this project's target. libakerror ships a `coverage` target of
|
||
its own and, unlike its `mutation` target, does not namespace it when embedded,
|
||
so a top-level `-DAKSL_COVERAGE=ON` build would collide on the name and fail to
|
||
configure at all. `CMakeLists.txt` renames the dependency's to `akerror_coverage`
|
||
on the way past — it drives its own instrumented build tree, so
|
||
`cmake --build build-coverage --target akerror_coverage` still works. The
|
||
workaround goes away when libakerror namespaces it upstream; see `TODO.md` §2.3.
|
||
|
||
CTest hides the output of a passing test, so `coverage_report` also writes
|
||
`build-coverage/coverage-summary.txt` (the same text report) and
|
||
`build-coverage/coverage.xml` (Cobertura, for CI publishers). The `coverage`
|
||
target above prints the report to the terminal for you; otherwise read the file
|
||
or use `ctest --test-dir build-coverage -V -R coverage_report`.
|
||
|
||
The report lists per-file line, branch and function coverage, then every
|
||
uncovered line and every function the suite never called — that listing is the
|
||
actionable part, the same way surviving mutants are for the harness below.
|
||
|
||
Drive the script directly for anything narrower:
|
||
|
||
```sh
|
||
scripts/coverage.py --build build-coverage # report on disk counters
|
||
scripts/coverage.py --build build-coverage --summary-only # totals only
|
||
scripts/coverage.py --build build-coverage --include tests # coverage of the tests themselves
|
||
scripts/coverage.py --build build-coverage --run-tests # reset, run ctest, report
|
||
scripts/coverage.py --build build-coverage --threshold 90 --branch-threshold 40
|
||
```
|
||
|
||
It needs nothing but Python 3 and gcc's own `gcov` — no lcov, gcovr or genhtml.
|
||
|
||
To gate on coverage, set the threshold at configure time; `coverage_report` then
|
||
fails below it, and the same regression-ratchet logic applies as for the mutation
|
||
score:
|
||
|
||
```sh
|
||
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON \
|
||
-DAKSL_COVERAGE_THRESHOLD=90 -DAKSL_COVERAGE_BRANCH_THRESHOLD=40
|
||
```
|
||
|
||
**Where it stands.** `src/stdlib.c` is at **99.0% of lines (200/202)**, **44.3% of
|
||
branches (481/1087)** and **21/21 functions**, so 90/40 above is a ratchet with
|
||
headroom rather than a target. The two uncovered lines are both
|
||
`} HANDLE(e, AKERR_ITERATOR_BREAK) {` — in libakerror that macro begins with the
|
||
`break;` belonging to `PROCESS`'s `case 0:` arm, which is only reachable when a
|
||
callback returns a non-NULL error context whose status is *zero*. That is the
|
||
pathological case §2.2.1 of `TODO.md` exists to remove, so it is left uncovered
|
||
deliberately rather than pinned by a test.
|
||
|
||
Branch coverage sits far below line coverage because most branches in this file
|
||
are inside the `FAIL_*`/`ATTEMPT`/`FINISH` macro expansions — pool exhaustion,
|
||
stack-trace buffer limits, `akerr_valid_error_address` failures — and belong to
|
||
libakerror's own suite rather than to this one. The libakerror 1.0.0 bump made
|
||
that gap wider without changing a line of this library: the branch denominator
|
||
went from 661 to 1087 as those macros grew, so the same tests that scored 51.0%
|
||
(337/661) now score 44.3% (481/1087). The gate moved 45 → 40 to match; line and
|
||
function coverage did not move at all.
|
||
|
||
Two caveats. Coverage is measured at `-O0`, because the optimizer reorders lines
|
||
until per-line counts stop matching the source — so a coverage build is not the
|
||
build to profile. And gcov flushes its counters at normal process exit, which an
|
||
`AKSL_WILL_FAIL_TESTS` entry that aborts by design never reaches: such a test
|
||
contributes no coverage data at all, so lines only it reaches are reported as
|
||
uncovered.
|
||
|
||
### 4. Mutation testing
|
||
|
||
The suite tells you the library works. Mutation testing tells you the *suite*
|
||
works: it breaks the library in small ways, one at a time, and checks that the
|
||
tests notice.
|
||
|
||
```sh
|
||
cmake --build build --target mutation # src/stdlib.c + include/akstdlib.h
|
||
```
|
||
|
||
or drive the script directly for a faster or narrower run:
|
||
|
||
```sh
|
||
scripts/mutation_test.py --target src/stdlib.c # C source only
|
||
scripts/mutation_test.py --target src/stdlib.c --list # enumerate, build nothing
|
||
scripts/mutation_test.py --target src/stdlib.c --max-mutants 20
|
||
scripts/mutation_test.py --target src/stdlib.c --threshold 80
|
||
```
|
||
|
||
A mutant that makes the tests fail is *killed* (good); one the tests still pass
|
||
is a *survivor*, and names a missing test. The score is `killed / total`, and the
|
||
run prints every survivor with `file:line` and the exact edit. The harness never
|
||
touches your working tree — it copies the repo to a scratch directory and mutates
|
||
the copy.
|
||
|
||
CI runs the `src/stdlib.c` set with `--threshold 80`. That is a regression ratchet
|
||
rather than a quality bar: the current score is **89.6% (155/173 killed)**, up
|
||
from 46.8% before the wrapper tests landed. Raise the threshold as the remaining
|
||
survivors are turned into assertions.
|
||
|
||
The 18 survivors cluster in three places, and each names a real gap rather than a
|
||
test-harness artifact:
|
||
|
||
- **Statements whose absence nothing observes** — deleting `free(ptr)`,
|
||
`obj->next = NULL`, or a `SUCCEED_RETURN` leaves behaviour the suite does not
|
||
look at (a leak, a stale pointer, a success that was already NULL).
|
||
- **The `aksl_list_append` cycle/tail walk** (`tail = slow`, `slow = slow->next`,
|
||
`tail = fast`) — the function is broken in exactly this area (`TODO.md` §2.1.1),
|
||
so its known-failing test cannot pin the internals yet.
|
||
- **`lalloc`/`lfree` defaulting in `aksl_tree_iterate`** — dead parameters
|
||
(§2.2.8): they are defaulted and then never called, so inverting the guard
|
||
changes nothing observable.
|
||
|
||
## The pre-push hook
|
||
|
||
`.githooks/pre-push` runs the fast harnesses — the default build and the
|
||
sanitizer build, each followed by `ctest` — before letting a push out. Enable it
|
||
once per clone:
|
||
|
||
```sh
|
||
git config core.hooksPath .githooks
|
||
```
|
||
|
||
It only builds when there are commits to push (a branch deletion is a no-op), and
|
||
it builds under `.git/aksl-prepush` so it never disturbs your own `build/`.
|
||
|
||
```sh
|
||
AKSL_HOOK_MUTATION=1 git push # also run the mutation gate (slow)
|
||
git push --no-verify # skip the hook entirely
|
||
```
|
||
|
||
Other knobs: `AKSL_MUTATION_THRESHOLD` (default 80, keep it in step with
|
||
`.gitea/workflows/ci.yaml`) and `AKSL_HOOK_BUILD_DIR`.
|