Andrew Kesterson
fd71bcc67b
akbasic is a ~6,300-line C interpreter built on this library. It is the first
consumer to exercise the whole surface rather than a corner of it, so what it
could not use is worth writing down: it prioritizes the section 3 wishlist by
what a real port actually reaches for, and it says which section 2 entries have
teeth.
The number that matters: across src/, akbasic makes 10 calls into this library
and 116 to raw libc. A library whose value proposition is turning silent libc
failures into error contexts is being bypassed 92% of the time by the consumer
most committed to it.
Four things it had to write for itself, each of which maps onto an existing
unchecked box. A strict strtoll/strtod wrapper, because 2.1.5 means aksl_atoi
would have turned four diagnosable errors into wrong answers -- VAL("garbage")
answering 0.0 instead of raising. A fixed-capacity string-keyed hash table,
which is 3.6's "hash map built on aksl_strhash_djb2", needed three times over
for variables, functions and labels. The bounded-copy-with-truncation-as-error
idiom at ten sites across six files. And case folding at three sites.
Also confirms 2.2.4 (aksl_sprintf unbounded -- akbasic deliberately never calls
it, and has 28 snprintf sites with no aksl_snprintf to route them through),
2.2.2 (aksl_fopen validation, reached from user input via DLOAD/DSAVE), 2.2.6
(the djb2 sign extension, benign here only because BASIC identifiers are ASCII)
and 2.1.4 (the missing va_end, which akbasic's text sink runs on every line of
program output).
Section 4.3 records what akbasic did *not* need, so the wishlist does not get
reordered purely by one consumer's shape: it allocates nothing and uses no lists
or trees, so a consumer that does allocate would weight 3.1's memory section far
higher.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
README
libakstdlib wraps C standard library functions so that they report failures
through libakerror's
ATTEMPT { ... } HANDLE { ... } error contexts instead of through return codes
and errno. It also provides a few data structures built on the same
convention (a doubly-linked list and a binary tree).
Every entry point returns akerr_ErrorContext * and is marked AKERR_NOIGNORE.
See TODO.md for the current state of the library: what is covered by tests,
which corner cases are still open, and which libc functions are not yet wrapped.
Building
git submodule update --init --recursive # deps/libakerror
cmake -S . -B build
cmake --build build
cmake --install build
A top-level build compiles the vendored deps/libakerror. When libakstdlib is
consumed as a subproject, it uses whatever akerror::akerror target or installed
package the parent provides instead.
This library's own version
libakstdlib is at 0.1.0. The version lives in exactly one place — the
project() call in CMakeLists.txt — and flows from there into everything
else, so a bump is a one-line edit:
| Artifact | Value at 0.1.0 | From |
|---|---|---|
AKSL_VERSION_MAJOR / _MINOR / _PATCH |
0 / 1 / 0 |
include/akstdlib_version.h.in |
AKSL_VERSION_STRING |
"0.1.0" |
same |
AKSL_VERSION_NUMBER |
100 |
same |
AKSL_VERSION_SONAME |
"0.1" |
same |
| shared library | libakstdlib.so.0.1.0, soname libakstdlib.so.0.1 |
VERSION / SOVERSION |
pkg-config --modversion akstdlib |
0.1.0 |
akstdlib.pc |
find_package(akstdlib 0.1) |
accepted; 0.2 and 1.0 refused |
akstdlibConfigVersion.cmake |
akstdlib_version.h is generated — that is why there is no such file in the
source tree, only the .in template beside akstdlib.h. Don't hand-edit the
copy in your build directory; change the template or project().
It is 0.x deliberately. TODO.md §2.1 still records four confirmed defects
whose fixes change documented behaviour, so the API is not being promised yet.
While the major version is 0, the soname carries MAJOR.MINOR: 0.1 and 0.2
are different ABIs and the loader will not substitute one for the other. At 1.0
the soname becomes MAJOR alone — the if(PROJECT_VERSION_MAJOR EQUAL 0) in
CMakeLists.txt and the matching #if in tests/test_version.c are the two
places that encode this, and they are tested against each other.
AKSL_VERSION_NUMBER is computed rather than written as a literal, because a
literal 000100 is octal in C and would make 0.1.0 compare as 64.
Compiled-against vs. loaded
The macros above record what a caller was compiled against. What it actually loaded is a different question, and the two can disagree:
int major, minor, patch;
akerr_ErrorContext *e = aksl_version(&major, &minor, &patch); /* the loaded .so */
const char *v = aksl_version_string(); /* likewise */
e = AKSL_VERSION_CHECK(); /* compares the two; AKERR_VALUE on a mismatch */
AKSL_VERSION_CHECK() is a macro on purpose: it expands at your call site, so
it captures the AKSL_VERSION_* you were built with and passes them to a
function that compares against the values baked into the library. Calling
aksl_version_check() with hand-written numbers defeats the whole mechanism.
Compatibility is defined as "same soname", so pre-1.0 both major and minor must match and the patch level is ignored — a caller built against 0.1.0 keeps working against 0.1.7, which is exactly the promise the shared soname makes.
In normal use the soname catches the mismatch first, at load time, and the check never fires. It earns its keep when the soname is bypassed: a hand-install that drops a 0.2.0 build in under the 0.1 filename, or a package that strips versioning. Then the loader is happy and only the check notices:
compiled against : 0.1.0 (soname 0.1)
loaded : 0.2.0 (0.2.0)
MISMATCH DETECTED: compiled against libakstdlib 0.1.0, loaded 0.2.0 (soname 0.2)
The libakerror version floor
libakerror 1.0.0 or newer is required. That release made the status-name
table private, moved consumer status codes into a band starting at
AKERR_FIRST_CONSUMER_STATUS (256), made range ownership enforced rather than
advisory, and gave the library an soname — see
deps/libakerror/UPGRADING.md. It is a source and ABI break, so pairing this
header with an older akerror.h is not a compile problem you can work around;
the pairing is simply invalid.
Three things enforce the floor, because no single one covers every way the library gets consumed:
| Mechanism | Where | Catches |
|---|---|---|
#error on a missing AKERR_FIRST_CONSUMER_STATUS |
include/akstdlib.h |
a stale akerror.h earlier on the include path, at the first diagnostic rather than as a pile of errors inside src/stdlib.c |
Requires: akerror >= 1.0.0 |
akstdlib.pc.in |
a pkg-config consumer, which also now gets -lakerror transitively |
find_dependency(akerror) |
cmake/akstdlib.cmake.in |
a find_package(akstdlib) consumer, which previously failed with a bare "akerror::akerror not found" out of the generated targets file |
The header guard feature-tests rather than version-tests because libakerror
publishes no version macro; AKERR_FIRST_CONSUMER_STATUS is the symbol 1.0.0
introduced, so its absence is what "older than 1.0.0" actually looks like. The
CMake path requests no version for the same kind of reason: libakerror installs
no akerrorConfigVersion.cmake, so find_dependency(akerror 1.0.0) would be
refused for want of a version file no matter which akerror is installed.
libakstdlib defines no status codes of its own. It raises libakerror's
AKERR_* codes and propagates the host's errno values, all of which live in
libakerror's reserved 0–255 band, so it reserves no range and an application
is free to allocate from AKERR_FIRST_CONSUMER_STATUS without coordinating with
it. tests/test_status_registry.c pins that, along with the requirement that
every status this library raises actually has a name registered — an unnamed one
degrades to "Unknown Error" in every later stack trace, which nothing else
would notice.
Testing
There are four harnesses. The first three take seconds; the fourth takes about half an hour.
1. The test suite
cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure
Tests live one per file in tests/test_<name>.c and share the helpers in
tests/aksl_capture.h — AKSL_CHECK() for plain assertions (unlike assert()
it survives -DNDEBUG), AKSL_CHECK_STATUS(call, expected) to run a wrapper and
assert on the status it returns, aksl_temp_file() for tests that need a real
file to work on, and an AKSL_RUN() driver that additionally fails any test which
leaks a slot from libakerror's error pool.
One file per area of the API: convert (the ato* family), format (the
printf family), stream (fopen/fread/fwrite/fclose), path
(aksl_realpath), strhash, memory, linkedlist, tree, and
status_registry (this library's side of the libakerror status-registry
contract — see "The libakerror version floor" above).
To add a test, drop tests/test_mything.c in place and add mything to
AKSL_TESTS in CMakeLists.txt.
Reading the results. CMakeLists.txt splits tests into three lists, and two
of them invert the meaning of "Passed":
| List | Meaning |
|---|---|
AKSL_TESTS |
Ordinary tests. Must exit 0. |
AKSL_WILL_FAIL_TESTS |
Expected to abort by design — an unhandled error reaching FINISH_NORETURN, or a deliberate contract violation. Marked WILL_FAIL, so a non-zero exit is a pass. |
AKSL_KNOWN_FAILING_TESTS |
Assert the correct behaviour of a confirmed defect (see TODO.md §2.1). Also marked WILL_FAIL. |
So ctest reporting all green does not mean the library is defect-free — it
means the known-good tests passed and the known-bad ones are still failing in the
documented way. When a defect is fixed, its test starts passing, CTest reports it
as failed with unexpectedly passed, and that is the cue to move it from
AKSL_KNOWN_FAILING_TESTS into AKSL_TESTS.
Every test is capped with a 30-second CTest TIMEOUT. The list and tree code is
full of loops whose termination hangs on a single condition, so a bug of that
shape hangs the suite rather than failing it.
2. Sanitizers
cmake -S . -B build-asan -DAKSL_SANITIZE=ON
cmake --build build-asan
ctest --test-dir build-asan --output-on-failure
Builds the library, the tests and the vendored libakerror with ASan + UBSan and
-fno-sanitize-recover=all. Several of the open items in TODO.md §2 only
misbehave under instrumentation — the uninitialised %s in aksl_realpath, the
unbounded vsprintf behind aksl_sprintf, the missing va_end in the printf
family — so new tests for those should be run this way.
3. Code coverage
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON
cmake --build build-coverage --target coverage
-DAKSL_COVERAGE=ON compiles the library and the tests with --coverage -O0,
and wires the report into the suite itself, so a plain
ctest --test-dir build-coverage also produces it. Two extra CTest entries
appear, held in place by a CTest fixture rather than by declaration order, so
they work under ctest -j too:
| Test | When | Does |
|---|---|---|
coverage_reset |
before every other test | deletes the accumulated .gcda counters |
coverage_report |
after every other test | aggregates gcov output, prints the summary, applies the threshold gate |
The reset matters: gcov counters are cumulative, so without it each report would fold in every earlier run and overstate coverage.
coverage here is this project's target. libakerror ships a coverage target of
its own and, unlike its mutation target, does not namespace it when embedded,
so a top-level -DAKSL_COVERAGE=ON build would collide on the name and fail to
configure at all. CMakeLists.txt renames the dependency's to akerror_coverage
on the way past — it drives its own instrumented build tree, so
cmake --build build-coverage --target akerror_coverage still works. The
workaround goes away when libakerror namespaces it upstream; see TODO.md §2.3.
CTest hides the output of a passing test, so coverage_report also writes
build-coverage/coverage-summary.txt (the same text report) and
build-coverage/coverage.xml (Cobertura, for CI publishers). The coverage
target above prints the report to the terminal for you; otherwise read the file
or use ctest --test-dir build-coverage -V -R coverage_report.
The report lists per-file line, branch and function coverage, then every uncovered line and every function the suite never called — that listing is the actionable part, the same way surviving mutants are for the harness below.
Drive the script directly for anything narrower:
scripts/coverage.py --build build-coverage # report on disk counters
scripts/coverage.py --build build-coverage --summary-only # totals only
scripts/coverage.py --build build-coverage --include tests # coverage of the tests themselves
scripts/coverage.py --build build-coverage --run-tests # reset, run ctest, report
scripts/coverage.py --build build-coverage --threshold 90 --branch-threshold 40
It needs nothing but Python 3 and gcc's own gcov — no lcov, gcovr or genhtml.
To gate on coverage, set the threshold at configure time; coverage_report then
fails below it, and the same regression-ratchet logic applies as for the mutation
score:
cmake -S . -B build-coverage -DAKSL_COVERAGE=ON \
-DAKSL_COVERAGE_THRESHOLD=90 -DAKSL_COVERAGE_BRANCH_THRESHOLD=40
Where it stands. src/stdlib.c is at 99.1% of lines (217/219), 45.1% of
branches (519/1151) and 25/25 functions, so 90/40 above is a ratchet with
headroom rather than a target. The two uncovered lines are both
} HANDLE(e, AKERR_ITERATOR_BREAK) { — in libakerror that macro begins with the
break; belonging to PROCESS's case 0: arm, which is only reachable when a
callback returns a non-NULL error context whose status is zero. That is the
pathological case §2.2.1 of TODO.md exists to remove, so it is left uncovered
deliberately rather than pinned by a test.
Branch coverage sits far below line coverage because most branches in this file
are inside the FAIL_*/ATTEMPT/FINISH macro expansions — pool exhaustion,
stack-trace buffer limits, akerr_valid_error_address failures — and belong to
libakerror's own suite rather than to this one. The libakerror 1.0.0 bump made
that gap wider without changing a line of this library: the branch denominator
went from 661 to 1087 as those macros grew, so the same tests that scored 51.0%
(337/661) dropped to 44.3% (481/1087). The gate moved 45 → 40 to match; line and
function coverage did not move at all. Adding the aksl_version_* family and its
tests brought the figure back to 45.1% (519/1151), but the gate stays at 40 —
0.1 points of headroom is not a ratchet.
Two caveats. Coverage is measured at -O0, because the optimizer reorders lines
until per-line counts stop matching the source — so a coverage build is not the
build to profile. And gcov flushes its counters at normal process exit, which an
AKSL_WILL_FAIL_TESTS entry that aborts by design never reaches: such a test
contributes no coverage data at all, so lines only it reaches are reported as
uncovered.
4. Mutation testing
The suite tells you the library works. Mutation testing tells you the suite works: it breaks the library in small ways, one at a time, and checks that the tests notice.
cmake --build build --target mutation # src/stdlib.c + include/akstdlib.h
or drive the script directly for a faster or narrower run:
scripts/mutation_test.py --target src/stdlib.c # C source only
scripts/mutation_test.py --target src/stdlib.c --list # enumerate, build nothing
scripts/mutation_test.py --target src/stdlib.c --max-mutants 20
scripts/mutation_test.py --target src/stdlib.c --threshold 80
A mutant that makes the tests fail is killed (good); one the tests still pass
is a survivor, and names a missing test. The score is killed / total, and the
run prints every survivor with file:line and the exact edit. The harness never
touches your working tree — it copies the repo to a scratch directory and mutates
the copy.
CI runs the src/stdlib.c set with --threshold 80. That is a regression ratchet
rather than a quality bar: the current score is 89.6% (155/173 killed), up
from 46.8% before the wrapper tests landed. Raise the threshold as the remaining
survivors are turned into assertions.
The 18 survivors cluster in three places, and each names a real gap rather than a test-harness artifact:
- Statements whose absence nothing observes — deleting
free(ptr),obj->next = NULL, or aSUCCEED_RETURNleaves behaviour the suite does not look at (a leak, a stale pointer, a success that was already NULL). - The
aksl_list_appendcycle/tail walk (tail = slow,slow = slow->next,tail = fast) — the function is broken in exactly this area (TODO.md§2.1.1), so its known-failing test cannot pin the internals yet. lalloc/lfreedefaulting inaksl_tree_iterate— dead parameters (§2.2.8): they are defaulted and then never called, so inverting the guard changes nothing observable.
The pre-push hook
.githooks/pre-push runs the fast harnesses — the default build and the
sanitizer build, each followed by ctest — before letting a push out. Enable it
once per clone:
git config core.hooksPath .githooks
It only builds when there are commits to push (a branch deletion is a no-op), and
it builds under .git/aksl-prepush so it never disturbs your own build/.
AKSL_HOOK_MUTATION=1 git push # also run the mutation gate (slow)
git push --no-verify # skip the hook entirely
Other knobs: AKSL_MUTATION_THRESHOLD (default 80, keep it in step with
.gitea/workflows/ci.yaml) and AKSL_HOOK_BUILD_DIR.