libakstdlib/TODO.md

# Record

**Outstanding work is in the issue tracker, not in this file:**
<https://source.starfort.tech/andrew/libakstdlib/issues>

What stays here is what a tracker has no place for: where the library stands, and
the decisions that would otherwise be re-litigated — what is deliberately *not*
wrapped, and which uncovered lines are uncoverable rather than untested.

Issues are labelled by kind and blast radius, and milestoned by what they can land
in: `0.2.x` for anything that breaks no ABI, `0.3.0` for new public symbols,
`1.0.0` for the large surfaces. Everything filed carries `status::grooming` until
it has been through grooming.

## Where the library stands

| | |
|---|---|
| Wrapped | 154 public functions across `src/stdlib.c`, `src/string.c`, `src/stream.c`, `src/collections.c` |
| Tests | 17 CTest binaries plus 2 negative-compile entries, green under the default and sanitizer builds |
| Line coverage | 99.5% (1708/1716) |
| Function coverage | 100% (154/154) |
| Doxygen | 100% of 154, gated — `cmake --build build --target docs` fails on an undocumented function, parameter or return |
| Mutation score | 72.3% (188/260 sampled from 1701), gated at 65 |
| Consumer adoption | akbasic ported onto 0.2.0 calls this library 301 times and raw libc 13 — **4.1% bypassed**, from 92.2% at first count. Ungated, and one consumer only |

The six confirmed defects that used to head this file are fixed and
`AKSL_KNOWN_FAILING_TESTS` is empty. What they were, and what changed as a result,
is in `UPGRADING.md`.

## What libakerror costs this library

Three of these are not fixable from inside this repository, and each costs
something here. They are filed in both places, because the fix is there and the
bill is here. The fourth turned out not to be blocked at all:

| Here | Upstream | What it costs |
|---|---|---|
| #2 | — | **Corrected while filing.** The unlocked error pool is a property of the libakerror this repository *pins* (1.0.0), not of libakerror (2.0.1, which locks it). `libakgl` and `akbasic` are both on 2.0.1. The work is a submodule bump and the verification that goes with it, not a wait |
| #3 | libakerror | `IGNORE()` leaks a context, so `aksl_tree_iterate` open-codes log-then-release in four lines that should be one |
| #4 | libakerror | The `coverage` target is not namespaced when embedded, so `-DAKSL_COVERAGE=ON` fails to configure and `CMakeLists.txt` shadows `add_custom_target` to work around it |
| #5 | libakerror | No `akerrorConfigVersion.cmake`, so `find_dependency(akerror)` cannot ask for the 1.0.0 floor |

## Uncovered lines that are uncoverable

Eight lines, and this is what they are, so the coverage listing does not read as an
oversight.

**Two are the short-transfer branch** in `aksl_fread`/`aksl_fwrite` — a short
transfer with neither EOF nor a stream error. The standard permits it, so the
branch is correct to have; every way of actually producing one on Linux sets `feof`
or `ferror` first. **It is the only error path in the library that has never
executed**, and reaching it needs a `FILE *` over a custom stream (`fopencookie`,
`funopen`). That is #6.

**Two are the string-buffer overflow guard** — the `capacity = needed` arm in
`strbuf_reserve`. Reaching it needs an `aksl_StrBuf` within a factor of two of
`SIZE_MAX`, **which is not a test, it is a hang.** It is there because doubling a
capacity is a multiplication, and an unguarded one is how a growable buffer turns
into a heap overflow. Nothing to do.

**Four are `HANDLE(e, AKERR_ITERATOR_BREAK)` lines**, and they are macro artifacts
rather than gaps. In libakerror that macro begins with the `break;` belonging to
`PROCESS`'s `case 0:` arm, reachable only when a callback returns a non-NULL context
whose status is *zero* — the pathological case the errno-fallback work removed. Left
uncovered deliberately rather than pinned by a test that would have to manufacture
it.

## `aksl_version_check()` ignores its `patch` argument

`src/stdlib.c`, the `(void)patch`. **Correct for the current "same soname" rule** —
patch level never breaks the ABI — but the parameter exists only so the error
message can name the caller's full version.

No consequence today. If a future compatibility rule needs the patch level to
participate, that is the line to change, and the `#if` in `tests/test_version.c` is
the test that encodes the rule.

## Deliberate omissions

Recorded so nobody adds them thinking they were forgotten. **Each is a decision,
and each can be revisited with an argument.**

| Not wrapped | Why |
|---|---|
| `sprintf`, `vsprintf` | Cannot be bounded. An error-handling wrapper around an unbounded write is the sharp edge this library exists to remove. `aksl_snprintf` and `aksl_asprintf` cover both real uses. |
| `strtok` | Keeps its state in a hidden static, so two interleaved tokenisations corrupt each other silently and any use from a thread is a bug. `aksl_strtok_r` and `aksl_strsep` cover it. |
| `strcpy`, `strcat` as libc spells them | Cannot be called safely without the destination's size. The wrappers take it. |
| `setbuf` | Exactly `setvbuf(stream, buf, buf ? _IOFBF : _IONBF, BUFSIZ)` and strictly less expressive. Wrapping it would add a second way to say one thing. |
| `perror` | Writes to stderr and consults a global. `aksl_strerror` is the akerror-native equivalent and knows this library's own statuses as well as errno's. |
| `strerror_r` | Two incompatible functions share that name and which one you get depends on feature-test macros a consumer cannot influence from inside this header. `aksl_strerror` is built on libakerror's registry instead. |

## What the mutation survivors mean

The harness samples 260 of 1701 mutants and kills 72.3%. **Most of the 72 survivors
are equivalent mutants rather than missing tests** — `README.md` has the full
breakdown — and knowing which is which is the point, because a ratchet built on the
wrong number is a ratchet that stops moving.

Three clusters are real work and are #7. Two survivors in that class were real and
are fixed: the right child's `depth + 1` in the depth-first walk, and
`aksl_tree_remove` on an empty tree.

## Evidence from the first full consumer

`akbasic` (`source.starfort.tech/andrew/akbasic`) is a C interpreter built on this
library and `libakerror` — ~6,300 lines of `src/` when it was first measured,
20,169 now. It was the first consumer to exercise the whole surface rather than a
corner of it, **and what it could not use is what prioritised everything that has
been built since.**

**The number that started it.** Across `src/`, akbasic made **10 calls into this
library and 119 to raw libc** — a library whose value proposition is "turn silent
libc failures into error contexts", bypassed **92%** of the time by the consumer
most committed to it.

| Raw libc it had to use | Count | Now available as |
|---|---|---|
| `strlen` | 37 | `aksl_strlen` |
| `snprintf` | 28 | `aksl_snprintf` |
| `strcmp` | 16 | `aksl_strcmp` |
| `memcpy` / `memset` | 16 | `aksl_memcpy` / `aksl_memset` |
| `strncpy` | 15 | `aksl_strncpy` |
| `strtoll` / `strtod` | 2 | `aksl_strtoll` / `aksl_strtod` |
| `fgets` | 2 | `aksl_fgets` |
| `strncmp` | 1 | `aksl_strncmp` |
| `memmove` | 1 | `aksl_memmove` |
| `strstr` | 1 | `aksl_strstr` |

Two corrections to that figure, both found by rebuilding it. It used to read 116;
the table it sat above summed to 117 and had no row for `strncmp` or `memmove`.
119 is what `scripts/consumer_calls.py` returns against akbasic `4e188b2`, and it
is the number everything below compares to. **The method is now a script rather
than a paragraph, because recovering it afterwards cost more than writing it down
would have.**

**All four things the port had to write for itself now exist here.**

1. **A strict `strtoll`/`strtod` wrapper** (`akbasic/src/convert.c`, ~60 lines). The
   `aksl_strto*` family is that, with the endptr/`errno`/range contract. akbasic
   formally banned the `aksl_ato*` family because routing four diagnosable errors
   through it would have turned them into wrong answers — `VAL("garbage")` silently
   returning `0.0`. **That ban can be lifted**: the `ato*` forms report failures now.
2. **A fixed-capacity string-keyed hash table** (`akbasic/src/symtab.c`, ~130 lines,
   needed three times over). `aksl_hashmap_*` is that table generalised, **with
   tombstones on delete, which the original did not have.**
3. **The bounded-copy-with-truncation-as-error idiom, at ten sites.** `aksl_strcpy`
   and `aksl_strncpy` are exactly that idiom.
4. **Uppercase folding for case-insensitive lookup, three times.** `aksl_strcasecmp`
   and `aksl_strncasecmp`.

**And the four confirmed-with-impact defects are closed.** The unbounded
`aksl_sprintf` is gone; `aksl_fopen`'s arguments are checked, so
`akbasic_cmd_dload`'s hand-rolled validation and its comment pointing here can go;
the sign-extended djb2 reads bytes unsigned; and the missing `va_end` — which
akbasic's stdio text sink ran on every line of program output — is fixed.

### The recount, against this release

akbasic's `src/` was ported onto 0.2.0 and counted again (#26). The port builds
clean at `-Wall -Wextra`, passes **112/112** of akbasic's ctest suite, and is
ASan+UBSan-clean.

| | libakstdlib | raw libc | bypassed |
|---|---|---|---|
| Baseline — akbasic `4e188b2`, 5,679 lines of `src/` | 10 | 119 | **92.2%** |
| Before the port — akbasic `330d731`, 20,169 lines | 45 | 285 | **86.4%** |
| **After the port — same tree** | **301** | **13** | **4.1%** |

**Read the third row against the second, not the first.** The tree tripled between
the baseline and the port, so 10/119 and 45/285 are counts of two different
programs; only 86.4% → 4.1% is a like-for-like measurement. The 45 in the middle
row is worth its own note — akbasic had already adopted `aksl_f*` across
`runtime_disk.c` on its own, without anybody counting.

**Nothing was blocked by a missing wrapper.** Every libc call akbasic makes had an
`aksl_*` counterpart. 272 of the 285 sites converted; the 13 that did not are
blocked by wrapper *shape*, and they are the useful output:

| Why it could not be used | Sites | Where |
|---|---|---|
| No error channel to route into — the enclosing function returns `bool` or `void`, or is a `bsearch` comparator whose signature libc fixes | 8 | `structtype.c` `word_is`, `environment.c` `akbasic_environment_is_waiting_for` (public API), `scanner.c` `is_at_end` and `peek_next`, `verbs.c` `verb_compare`, `format.c` `overflow`, `sink_akgl.c` `scroll` (×2) |
| Truncation is the answer, not the error | 4 | `format.c`, `structtype.c`, `runtime_struct.c`, `renumber.c` |
| Short-circuit is memory-safety-load-bearing and the compare cannot be hoisted past the NULL arm guarding it | 1 | `runtime_trap.c` |

The truncation four are worth spelling out, because they are a contract decision
rather than an accident. `PRINT USING "###"; 1E300` prints `***` today: the render
truncates, the truncated text has no `.`, and the formatter takes its overflow
path on exactly that. Through `aksl_snprintf` it raises `AKERR_OUTOFBOUNDS` out of
the interpreter instead. Two more are truncation-tolerant renderers that print
what fits and stop, and the fourth uses `snprintf`'s return to raise akbasic's
own `AKBASIC_ERR_BOUNDS` with its own message.

### What the recount found, and where it went

Every one of the 13 blocked sites came back to wrapper *shape* rather than a
missing wrapper, and the same seven shapes recurred across ten independent
conversion passes. They are filed, not listed here:

| Finding | Filed as |
|---|---|
| `aksl_snprintf`'s `count` out-param is required, so ~20 sites carry an `int written` that is written and never read. Raised by all ten passes. `-Wall -Wextra` cannot see it — `&written` is a use | #32 |
| No equality comparison. All 43 comparison sites flatten the three-way `int` to `== 0`; not one wants an ordering, and five now need a sentinel whose *initial value is load-bearing* | #33 |
| No truncating format and no length query, which is the whole of the truncation-four above | #34 |
| `aksl_hashmap_*` carries one payload, which is the only reason `akbasic/src/symtab.c` still exists | #35 |
| `aksl_fgets` signals end of input by raising, so a read loop cannot be a condition | #36 |
| A caller cannot add its own context to a wrapper's error, so it raises and discards instead — eight lines where there were two | #37 |
| No form a `bool` predicate or a `void` function can call, which is 8 of the 13 blocked sites. Carries the `ctype.h` question and the infallible-`memset` question with it | #38 |

`#14` already covered the `bsearch` comparator, and the port confirmed it from the
consumer side.

**The one thing the wrappers did better than the libc they replaced** is worth
recording next to the complaints: `aksl_fgets`'s `len_out` **deleted** two `strlen`
calls rather than converting them, and is more correct than what it replaced for a
line containing an embedded NUL. It is the only one of 272 conversions that
produced less code than it started with.

### Still true, and still the reason one count is not a plan

akbasic uses no allocator, no lists and no trees, drawing everything from fixed
pools by design, and porting it did not change that. Of the 301 calls it now
makes:

| Area | Calls | |
|---|---|---|
| Strings | 122 | 40.5% |
| Memory | 69 | 22.9% |
| Formatted output | 59 | 19.6% |
| Streams | 38 | 12.6% |
| String → number | 12 | 4.0% |
| Hashing | 1 | 0.3% |
| **Collections** | **0** | **0%** |

**Four fifths of the evidence is strings, memory and formatting.** The collections
work — list, tree, hash map, string buffer, `src/collections.c` and the largest
single body of code in this library — has **not one consumer call site**, and the
single hashing call next to it is `aksl_strhash_djb2` feeding a hash table akbasic
wrote for itself. **A consumer that does allocate would weight the
`open`/`read`/`write` work far higher than this one does**, so this remains
evidence and not a plan.

**The number to distrust is not the 4.1%; it is the 0%.** A recount that moves
92% to 4% on one consumer says the string, memory and format wrappers fit the
consumer that asked for them. It says nothing at all about the half of the library
that consumer never calls, and it cannot, however many times it is run. What would
say something is a second consumer with different shape — one that allocates.

`akbasic/src/symtab.c` is the sharpest instance. It is the hand-rolled fixed-capacity
string-keyed hash table `aksl_hashmap_*` was generalised from, it survived the port
untouched, and the reason turned out to be one field rather than a design
disagreement — everything else about the two already lines up. #35 has it, and it
is the first collections work with a consumer actually waiting for it.