libakstdlib/TODO.md

# Record

**Outstanding work is in the issue tracker, not in this file:**
<https://source.starfort.tech/andrew/libakstdlib/issues>

What stays here is what a tracker has no place for: where the library stands, and
the decisions that would otherwise be re-litigated — what is deliberately *not*
wrapped, and which uncovered lines are uncoverable rather than untested.

Issues are labelled by kind and blast radius, and milestoned by what they can land
in: `0.2.x` for anything that breaks no ABI, `0.3.0` for new public symbols,
`1.0.0` for the large surfaces. Everything filed carries `status::grooming` until
it has been through grooming.

## Where the library stands

| | |
|---|---|
| Wrapped | 154 public functions across `src/stdlib.c`, `src/string.c`, `src/stream.c`, `src/collections.c` |
| Tests | 17 CTest binaries plus 2 negative-compile entries, green under the default and sanitizer builds |
| Line coverage | 99.5% (1708/1716) |
| Function coverage | 100% (154/154) |
| Doxygen | 100% of 154, gated — `cmake --build build --target docs` fails on an undocumented function, parameter or return |
| Mutation score | 72.3% (188/260 sampled from 1701), gated at 65 |
| Consumer adoption | akbasic ported onto 0.2.0 calls this library 313 times and raw libc 7 — **2.2% bypassed**, from 92.2% at first count. Ungated, and one consumer only |

The six confirmed defects that used to head this file are fixed and
`AKSL_KNOWN_FAILING_TESTS` is empty. What they were, and what changed as a result,
is in `UPGRADING.md`.

## What libakerror costs this library

Three of these are not fixable from inside this repository, and each costs
something here. They are filed in both places, because the fix is there and the
bill is here. The fourth turned out not to be blocked at all:

| Here | Upstream | What it costs |
|---|---|---|
| #2 | — | **Corrected while filing.** The unlocked error pool is a property of the libakerror this repository *pins* (1.0.0), not of libakerror (2.0.1, which locks it). `libakgl` and `akbasic` are both on 2.0.1. The work is a submodule bump and the verification that goes with it, not a wait |
| #3 | libakerror | `IGNORE()` leaks a context, so `aksl_tree_iterate` open-codes log-then-release in four lines that should be one |
| #4 | libakerror | The `coverage` target is not namespaced when embedded, so `-DAKSL_COVERAGE=ON` fails to configure and `CMakeLists.txt` shadows `add_custom_target` to work around it |
| #5 | libakerror | No `akerrorConfigVersion.cmake`, so `find_dependency(akerror)` cannot ask for the 1.0.0 floor |

## Uncovered lines that are uncoverable

Eight lines, and this is what they are, so the coverage listing does not read as an
oversight.

**Two are the short-transfer branch** in `aksl_fread`/`aksl_fwrite` — a short
transfer with neither EOF nor a stream error. The standard permits it, so the
branch is correct to have; every way of actually producing one on Linux sets `feof`
or `ferror` first. **It is the only error path in the library that has never
executed**, and reaching it needs a `FILE *` over a custom stream (`fopencookie`,
`funopen`). That is #6.

**Two are the string-buffer overflow guard** — the `capacity = needed` arm in
`strbuf_reserve`. Reaching it needs an `aksl_StrBuf` within a factor of two of
`SIZE_MAX`, **which is not a test, it is a hang.** It is there because doubling a
capacity is a multiplication, and an unguarded one is how a growable buffer turns
into a heap overflow. Nothing to do.

**Four are `HANDLE(e, AKERR_ITERATOR_BREAK)` lines**, and they are macro artifacts
rather than gaps. In libakerror that macro begins with the `break;` belonging to
`PROCESS`'s `case 0:` arm, reachable only when a callback returns a non-NULL context
whose status is *zero* — the pathological case the errno-fallback work removed. Left
uncovered deliberately rather than pinned by a test that would have to manufacture
it.

## `aksl_version_check()` ignores its `patch` argument

`src/stdlib.c`, the `(void)patch`. **Correct for the current "same soname" rule** —
patch level never breaks the ABI — but the parameter exists only so the error
message can name the caller's full version.

No consequence today. If a future compatibility rule needs the patch level to
participate, that is the line to change, and the `#if` in `tests/test_version.c` is
the test that encodes the rule.

## Deliberate omissions

Recorded so nobody adds them thinking they were forgotten. **Each is a decision,
and each can be revisited with an argument.**

| Not wrapped | Why |
|---|---|
| `sprintf`, `vsprintf` | Cannot be bounded. An error-handling wrapper around an unbounded write is the sharp edge this library exists to remove. `aksl_snprintf` and `aksl_asprintf` cover both real uses. |
| `strtok` | Keeps its state in a hidden static, so two interleaved tokenisations corrupt each other silently and any use from a thread is a bug. `aksl_strtok_r` and `aksl_strsep` cover it. |
| `strcpy`, `strcat` as libc spells them | Cannot be called safely without the destination's size. The wrappers take it. |
| `setbuf` | Exactly `setvbuf(stream, buf, buf ? _IOFBF : _IONBF, BUFSIZ)` and strictly less expressive. Wrapping it would add a second way to say one thing. |
| `perror` | Writes to stderr and consults a global. `aksl_strerror` is the akerror-native equivalent and knows this library's own statuses as well as errno's. |
| `strerror_r` | Two incompatible functions share that name and which one you get depends on feature-test macros a consumer cannot influence from inside this header. `aksl_strerror` is built on libakerror's registry instead. |

## What the mutation survivors mean

The harness samples 260 of 1701 mutants and kills 72.3%. **Most of the 72 survivors
are equivalent mutants rather than missing tests** — `README.md` has the full
breakdown — and knowing which is which is the point, because a ratchet built on the
wrong number is a ratchet that stops moving.

Three clusters are real work and are #7. Two survivors in that class were real and
are fixed: the right child's `depth + 1` in the depth-first walk, and
`aksl_tree_remove` on an empty tree.

## Evidence from the first full consumer

`akbasic` (`source.starfort.tech/andrew/akbasic`) is a C interpreter built on this
library and `libakerror` — ~6,300 lines of `src/` when it was first measured,
20,169 now. It was the first consumer to exercise the whole surface rather than a
corner of it, **and what it could not use is what prioritised everything that has
been built since.**

**The number that started it.** Across `src/`, akbasic made **10 calls into this
library and 119 to raw libc** — a library whose value proposition is "turn silent
libc failures into error contexts", bypassed **92%** of the time by the consumer
most committed to it.

| Raw libc it had to use | Count | Now available as |
|---|---|---|
| `strlen` | 37 | `aksl_strlen` |
| `snprintf` | 28 | `aksl_snprintf` |
| `strcmp` | 16 | `aksl_strcmp` |
| `memcpy` / `memset` | 16 | `aksl_memcpy` / `aksl_memset` |
| `strncpy` | 15 | `aksl_strncpy` |
| `strtoll` / `strtod` | 2 | `aksl_strtoll` / `aksl_strtod` |
| `fgets` | 2 | `aksl_fgets` |
| `strncmp` | 1 | `aksl_strncmp` |
| `memmove` | 1 | `aksl_memmove` |
| `strstr` | 1 | `aksl_strstr` |

Two corrections to that figure, both found by rebuilding it. It used to read 116;
the table it sat above summed to 117 and had no row for `strncmp` or `memmove`.
119 is what `scripts/consumer_calls.py` returns against akbasic `4e188b2`, and it
is the number everything below compares to. **The method is now a script rather
than a paragraph, because recovering it afterwards cost more than writing it down
would have.**

**All four things the port had to write for itself now exist here.**

1. **A strict `strtoll`/`strtod` wrapper** (`akbasic/src/convert.c`, ~60 lines). The
   `aksl_strto*` family is that, with the endptr/`errno`/range contract. akbasic
   formally banned the `aksl_ato*` family because routing four diagnosable errors
   through it would have turned them into wrong answers — `VAL("garbage")` silently
   returning `0.0`. **That ban can be lifted**: the `ato*` forms report failures now.
2. **A fixed-capacity string-keyed hash table** (`akbasic/src/symtab.c`, ~130 lines,
   needed three times over). `aksl_hashmap_*` is that table generalised, **with
   tombstones on delete, which the original did not have.**
3. **The bounded-copy-with-truncation-as-error idiom, at ten sites.** `aksl_strcpy`
   and `aksl_strncpy` are exactly that idiom.
4. **Uppercase folding for case-insensitive lookup, three times.** `aksl_strcasecmp`
   and `aksl_strncasecmp`.

**And the four confirmed-with-impact defects are closed.** The unbounded
`aksl_sprintf` is gone; `aksl_fopen`'s arguments are checked, so
`akbasic_cmd_dload`'s hand-rolled validation and its comment pointing here can go;
the sign-extended djb2 reads bytes unsigned; and the missing `va_end` — which
akbasic's stdio text sink ran on every line of program output — is fixed.

### The recount, against this release

akbasic's `src/` was ported onto 0.2.0 and counted again (#26). The port builds
clean at `-Wall -Wextra`, passes **112/112** of akbasic's ctest suite, and is
ASan+UBSan-clean.

| | libakstdlib | raw libc | bypassed |
|---|---|---|---|
| Baseline — akbasic `4e188b2`, 5,679 lines of `src/` | 10 | 119 | **92.2%** |
| Before the port — akbasic `330d731`, 20,169 lines | 45 | 285 | **86.4%** |
| **After the port — same tree** | **313** | **7** | **2.2%** |

**Read the third row against the second, not the first.** The tree tripled between
the baseline and the port, so 10/119 and 45/285 are counts of two different
programs; only 86.4% → 2.2% is a like-for-like measurement. The 45 in the middle
row is worth its own note — akbasic had already adopted `aksl_f*` across
`runtime_disk.c` on its own, without anybody counting.

**Nothing was blocked by a missing wrapper.** Every libc call akbasic makes had an
`aksl_*` counterpart. 278 of the 285 sites converted; the 7 that did not are
blocked by wrapper *shape*, and they are the useful output:

| Why it could not be used | Sites | Where |
|---|---|---|
| Truncation is the answer, not the error | 6 | `runtime.c` `akbasic_runtime_error`, `host.c` (×3), `runtime_struct.c` (×2) |
| A `bsearch(3)` comparator, whose signature libc fixes, so there is no out parameter to report through | 1 | `verbs.c` `verb_compare` |

**The eight sites that used to head this table are gone, and how they went is the
finding.** They were the `bool` predicates and `void` helpers with no error channel
to route into, and the first filing (#38) asked this library for a form they could
call. Andrew ruled that invalid: a function that cannot report an error changes its
own signature rather than being handed a way to swallow one. Seven of the eight did
exactly that — they now return `akerr_ErrorContext *` and hand the answer back
through an out parameter, one of them (`akbasic_environment_is_waiting_for` and its
sibling) as a public header change. Four more functions on the same call chains
(`scanner.c` `peek` and `match_next_char`, `sink_akgl.c` `putchar_at`, `echo_line`
and `edit_key`) had to move with them. **The wrapper shape was right and the
consumer's signatures were wrong**, which is not what the first count assumed.

The truncation six are a contract decision rather than an accident, and they are
what is genuinely left. The sharpest is `akbasic_runtime_error` — the one function
that tells the user *what went wrong*, formatting a 12,384-byte error message into
a 512-byte line. Truncating a report there is correct; raising `AKERR_OUTOFBOUNDS`
would replace the diagnosis with a second, different failure. Two more are
truncation-tolerant renderers that print what fits and stop, one of which reads
`snprintf`'s return value to *detect* the truncation and skip the rest of the
render. The remaining three read host-supplied strings into fixed fields.

Two of those three, in `akbasic/src/host.c`, are a latent defect the port surfaced
rather than a decision: a host-registered type name over 31 characters truncates
silently, and two names sharing a 31-character prefix then collide in
`akbasic_structtype_find` — where `structtype.c` refuses the identical case
outright with a limit message. The two registration paths disagree. That is
akbasic's to fix, and it is flagged at the site.

### What the recount found, and where it went

Every blocked site came back to wrapper *shape* rather than a missing wrapper, and
the same seven shapes recurred across ten independent conversion passes. They are
filed, not listed here:

| Finding | Filed as |
|---|---|
| `aksl_snprintf`'s `count` out-param is required, so ~20 sites carry an `int written` that is written and never read. Raised by all ten passes. `-Wall -Wextra` cannot see it — `&written` is a use | #32 |
| No equality comparison. All 43 comparison sites flatten the three-way `int` to `== 0`; not one wants an ordering, and five now need a sentinel whose *initial value is load-bearing* | #33 |
| No truncating format and no length query, which is the whole of the truncation-six above and the only shape still blocking a conversion | #34 |
| `aksl_hashmap_*` carries one payload, which is the only reason `akbasic/src/symtab.c` still exists | #35 |
| `aksl_fgets` signals end of input by raising, so a read loop cannot be a condition | #36 |
| A caller cannot add its own context to a wrapper's error, so it raises and discards instead — eight lines where there were two | #37 |
| No form a `bool` predicate or a `void` function can call, which was 8 of the 13 sites the first count could not convert. **Ruled invalid** — the consumer changes its own signature, and now has | #38 |

**#38 is the one worth reading, because it is the one that was wrong.** It asked
this library to grow a form a `bool` predicate could call, and the answer was that
a predicate which cannot report an error should stop returning `bool`. Seven of its
eight sites converted on that basis, and they are why the count is 2.2% and not 4.1%.
The `ctype.h` half of the same filing is settled too: `isspace`, `isdigit`,
`isalnum` and `toupper` cannot fail, so there is nothing for a wrapper to return
and no reason to add one. What a caller does need is the `(unsigned char)` cast
every correct `ctype.h` call takes, and that is akbasic's note to keep, not this
library's.

The `bsearch` comparator has **no issue of its own**. #38 attributed it to akbasic
`#14`, which is a mis-citation — that issue is the `COLLISION`/`BUMP` pairing
threshold. Converting the comparator means dropping `bsearch(3)` for an in-house
binary search that can propagate, on a lookup that runs once per scanned
identifier, and that wants filing against akbasic before anybody does it.

**The one thing the wrappers did better than the libc they replaced** is worth
recording next to the complaints: `aksl_fgets`'s `len_out` **deleted** two `strlen`
calls rather than converting them, and is more correct than what it replaced for a
line containing an embedded NUL. It is the only one of 278 conversions that
produced less code than it started with.

**The port also found a defect in akbasic rather than in this library.** `DLOAD`
leaked a file descriptor: its read loop sat inside an `ATTEMPT` block and the
`PASS` in it returned past `CLEANUP`, so a scan error left the file open. Hoisting
the loop into its own helper — which converting `fgets` required anyway, because
neither `CATCH` nor `PASS` is legal in a loop inside an `ATTEMPT` — fixes it. That
is the protocol's own rule catching a real leak the moment somebody had to obey it.

### Still true, and still the reason one count is not a plan

akbasic uses no allocator, no lists and no trees, drawing everything from fixed
pools by design, and porting it did not change that. Of the 313 calls it now
makes:

| Area | Calls | |
|---|---|---|
| Strings | 149 | 47.6% |
| Memory | 72 | 23.0% |
| Formatted output | 43 | 13.7% |
| Streams and files | 36 | 11.5% |
| String → number | 12 | 3.8% |
| Hashing | 1 | 0.3% |
| **Collections** | **0** | **0%** |

**Five sixths of the evidence is strings, memory and formatting.** The collections
work — list, tree, hash map, string buffer, `src/collections.c` and the largest
single body of code in this library — has **not one consumer call site**, and the
single hashing call next to it is `aksl_strhash_djb2` feeding a hash table akbasic
wrote for itself. **A consumer that does allocate would weight the
`open`/`read`/`write` work far higher than this one does**, so this remains
evidence and not a plan.

**The number to distrust is not the 2.2%; it is the 0%.** A recount that moves
92% to 2% on one consumer says the string, memory and format wrappers fit the
consumer that asked for them. It says nothing at all about the half of the library
that consumer never calls, and it cannot, however many times it is run. What would
say something is a second consumer with different shape — one that allocates.

`akbasic/src/symtab.c` is the sharpest instance. It is the hand-rolled fixed-capacity
string-keyed hash table `aksl_hashmap_*` was generalised from, it survived the port
untouched, and the reason turned out to be one field rather than a design
disagreement — everything else about the two already lines up. #35 has it, and it
is the first collections work with a consumer actually waiting for it.