Recount the consumer calls against this release

akbasic's src/ ported onto 0.2.0 calls this library 301 times and raw libc
13 -- 4.1% bypassed, against 86.4% on the same tree before the port and
92.2% at the first count. The port builds clean at -Wall -Wextra, passes
112/112 of akbasic's ctest suite and is ASan+UBSan-clean.

The method was never written down and the figure was not reproducible.
scripts/consumer_calls.py is that method, and reproducing it turned up two
corrections: the old 116 was 117 by its own table's arithmetic, and 119 by
a complete count -- the table had no row for strncmp or memmove.

Nothing was blocked by a missing wrapper. 272 of 285 sites converted; the
13 that did not are blocked by wrapper shape, and are filed as #32-#38.

Say plainly what the number does not cover: akbasic makes 0 calls into
list, tree, hash map and string buffer combined, so the recount is
evidence about the string, memory and format surface and about nothing
else.

Refs #26

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

README.md

@@ -181,8 +181,9 @@ would notice.
 
 ## Testing
 
-There are four harnesses. The first three take seconds; the fourth takes about
+There are five harnesses. The first three take seconds and the fifth is instant;
-half an hour.
+the fourth takes about half an hour. The fifth is the only one that measures
+something outside this repository.
 
 ### 1. The test suite
 
@@ -469,6 +470,41 @@ right-leaning tree would have blown the stack the depth cap exists to protect),
 and `aksl_tree_remove` on an empty tree, which without its guard dereferences
 NULL. Both are in the suite now — which is what the harness is for.
 
+### 5. Consumer adoption
+
+Coverage says the tests reach the code and mutation testing says they would
+notice it breaking. Neither says anybody *wanted* the code. That question only has
+an external answer, so there is a harness for it too:
+
+```sh
+scripts/consumer_calls.py ../akbasic/src                     # the ratio
+scripts/consumer_calls.py ../akbasic/src --detail --per-file # where it comes from
+scripts/consumer_calls.py ../akbasic/src --baseline 301/13   # against a past count
+```
+
+It counts, across a consumer's source directory, how often that consumer calls
+this library against how often it reaches past it to the libc function this
+library wraps. Calls to libc functions **not** wrapped here — `isdigit`, `exit`,
+`qsort` — score on neither side; the question is how often an *available* wrapper
+gets bypassed. Comments and string literals are stripped before counting, and the
+wrapped-libc set is read out of `include/akstdlib.h` rather than hardcoded, so a
+recount after a release measures the surface that release actually shipped.
+
+A wrapper nobody calls either does not fit or is not discoverable, and both are
+this library's problem rather than the consumer's. `TODO.md` carries the standing
+figures and what they did and did not justify.
+
+Two warnings, both learned the hard way and both printed by `--baseline`:
+
+- **A rate is only comparable between two counts of the same tree.** If the
+  consumer grew between them, compare the percentage and say which commit each
+  number came from. Comparing the totals across a tree that tripled in size is how
+  a real improvement gets reported as a regression, or the reverse.
+- **One consumer's ratio is evidence, not a plan.** A consumer that draws
+  everything from fixed pools will never call the allocator however good the
+  allocator is. Weight the result by what the consumer is, and get a second
+  consumer before treating any ranking as settled.
+
 ## The pre-push hook
 
 `.githooks/pre-push` runs the fast harnesses — the default build and the

TODO.md

@@ -22,6 +22,7 @@ it has been through grooming.
 | Function coverage | 100% (154/154) |
 | Doxygen | 100% of 154, gated — `cmake --build build --target docs` fails on an undocumented function, parameter or return |
 | Mutation score | 72.3% (188/260 sampled from 1701), gated at 65 |
+| Consumer adoption | akbasic ported onto 0.2.0 calls this library 301 times and raw libc 13 — **4.1% bypassed**, from 92.2% at first count. Ungated, and one consumer only |
 
 The six confirmed defects that used to head this file are fixed and
 `AKSL_KNOWN_FAILING_TESTS` is empty. What they were, and what changed as a result,
@@ -102,15 +103,16 @@ are fixed: the right child's `depth + 1` in the depth-first walk, and
 
 ## Evidence from the first full consumer
 
-`akbasic` (`source.starfort.tech/andrew/akbasic`) is a ~6,300-line C interpreter
+`akbasic` (`source.starfort.tech/andrew/akbasic`) is a C interpreter built on this
-built on this library and `libakerror`. It was the first consumer to exercise the
+library and `libakerror` — ~6,300 lines of `src/` when it was first measured,
-whole surface rather than a corner of it, **and what it could not use is what
+20,169 now. It was the first consumer to exercise the whole surface rather than a
-prioritised everything that has been built since.**
+corner of it, **and what it could not use is what prioritised everything that has
+been built since.**
 
 **The number that started it.** Across `src/`, akbasic made **10 calls into this
-library and 116 to raw libc** — a library whose value proposition is "turn silent
+library and 119 to raw libc** — a library whose value proposition is "turn silent
-libc failures into error contexts", bypassed 92% of the time by the consumer most
+libc failures into error contexts", bypassed **92%** of the time by the consumer
-committed to it.
+most committed to it.
 
 | Raw libc it had to use | Count | Now available as |
 |---|---|---|
@@ -121,8 +123,17 @@ committed to it.
 | `strncpy` | 15 | `aksl_strncpy` |
 | `strtoll` / `strtod` | 2 | `aksl_strtoll` / `aksl_strtod` |
 | `fgets` | 2 | `aksl_fgets` |
+| `strncmp` | 1 | `aksl_strncmp` |
+| `memmove` | 1 | `aksl_memmove` |
 | `strstr` | 1 | `aksl_strstr` |
 
+Two corrections to that figure, both found by rebuilding it. It used to read 116;
+the table it sat above summed to 117 and had no row for `strncmp` or `memmove`.
+119 is what `scripts/consumer_calls.py` returns against akbasic `4e188b2`, and it
+is the number everything below compares to. **The method is now a script rather
+than a paragraph, because recovering it afterwards cost more than writing it down
+would have.**
+
 **All four things the port had to write for itself now exist here.**
 
 1. **A strict `strtoll`/`strtod` wrapper** (`akbasic/src/convert.c`, ~60 lines). The
@@ -144,8 +155,99 @@ committed to it.
 the sign-extended djb2 reads bytes unsigned; and the missing `va_end` — which
 akbasic's stdio text sink ran on every line of program output — is fixed.
 
-**Still true, and still shaping the wishlist.** akbasic uses no allocator, no lists
+### The recount, against this release
-and no trees, drawing everything from fixed pools by design. **A consumer that does
+
-allocate would weight the `open`/`read`/`write` work far higher than this one
+akbasic's `src/` was ported onto 0.2.0 and counted again (#26). The port builds
-does**, so one consumer's count is evidence, not a plan. Re-counting against this
+clean at `-Wall -Wextra`, passes **112/112** of akbasic's ctest suite, and is
-release is #26.
+ASan+UBSan-clean.
+
+| | libakstdlib | raw libc | bypassed |
+|---|---|---|---|
+| Baseline — akbasic `4e188b2`, 5,679 lines of `src/` | 10 | 119 | **92.2%** |
+| Before the port — akbasic `330d731`, 20,169 lines | 45 | 285 | **86.4%** |
+| **After the port — same tree** | **301** | **13** | **4.1%** |
+
+**Read the third row against the second, not the first.** The tree tripled between
+the baseline and the port, so 10/119 and 45/285 are counts of two different
+programs; only 86.4% → 4.1% is a like-for-like measurement. The 45 in the middle
+row is worth its own note — akbasic had already adopted `aksl_f*` across
+`runtime_disk.c` on its own, without anybody counting.
+
+**Nothing was blocked by a missing wrapper.** Every libc call akbasic makes had an
+`aksl_*` counterpart. 272 of the 285 sites converted; the 13 that did not are
+blocked by wrapper *shape*, and they are the useful output:
+
+| Why it could not be used | Sites | Where |
+|---|---|---|
+| No error channel to route into — the enclosing function returns `bool` or `void`, or is a `bsearch` comparator whose signature libc fixes | 8 | `structtype.c` `word_is`, `environment.c` `akbasic_environment_is_waiting_for` (public API), `scanner.c` `is_at_end` and `peek_next`, `verbs.c` `verb_compare`, `format.c` `overflow`, `sink_akgl.c` `scroll` (×2) |
+| Truncation is the answer, not the error | 4 | `format.c`, `structtype.c`, `runtime_struct.c`, `renumber.c` |
+| Short-circuit is memory-safety-load-bearing and the compare cannot be hoisted past the NULL arm guarding it | 1 | `runtime_trap.c` |
+
+The truncation four are worth spelling out, because they are a contract decision
+rather than an accident. `PRINT USING "###"; 1E300` prints `***` today: the render
+truncates, the truncated text has no `.`, and the formatter takes its overflow
+path on exactly that. Through `aksl_snprintf` it raises `AKERR_OUTOFBOUNDS` out of
+the interpreter instead. Two more are truncation-tolerant renderers that print
+what fits and stop, and the fourth uses `snprintf`'s return to raise akbasic's
+own `AKBASIC_ERR_BOUNDS` with its own message.
+
+### What the recount found, and where it went
+
+Every one of the 13 blocked sites came back to wrapper *shape* rather than a
+missing wrapper, and the same seven shapes recurred across ten independent
+conversion passes. They are filed, not listed here:
+
+| Finding | Filed as |
+|---|---|
+| `aksl_snprintf`'s `count` out-param is required, so ~20 sites carry an `int written` that is written and never read. Raised by all ten passes. `-Wall -Wextra` cannot see it — `&written` is a use | #32 |
+| No equality comparison. All 43 comparison sites flatten the three-way `int` to `== 0`; not one wants an ordering, and five now need a sentinel whose *initial value is load-bearing* | #33 |
+| No truncating format and no length query, which is the whole of the truncation-four above | #34 |
+| `aksl_hashmap_*` carries one payload, which is the only reason `akbasic/src/symtab.c` still exists | #35 |
+| `aksl_fgets` signals end of input by raising, so a read loop cannot be a condition | #36 |
+| A caller cannot add its own context to a wrapper's error, so it raises and discards instead — eight lines where there were two | #37 |
+| No form a `bool` predicate or a `void` function can call, which is 8 of the 13 blocked sites. Carries the `ctype.h` question and the infallible-`memset` question with it | #38 |
+
+`#14` already covered the `bsearch` comparator, and the port confirmed it from the
+consumer side.
+
+**The one thing the wrappers did better than the libc they replaced** is worth
+recording next to the complaints: `aksl_fgets`'s `len_out` **deleted** two `strlen`
+calls rather than converting them, and is more correct than what it replaced for a
+line containing an embedded NUL. It is the only one of 272 conversions that
+produced less code than it started with.
+
+### Still true, and still the reason one count is not a plan
+
+akbasic uses no allocator, no lists and no trees, drawing everything from fixed
+pools by design, and porting it did not change that. Of the 301 calls it now
+makes:
+
+| Area | Calls | |
+|---|---|---|
+| Strings | 122 | 40.5% |
+| Memory | 69 | 22.9% |
+| Formatted output | 59 | 19.6% |
+| Streams | 38 | 12.6% |
+| String → number | 12 | 4.0% |
+| Hashing | 1 | 0.3% |
+| **Collections** | **0** | **0%** |
+
+**Four fifths of the evidence is strings, memory and formatting.** The collections
+work — list, tree, hash map, string buffer, `src/collections.c` and the largest
+single body of code in this library — has **not one consumer call site**, and the
+single hashing call next to it is `aksl_strhash_djb2` feeding a hash table akbasic
+wrote for itself. **A consumer that does allocate would weight the
+`open`/`read`/`write` work far higher than this one does**, so this remains
+evidence and not a plan.
+
+**The number to distrust is not the 4.1%; it is the 0%.** A recount that moves
+92% to 4% on one consumer says the string, memory and format wrappers fit the
+consumer that asked for them. It says nothing at all about the half of the library
+that consumer never calls, and it cannot, however many times it is run. What would
+say something is a second consumer with different shape — one that allocates.
+
+`akbasic/src/symtab.c` is the sharpest instance. It is the hand-rolled fixed-capacity
+string-keyed hash table `aksl_hashmap_*` was generalised from, it survived the port
+untouched, and the reason turned out to be one field rather than a design
+disagreement — everything else about the two already lines up. #35 has it, and it
+is the first collections work with a consumer actually waiting for it.

scripts/consumer_calls.py

@@ -0,0 +1,245 @@
+#!/usr/bin/env python3
+"""
+Consumer adoption harness for libakstdlib.
+
+Answers one question about a consumer's source tree: how often does it call
+this library, and how often does it reach past this library to the libc
+function this library wraps?
+
+The ratio is the only external evidence there is about whether the surface that
+got built is the surface anyone wanted. A wrapper nobody calls is a wrapper that
+either does not fit or is not discoverable, and both are the library's problem.
+
+The number this reports is only worth something if it is reproducible, which is
+why this is a script and not a paragraph. TODO.md's first consumer figure was
+recorded without one, and recovering the method afterwards cost more than
+writing it down would have.
+
+WHAT COUNTS
+
+  * The corpus is every *.c and *.h directly under the given directory. It does
+    not recurse: a consumer's src/ is the thing being measured, not its vendored
+    dependencies, and those are usually a subdirectory.
+
+  * Comments and string/character literals are stripped before anything is
+    counted, so a function named in prose or inside a format string does not
+    score. This matters more than it sounds -- "strlen" appears in doc comments
+    throughout a codebase that has been thinking about strlen.
+
+  * A call site is IDENT immediately followed by '(', where IDENT is not
+    preceded by an identifier character. Declarations are not distinguished from
+    calls; a consumer that declares a function named for a libc entry point will
+    over-count by one per declaration, which is visible in --detail.
+
+  * A library call is any IDENT matching ^aksl_.
+
+  * A bypass is any IDENT naming a libc function this library wraps. That set is
+    read out of include/akstdlib.h rather than hardcoded, so it grows when the
+    library grows and a recount after a release measures the surface that
+    release actually shipped.
+
+  * libc functions this library does NOT wrap -- isdigit, exit, qsort -- score
+    on neither side. The question is how often a consumer bypasses an available
+    wrapper, not how much libc it uses. Adding a wrapper for something and
+    having it ignored is a finding; a consumer calling exit() is not.
+
+WHAT IT CANNOT TELL YOU
+
+  One consumer's ratio is evidence, not a plan. A consumer that draws
+  everything from fixed pools will never call the allocator no matter how good
+  the allocator is, and will weight the string wrappers accordingly. Weight the
+  result by what the consumer is, and get a second consumer before treating any
+  ranking as settled.
+
+Usage:
+    scripts/consumer_calls.py DIR [options]
+
+    DIR                 consumer source directory to measure, e.g.
+                        ../akbasic/src
+    --header PATH       akstdlib.h to read the wrapped-libc set from
+                        (default: include/akstdlib.h beside this script's repo)
+    --detail            list the per-function breakdown on both sides
+    --per-file          list per-file counts, worst bypass ratio first
+    --baseline A/B      compare against a previous count, e.g. --baseline 10/119
+    --json              emit the whole result as JSON instead of text
+"""
+
+import argparse
+import json
+import os
+import re
+import sys
+from collections import Counter
+
+# Wrapper families that are this library's own constructs rather than a libc
+# function under a new name. aksl_list_append has no libc counterpart, so
+# "append" must not become a name a consumer can be scored for bypassing.
+LIBRARY_ONLY_PREFIXES = ("hashmap", "list", "tree", "strbuf", "version",
+                         "strhash")
+
+# Library-only names whose first underscore-separated word is shared with a real
+# libc entry point, so a prefix rule cannot separate them. aksl_realpath wraps
+# realpath(3) and must score; aksl_realpath_alloc is this library's own.
+LIBRARY_ONLY_NAMES = frozenset(("freep", "realpath_alloc"))
+
+CALL = re.compile(r"(?<![A-Za-z0-9_])([A-Za-z_][A-Za-z0-9_]*)\s*\(")
+
+
+def wrapped_libc(header):
+    """The set of libc names this library wraps, read out of the header."""
+    with open(header, encoding="utf-8", errors="replace") as handle:
+        text = handle.read()
+    names = set()
+    for match in re.finditer(r"\baksl_([a-z0-9_]+)\s*\(", text):
+        name = match.group(1)
+        if name.split("_")[0] in LIBRARY_ONLY_PREFIXES:
+            continue
+        if name in LIBRARY_ONLY_NAMES:
+            continue
+        names.add(name)
+    return names
+
+
+def strip_c(src):
+    """Remove comments and string/char literals, preserving everything else."""
+    out = []
+    i, end = 0, len(src)
+    while i < end:
+        char = src[i]
+        if char == "/" and i + 1 < end and src[i + 1] == "/":
+            while i < end and src[i] != "\n":
+                i += 1
+        elif char == "/" and i + 1 < end and src[i + 1] == "*":
+            i += 2
+            while i + 1 < end and not (src[i] == "*" and src[i + 1] == "/"):
+                i += 1
+            i += 2
+        elif char in ('"', "'"):
+            quote = char
+            i += 1
+            while i < end and src[i] != quote:
+                if src[i] == "\\":
+                    i += 1
+                i += 1
+            i += 1
+            out.append(" ")
+        else:
+            out.append(char)
+            i += 1
+    return "".join(out)
+
+
+def measure(srcdir, libc):
+    """Count library and bypass call sites across one directory."""
+    library, bypass, per_file = Counter(), Counter(), {}
+    names = sorted(name for name in os.listdir(srcdir)
+                   if name.endswith((".c", ".h")))
+    for name in names:
+        with open(os.path.join(srcdir, name), encoding="utf-8",
+                  errors="replace") as handle:
+            text = strip_c(handle.read())
+        here_lib = here_raw = 0
+        for match in CALL.finditer(text):
+            ident = match.group(1)
+            if ident.startswith("aksl_"):
+                library[ident] += 1
+                here_lib += 1
+            elif ident in libc:
+                bypass[ident] += 1
+                here_raw += 1
+        if here_lib or here_raw:
+            per_file[name] = (here_lib, here_raw)
+    return library, bypass, per_file
+
+
+def rate(bypassed, total):
+    return 100.0 * bypassed / total if total else 0.0
+
+
+def main():
+    parser = argparse.ArgumentParser(add_help=True)
+    parser.add_argument("srcdir")
+    parser.add_argument("--header")
+    parser.add_argument("--detail", action="store_true")
+    parser.add_argument("--per-file", action="store_true")
+    parser.add_argument("--baseline")
+    parser.add_argument("--json", action="store_true")
+    args = parser.parse_args()
+
+    header = args.header or os.path.join(
+        os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
+        "include", "akstdlib.h")
+    if not os.path.isfile(header):
+        sys.stderr.write(f"error: no such header: {header}\n")
+        return 2
+    if not os.path.isdir(args.srcdir):
+        sys.stderr.write(f"error: no such directory: {args.srcdir}\n")
+        return 2
+
+    libc = wrapped_libc(header)
+    library, bypass, per_file = measure(args.srcdir, libc)
+    lib_total, raw_total = sum(library.values()), sum(bypass.values())
+    total = lib_total + raw_total
+
+    if args.json:
+        print(json.dumps({
+            "source": os.path.abspath(args.srcdir),
+            "header": os.path.abspath(header),
+            "wrapped_libc_names": len(libc),
+            "library_calls": lib_total,
+            "bypass_calls": raw_total,
+            "bypass_pct": round(rate(raw_total, total), 1),
+            "library_breakdown": dict(library.most_common()),
+            "bypass_breakdown": dict(bypass.most_common()),
+            "per_file": {k: {"library": v[0], "bypass": v[1]}
+                         for k, v in per_file.items()},
+        }, indent=2))
+        return 0
+
+    print(f"consumer   : {os.path.abspath(args.srcdir)}")
+    print(f"measured against: {os.path.abspath(header)} "
+          f"({len(libc)} wrapped libc names)")
+    print()
+    print(f"libakstdlib calls : {lib_total}")
+    print(f"bypassed to libc  : {raw_total}")
+    print(f"bypass rate       : {rate(raw_total, total):.1f}%  "
+          f"({raw_total}/{total})")
+
+    if args.baseline:
+        try:
+            was_lib, was_raw = (int(part) for part in args.baseline.split("/"))
+        except ValueError:
+            sys.stderr.write("error: --baseline wants LIBRARY/BYPASS, "
+                             "e.g. 10/119\n")
+            return 2
+        was_total = was_lib + was_raw
+        print()
+        print(f"baseline          : {was_lib} / {was_raw}  "
+              f"({rate(was_raw, was_total):.1f}% bypass)")
+        print(f"change            : {lib_total - was_lib:+d} library, "
+              f"{raw_total - was_raw:+d} bypass, "
+              f"{rate(raw_total, total) - rate(was_raw, was_total):+.1f} pt")
+        print()
+        print("A bypass rate is only comparable between two counts of the same")
+        print("tree. If the consumer grew between them, compare the rate and")
+        print("not the totals -- and say which tree each number came from.")
+
+    if args.detail:
+        print("\nbypassed to libc")
+        for name, count in bypass.most_common():
+            print(f"  {name:<22}{count}")
+        print("\ncalls into libakstdlib")
+        for name, count in library.most_common():
+            print(f"  {name:<22}{count}")
+
+    if args.per_file:
+        print("\nper file (library, bypass), worst bypass first")
+        order = sorted(per_file.items(), key=lambda kv: (-kv[1][1], kv[0]))
+        for name, (lib, raw) in order:
+            print(f"  {name:<32}{lib:>5}{raw:>6}")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())