Recount the consumer calls against this release
All checks were successful
All checks were successful
akbasic's src/ ported onto 0.2.0 calls this library 301 times and raw libc 13 -- 4.1% bypassed, against 86.4% on the same tree before the port and 92.2% at the first count. The port builds clean at -Wall -Wextra, passes 112/112 of akbasic's ctest suite and is ASan+UBSan-clean. The method was never written down and the figure was not reproducible. scripts/consumer_calls.py is that method, and reproducing it turned up two corrections: the old 116 was 117 by its own table's arithmetic, and 119 by a complete count -- the table had no row for strncmp or memmove. Nothing was blocked by a missing wrapper. 272 of 285 sites converted; the 13 that did not are blocked by wrapper shape, and are filed as #32-#38. Say plainly what the number does not cover: akbasic makes 0 calls into list, tree, hash map and string buffer combined, so the recount is evidence about the string, memory and format surface and about nothing else. Refs #26 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
40
README.md
40
README.md
@@ -181,8 +181,9 @@ would notice.
|
||||
|
||||
## Testing
|
||||
|
||||
There are four harnesses. The first three take seconds; the fourth takes about
|
||||
half an hour.
|
||||
There are five harnesses. The first three take seconds and the fifth is instant;
|
||||
the fourth takes about half an hour. The fifth is the only one that measures
|
||||
something outside this repository.
|
||||
|
||||
### 1. The test suite
|
||||
|
||||
@@ -469,6 +470,41 @@ right-leaning tree would have blown the stack the depth cap exists to protect),
|
||||
and `aksl_tree_remove` on an empty tree, which without its guard dereferences
|
||||
NULL. Both are in the suite now — which is what the harness is for.
|
||||
|
||||
### 5. Consumer adoption
|
||||
|
||||
Coverage says the tests reach the code and mutation testing says they would
|
||||
notice it breaking. Neither says anybody *wanted* the code. That question only has
|
||||
an external answer, so there is a harness for it too:
|
||||
|
||||
```sh
|
||||
scripts/consumer_calls.py ../akbasic/src # the ratio
|
||||
scripts/consumer_calls.py ../akbasic/src --detail --per-file # where it comes from
|
||||
scripts/consumer_calls.py ../akbasic/src --baseline 301/13 # against a past count
|
||||
```
|
||||
|
||||
It counts, across a consumer's source directory, how often that consumer calls
|
||||
this library against how often it reaches past it to the libc function this
|
||||
library wraps. Calls to libc functions **not** wrapped here — `isdigit`, `exit`,
|
||||
`qsort` — score on neither side; the question is how often an *available* wrapper
|
||||
gets bypassed. Comments and string literals are stripped before counting, and the
|
||||
wrapped-libc set is read out of `include/akstdlib.h` rather than hardcoded, so a
|
||||
recount after a release measures the surface that release actually shipped.
|
||||
|
||||
A wrapper nobody calls either does not fit or is not discoverable, and both are
|
||||
this library's problem rather than the consumer's. `TODO.md` carries the standing
|
||||
figures and what they did and did not justify.
|
||||
|
||||
Two warnings, both learned the hard way and both printed by `--baseline`:
|
||||
|
||||
- **A rate is only comparable between two counts of the same tree.** If the
|
||||
consumer grew between them, compare the percentage and say which commit each
|
||||
number came from. Comparing the totals across a tree that tripled in size is how
|
||||
a real improvement gets reported as a regression, or the reverse.
|
||||
- **One consumer's ratio is evidence, not a plan.** A consumer that draws
|
||||
everything from fixed pools will never call the allocator however good the
|
||||
allocator is. Weight the result by what the consumer is, and get a second
|
||||
consumer before treating any ranking as settled.
|
||||
|
||||
## The pre-push hook
|
||||
|
||||
`.githooks/pre-push` runs the fast harnesses — the default build and the
|
||||
|
||||
Reference in New Issue
Block a user