Measure the library: perf suites, a recorded baseline, and targets

tests/perf.c and tests/perf_render.c drive the hot paths hard enough to
time them -- pool acquire and release at both ends of the scan, the
registry, the state-to-sprite lookup, the per-actor update and render, the
physics sweep, an all-pairs collision sweep, the JSON accessors, path
resolution, the drawing primitives, text, asset loading, a screenful of
tiles, and a whole frame through akgl_game_update. They are registered like
any other suite but carry the `perf` label, so `ctest -L perf` runs only
them and `-LE perf` leaves them out. Every measurement is held to a budget
at roughly ten times the recorded baseline, enforced only in an optimized
build at full scale.

Three things had to be got right before the numbers meant anything, and
each was wrong first:

- No error checking inside the clock. PASS and CATCH call
  akerr_valid_error_address, which walks AKERR_ARRAY_ERROR -- more work than
  several of the calls being measured.
- Flush the renderer before stopping it. SDL batches, so the first version
  measured queueing, reported a tilemap frame 250 times faster than it is,
  and paid the real cost at teardown inside SDL_DestroyTexture.
- A drawing benchmark needs a raw-SDL control doing the same pixel work
  with the same access pattern. With one, akgl_tilemap_draw turns out to
  cost 0.2% of the frame it appeared to own: 16.26 ms against a control's
  16.23 ms for the same 1200 blits. The rasterizer is the frame.

PERFORMANCE.md records the baseline, the frame budget it adds up to, and
what the numbers say -- including that the string pool's acquire is 64x
slower full than empty because it is a megabyte of PATH_MAX buffers, that a
handled missing-sprite condition costs nine times the update it replaces,
that text rasterizes and throws away a texture on every call, and that 28 MB
of the library's BSS is one akgl_Tilemap.

TODO.md gains a Performance section: five defects the stress tests found
that the unit suites do not reach (a tilemap load leaking five pooled
strings, two JSON accessors that turn pool exhaustion into a segfault
rather than AKGL_ERR_HEAP, akgl_game_update crashing without
akgl_game_init, and its actor update sweep running once per tilemap layer),
and eighteen targets with today's number and whether it is met.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 14:24:05 -04:00
parent c6227545a6
commit bcd49fc5b1
8 changed files with 2668 additions and 6 deletions

View File

@@ -62,6 +62,8 @@ ctest --test-dir build --output-on-failure
Run one test while iterating, for example `ctest --test-dir build -R sprite --output-on-failure`. The `rebuild.sh` script also installs into a developer-specific `/home/andrew/local` prefix and removes existing outputs; prefer the portable commands above unless that exact workflow is intended.
Run only the performance suites with `ctest --test-dir build -L perf --output-on-failure`, or leave them out of an ordinary run with `-LE perf`. They take about 30 seconds together, print a table of nanoseconds per operation, and fail only when a measurement exceeds a budget set at roughly ten times the recorded baseline. `AKGL_BENCH_SCALE` scales every iteration count — `AKGL_BENCH_SCALE=0.1 ctest --test-dir build -L perf` for a quick look — and below 1.0 the budgets are reported but not enforced. The recorded baseline and what it means are in `PERFORMANCE.md`.
Run mutation testing with `cmake --build build --target mutation`. For a quick smoke run, use `scripts/mutation_test.py --target src/tilemap.c --max-mutants 10`; the harness mutates only a scratch copy and excludes the intentionally failing character test.
Generate HTML and Cobertura coverage reports with `cmake -S . -B build-coverage -DAKGL_COVERAGE=ON -DCMAKE_BUILD_TYPE=Debug`, then build and run CTest. Reports are written to `build-coverage/coverage/`; this mode requires `gcovr` and GCC or Clang.
@@ -269,6 +271,14 @@ because the failure mode is silent.
- `CATCH` reports failure by `break`ing, which binds to the innermost enclosing
loop or `switch`. A `CATCH` written directly inside a `while` exits the loop
rather than the function — put the `ATTEMPT` block inside the loop.
- **Never `return` from inside a `HANDLE` block either.** `FINISH` ends with
`RELEASE_ERROR`, so leaving before it means the handled context is never given
back to `AKERR_ARRAY_ERROR` — one leaked slot per call, and the 129th call
aborts the process with "Unable to pull an error context from the array!".
`SUCCEED_RETURN` is safe because it releases first; a bare `return`, or a
`return f(...)` that tail-calls the fallback, is not. Set a flag in the
`HANDLE` block and act on it after `FINISH`. This has already cost a
process-killing leak in `akgl_path_relative`; see TODO.md, "Performance".
- Validate every pointer parameter before dereferencing it, including the ones a
sibling function happens not to check.
@@ -296,6 +306,33 @@ Declare no-argument functions as `(void)`, not `()`.
Tests use simple executable return codes and are registered through CTest in `CMakeLists.txt`; there is no declared coverage threshold. Add focused tests as `tests/<feature>.c`, create a matching `test_<feature>` target, and register it with `add_test`. Put reusable fixtures in `tests/assets/` and keep paths compatible with tests launched from the build tree. Coverage mode wraps the suite in a CTest fixture so counters are reset before tests and reports are generated afterward.
### Performance suites
`tests/perf.c` (nothing that draws) and `tests/perf_render.c` (everything that
does) are benchmarks, built and registered like any other suite but listed in
`AKGL_PERF_SUITES` so they carry the `perf` label and a longer timeout. The
harness is `tests/benchutil.h`. Three rules keep the numbers honest, and all
three were learned by getting them wrong first:
- **Nothing that checks an error goes inside the clock.** `PASS` and `CATCH`
call `akerr_valid_error_address`, which walks `AKERR_ARRAY_ERROR` — more work
than several of the calls being measured. Use `BENCH_LOOP`, which stashes the
context and stops at the first failure, and hand it to `PASS` afterwards.
- **Flush the renderer before stopping the clock.** SDL batches: a `draw_*` call
queues a command and returns. `BENCH_FLUSH_STOP` in `tests/perf_render.c` is
there because the first version of that suite measured queueing, reported a
tilemap frame 250 times faster than it is, and paid the real cost at teardown
inside `SDL_DestroyTexture`.
- **A drawing benchmark needs a raw-SDL control that does the same pixel work.**
Without one there is no way to separate what libakgl costs from what the
rasterizer costs, and the answer is not the one you would guess — see
`PERFORMANCE.md`.
Budgets are per-operation ceilings at roughly ten times the recorded baseline;
they are enforced only in an optimized build at full scale. When a change moves
a number for a good reason, re-record the baseline in `PERFORMANCE.md` in the
same commit and say why it moved.
## Commit & Pull Request Guidelines
Recent commits use concise, imperative summaries such as `Fix a scale bug...` and often explain related changes in one sentence. Keep each commit scoped and describe user-visible behavior. Pull requests should summarize the change, identify affected modules, list the CMake/CTest commands run, and link relevant issues. Include screenshots only for rendering, map, or `charviewer` changes.