Measure the library: perf suites, a recorded baseline, and targets
tests/perf.c and tests/perf_render.c drive the hot paths hard enough to time them -- pool acquire and release at both ends of the scan, the registry, the state-to-sprite lookup, the per-actor update and render, the physics sweep, an all-pairs collision sweep, the JSON accessors, path resolution, the drawing primitives, text, asset loading, a screenful of tiles, and a whole frame through akgl_game_update. They are registered like any other suite but carry the `perf` label, so `ctest -L perf` runs only them and `-LE perf` leaves them out. Every measurement is held to a budget at roughly ten times the recorded baseline, enforced only in an optimized build at full scale. Three things had to be got right before the numbers meant anything, and each was wrong first: - No error checking inside the clock. PASS and CATCH call akerr_valid_error_address, which walks AKERR_ARRAY_ERROR -- more work than several of the calls being measured. - Flush the renderer before stopping it. SDL batches, so the first version measured queueing, reported a tilemap frame 250 times faster than it is, and paid the real cost at teardown inside SDL_DestroyTexture. - A drawing benchmark needs a raw-SDL control doing the same pixel work with the same access pattern. With one, akgl_tilemap_draw turns out to cost 0.2% of the frame it appeared to own: 16.26 ms against a control's 16.23 ms for the same 1200 blits. The rasterizer is the frame. PERFORMANCE.md records the baseline, the frame budget it adds up to, and what the numbers say -- including that the string pool's acquire is 64x slower full than empty because it is a megabyte of PATH_MAX buffers, that a handled missing-sprite condition costs nine times the update it replaces, that text rasterizes and throws away a texture on every call, and that 28 MB of the library's BSS is one akgl_Tilemap. TODO.md gains a Performance section: five defects the stress tests found that the unit suites do not reach (a tilemap load leaking five pooled strings, two JSON accessors that turn pool exhaustion into a segfault rather than AKGL_ERR_HEAP, akgl_game_update crashing without akgl_game_init, and its actor update sweep running once per tilemap layer), and eighteen targets with today's number and whether it is met. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
37
AGENTS.md
37
AGENTS.md
@@ -62,6 +62,8 @@ ctest --test-dir build --output-on-failure
|
||||
|
||||
Run one test while iterating, for example `ctest --test-dir build -R sprite --output-on-failure`. The `rebuild.sh` script also installs into a developer-specific `/home/andrew/local` prefix and removes existing outputs; prefer the portable commands above unless that exact workflow is intended.
|
||||
|
||||
Run only the performance suites with `ctest --test-dir build -L perf --output-on-failure`, or leave them out of an ordinary run with `-LE perf`. They take about 30 seconds together, print a table of nanoseconds per operation, and fail only when a measurement exceeds a budget set at roughly ten times the recorded baseline. `AKGL_BENCH_SCALE` scales every iteration count — `AKGL_BENCH_SCALE=0.1 ctest --test-dir build -L perf` for a quick look — and below 1.0 the budgets are reported but not enforced. The recorded baseline and what it means are in `PERFORMANCE.md`.
|
||||
|
||||
Run mutation testing with `cmake --build build --target mutation`. For a quick smoke run, use `scripts/mutation_test.py --target src/tilemap.c --max-mutants 10`; the harness mutates only a scratch copy and excludes the intentionally failing character test.
|
||||
|
||||
Generate HTML and Cobertura coverage reports with `cmake -S . -B build-coverage -DAKGL_COVERAGE=ON -DCMAKE_BUILD_TYPE=Debug`, then build and run CTest. Reports are written to `build-coverage/coverage/`; this mode requires `gcovr` and GCC or Clang.
|
||||
@@ -269,6 +271,14 @@ because the failure mode is silent.
|
||||
- `CATCH` reports failure by `break`ing, which binds to the innermost enclosing
|
||||
loop or `switch`. A `CATCH` written directly inside a `while` exits the loop
|
||||
rather than the function — put the `ATTEMPT` block inside the loop.
|
||||
- **Never `return` from inside a `HANDLE` block either.** `FINISH` ends with
|
||||
`RELEASE_ERROR`, so leaving before it means the handled context is never given
|
||||
back to `AKERR_ARRAY_ERROR` — one leaked slot per call, and the 129th call
|
||||
aborts the process with "Unable to pull an error context from the array!".
|
||||
`SUCCEED_RETURN` is safe because it releases first; a bare `return`, or a
|
||||
`return f(...)` that tail-calls the fallback, is not. Set a flag in the
|
||||
`HANDLE` block and act on it after `FINISH`. This has already cost a
|
||||
process-killing leak in `akgl_path_relative`; see TODO.md, "Performance".
|
||||
- Validate every pointer parameter before dereferencing it, including the ones a
|
||||
sibling function happens not to check.
|
||||
|
||||
@@ -296,6 +306,33 @@ Declare no-argument functions as `(void)`, not `()`.
|
||||
|
||||
Tests use simple executable return codes and are registered through CTest in `CMakeLists.txt`; there is no declared coverage threshold. Add focused tests as `tests/<feature>.c`, create a matching `test_<feature>` target, and register it with `add_test`. Put reusable fixtures in `tests/assets/` and keep paths compatible with tests launched from the build tree. Coverage mode wraps the suite in a CTest fixture so counters are reset before tests and reports are generated afterward.
|
||||
|
||||
### Performance suites
|
||||
|
||||
`tests/perf.c` (nothing that draws) and `tests/perf_render.c` (everything that
|
||||
does) are benchmarks, built and registered like any other suite but listed in
|
||||
`AKGL_PERF_SUITES` so they carry the `perf` label and a longer timeout. The
|
||||
harness is `tests/benchutil.h`. Three rules keep the numbers honest, and all
|
||||
three were learned by getting them wrong first:
|
||||
|
||||
- **Nothing that checks an error goes inside the clock.** `PASS` and `CATCH`
|
||||
call `akerr_valid_error_address`, which walks `AKERR_ARRAY_ERROR` — more work
|
||||
than several of the calls being measured. Use `BENCH_LOOP`, which stashes the
|
||||
context and stops at the first failure, and hand it to `PASS` afterwards.
|
||||
- **Flush the renderer before stopping the clock.** SDL batches: a `draw_*` call
|
||||
queues a command and returns. `BENCH_FLUSH_STOP` in `tests/perf_render.c` is
|
||||
there because the first version of that suite measured queueing, reported a
|
||||
tilemap frame 250 times faster than it is, and paid the real cost at teardown
|
||||
inside `SDL_DestroyTexture`.
|
||||
- **A drawing benchmark needs a raw-SDL control that does the same pixel work.**
|
||||
Without one there is no way to separate what libakgl costs from what the
|
||||
rasterizer costs, and the answer is not the one you would guess — see
|
||||
`PERFORMANCE.md`.
|
||||
|
||||
Budgets are per-operation ceilings at roughly ten times the recorded baseline;
|
||||
they are enforced only in an optimized build at full scale. When a change moves
|
||||
a number for a good reason, re-record the baseline in `PERFORMANCE.md` in the
|
||||
same commit and say why it moved.
|
||||
|
||||
## Commit & Pull Request Guidelines
|
||||
|
||||
Recent commits use concise, imperative summaries such as `Fix a scale bug...` and often explain related changes in one sentence. Keep each commit scoped and describe user-visible behavior. Pull requests should summarize the change, identify affected modules, list the CMake/CTest commands run, and link relevant issues. Include screenshots only for rendering, map, or `charviewer` changes.
|
||||
|
||||
Reference in New Issue
Block a user