Check memory with the suites that already exist

`cmake --build build --target memcheck` runs every registered CTest suite
under valgrind. There are no new test programs, and there should not be any:
tests/benchutil.h notices valgrind in LD_PRELOAD and drops the benchmark
scale to 0.0005, which turns the perf suites into the broadest path coverage
in the tree at a cost valgrind can survive. A benchmark walks one path a
hundred thousand times; a leak check wants every path walked once. Same
binaries, one flag apart, and the whole run is about thirty seconds.

scripts/memcheck.sh wraps `ctest -T memcheck` because that command records
defects and still exits 0, which cannot gate anything. It also forces the
headless drivers, so the vendor GPU stack is never loaded -- that removes
thousands of unfixable findings inside amdgpu_dri.so without suppressing
anything, and it is what the suites are written for anyway. Only definite
losses, invalid accesses and uninitialised reads count; "still reachable" is
what SDL and FreeType keep for the process lifetime and says nothing about
this library. The three suppressions in scripts/valgrind.supp are each a
decision that a finding belongs to somebody else.

Six defects, all filed in TODO.md under "Memory checking", the first of
which is the one that matters: not one of the four json_load_file calls in
src/ is ever matched by a json_decref, so every asset load abandons its
parsed document -- 1.5 KB per sprite, 2.1 KB per character, 9 KB per load of
the 2x2 fixture map, and on the order of a megabyte for a real one. A game
that reloads a level on death leaks a level's worth of JSON every time.
akgl_get_property also reads up to 4 KiB past the end of every property
value it copies, and the savegame name tables read past the end of every
registry key and write what they find into the file.

tests/util.c zeroes three fixtures it used to leave as stack garbage. The
library was never at fault there -- the tests handed it uninitialised floats
and then made one real call with them -- but sixteen findings of noise in a
new gate is how a gate gets ignored.

The unit suites' TIMEOUT goes from 30 to 300 because CTest applies the same
property to the checked run, where everything is twenty times slower.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 14:59:57 -04:00
parent 8a920860c5
commit af304dc2f9
9 changed files with 476 additions and 13 deletions

View File

@@ -64,6 +64,17 @@ Run one test while iterating, for example `ctest --test-dir build -R sprite --ou
Run only the performance suites with `ctest --test-dir build -L perf --output-on-failure`, or leave them out of an ordinary run with `-LE perf`. They take about 30 seconds together, print a table of nanoseconds per operation, and fail only when a measurement exceeds a budget set at roughly ten times the recorded baseline. `AKGL_BENCH_SCALE` scales every iteration count — `AKGL_BENCH_SCALE=0.1 ctest --test-dir build -L perf` for a quick look — and below 1.0 the budgets are reported but not enforced. The recorded baseline and what it means are in `PERFORMANCE.md`.
Check for memory defects with `cmake --build build --target memcheck`, or
`scripts/memcheck.sh` directly — it takes ctest's selection flags, so
`scripts/memcheck.sh -R tilemap` and `scripts/memcheck.sh -LE perf` both work.
It runs the suites that already exist under valgrind (`ctest -T memcheck`) with
the headless drivers forced, and exits non-zero when valgrind finds a definite
leak, an invalid access, or a read of uninitialised memory — which `ctest -T
memcheck` on its own will not do. The whole run takes about thirty seconds
because the perf suites scale themselves down under valgrind. Suppressions for
third-party findings live in `scripts/valgrind.supp`; add one only when the
finding genuinely cannot be fixed from this repository.
Run mutation testing with `cmake --build build --target mutation`. For a quick smoke run, use `scripts/mutation_test.py --target src/tilemap.c --max-mutants 10`; the harness mutates only a scratch copy and excludes the intentionally failing character test.
Generate HTML and Cobertura coverage reports with `cmake -S . -B build-coverage -DAKGL_COVERAGE=ON -DCMAKE_BUILD_TYPE=Debug`, then build and run CTest. Reports are written to `build-coverage/coverage/`; this mode requires `gcovr` and GCC or Clang.
@@ -333,6 +344,13 @@ they are enforced only in an optimized build at full scale. When a change moves
a number for a good reason, re-record the baseline in `PERFORMANCE.md` in the
same commit and say why it moved.
These two suites are also the memory-check vehicle. `tests/benchutil.h` detects
valgrind from `LD_PRELOAD` and drops the scale to
`AKGL_BENCH_VALGRIND_SCALE`, so the same binaries walk every path once instead
of a hundred thousand times, and the timings a checked run prints are labelled
as meaningless. **Do not add memory-check suites**: a new path worth checking
belongs in a benchmark, where it gets both.
## Commit & Pull Request Guidelines
Recent commits use concise, imperative summaries such as `Fix a scale bug...` and often explain related changes in one sentence. Keep each commit scoped and describe user-visible behavior. Pull requests should summarize the change, identify affected modules, list the CMake/CTest commands run, and link relevant issues. Include screenshots only for rendering, map, or `charviewer` changes.