andrew/libakgl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Check memory with the suites that already exist

Browse Source 
`cmake --build build --target memcheck` runs every registered CTest suite
under valgrind. There are no new test programs, and there should not be any:
tests/benchutil.h notices valgrind in LD_PRELOAD and drops the benchmark
scale to 0.0005, which turns the perf suites into the broadest path coverage
in the tree at a cost valgrind can survive. A benchmark walks one path a
hundred thousand times; a leak check wants every path walked once. Same
binaries, one flag apart, and the whole run is about thirty seconds.

scripts/memcheck.sh wraps `ctest -T memcheck` because that command records
defects and still exits 0, which cannot gate anything. It also forces the
headless drivers, so the vendor GPU stack is never loaded -- that removes
thousands of unfixable findings inside amdgpu_dri.so without suppressing
anything, and it is what the suites are written for anyway. Only definite
losses, invalid accesses and uninitialised reads count; "still reachable" is
what SDL and FreeType keep for the process lifetime and says nothing about
this library. The three suppressions in scripts/valgrind.supp are each a
decision that a finding belongs to somebody else.

Six defects, all filed in TODO.md under "Memory checking", the first of
which is the one that matters: not one of the four json_load_file calls in
src/ is ever matched by a json_decref, so every asset load abandons its
parsed document -- 1.5 KB per sprite, 2.1 KB per character, 9 KB per load of
the 2x2 fixture map, and on the order of a megabyte for a real one. A game
that reloads a level on death leaks a level's worth of JSON every time.
akgl_get_property also reads up to 4 KiB past the end of every property
value it copies, and the savegame name tables read past the end of every
registry key and write what they find into the file.

tests/util.c zeroes three fixtures it used to leave as stack garbage. The
library was never at fault there -- the tests handed it uninitialised floats
and then made one real call with them -- but sixteen findings of noise in a
new gate is how a gate gets ignored.

The unit suites' TIMEOUT goes from 30 to 300 because CTest applies the same
property to the checked run, where everything is twenty times slower.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Kesterson
2026-07-31 14:59:57 -04:00
parent 8a920860c5
commit af304dc2f9
9 changed files with 476 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
tests/benchutil.h
View File

@@ -33,6 +33,16 @@
* Set `AKGL_BENCH_SCALE` in the environment to change how long the suite runs:
* 0.1 for a tenth of the iterations, 10 for ten times as many. Below 1.0 the
* budgets are not enforced, because a short run is a noisy one.
*
* **Under valgrind these suites become the memory-check suites**, and the scale
* drops itself to #AKGL_BENCH_VALGRIND_SCALE without being asked. That is the
* whole reason `ctest -T memcheck` does not need benchmark programs of its own:
* a leak check wants every path walked *once*, and a benchmark is a program that
* walks one path a hundred thousand times. Divide the iteration counts by two
* thousand and the same binary is exactly the right shape for memcheck --
* broader coverage of asset loading, drawing, and the frame loop than any unit
* suite, at a runtime valgrind can survive. Timings from such a run are
* meaningless and the report says so.
*/
#ifndef _AKGL_BENCHUTIL_H_
@@ -54,6 +64,8 @@
#define AKGL_BENCH_REPETITIONS 5
/** @brief Environment variable scaling every iteration count. */
#define AKGL_BENCH_SCALE_ENV "AKGL_BENCH_SCALE"
/** @brief Scale forced on when running under valgrind. Enough to walk every path, few enough to finish. */
#define AKGL_BENCH_VALGRIND_SCALE 0.0005
/** @brief One row of the report: what was measured, how fast, and what it was allowed to cost. */
typedef struct akgl_Benchmark {
@@ -75,10 +87,38 @@ static akgl_Benchmark *bench_running = NULL;
/** @brief `SDL_GetTicksNS()` at the last bench_start(). */
static uint64_t bench_started_ns = 0;
/**
* @brief Report whether this process is running under valgrind.
*
* Detected from `LD_PRELOAD`, which valgrind fills with its own
* `vgpreload_*.so` before handing the process over. That is a deliberate choice
* over `RUNNING_ON_VALGRIND` from `valgrind/valgrind.h`: the macro is exact, but
* it makes the test suite fail to compile anywhere the valgrind headers are not
* installed, and a benchmark is not worth a build dependency. The cost of the
* check being wrong is a slow run or a fast one, never a wrong answer.
*/
static bool bench_under_valgrind(void)
{
static int detected = -1;
const char *preload = NULL;
if ( detected >= 0 ) {
return ( detected == 1 );
}
detected = 0;
preload = SDL_getenv("LD_PRELOAD");
if ( preload != NULL && strstr(preload, "vgpreload") != NULL ) {
detected = 1;
}
return ( detected == 1 );
}
/**
* @brief Multiplier applied to every iteration count, from `AKGL_BENCH_SCALE`.
*
* Read once and cached. Anything unparseable, negative, or absent gives 1.0.
* Under valgrind the result is capped at #AKGL_BENCH_VALGRIND_SCALE -- the
* smaller of the two wins, so asking for an even shorter run still works.
*/
static double bench_scale(void)
{
@@ -96,6 +136,9 @@ static double bench_scale(void)
scale = 1.0;
}
}
if ( bench_under_valgrind() == true && scale > AKGL_BENCH_VALGRIND_SCALE ) {
scale = AKGL_BENCH_VALGRIND_SCALE;
}
return scale;
}
@@ -117,11 +160,15 @@ static int bench_iterations(int count)
/**
* @brief Report whether budgets are being enforced in this run.
*
* An unoptimized build measures the instrumentation rather than the library,
* and a scaled-down run is too short to trust, so both report without failing.
* An unoptimized build measures the instrumentation rather than the library, a
* scaled-down run is too short to trust, and a run under valgrind is measuring
* valgrind. All three report without failing.
*/
static bool bench_budgets_enforced(void)
{
if ( bench_under_valgrind() == true ) {
return false;
}
#ifdef __OPTIMIZE__
return ( bench_scale() >= 1.0 );
#else
@@ -229,7 +276,11 @@ static int bench_report(void)
char *verdict = NULL;
printf("\n");
printf("scale %.2fx, best of %d runs, budgets %s\n",
if ( bench_under_valgrind() == true ) {
printf("running under valgrind: this is a memory check, not a measurement.\n");
printf("The timings below are valgrind's and mean nothing about libakgl.\n");
}
printf("scale %.4gx, best of %d runs, budgets %s\n",
bench_scale(),
AKGL_BENCH_REPETITIONS,
enforced ? "enforced" : "reported only");

24
tests/util.c
View File

@@ -1,4 +1,5 @@
#include <SDL3/SDL.h>
#include <string.h>
#include <akerror.h>
#include <akgl/error.h>
#include <akgl/heap.h>
@@ -28,9 +29,14 @@ static int live_error_contexts(void)
akerr_ErrorContext *test_akgl_rectangle_points_nullpointers(void)
{
RectanglePoints points;
SDL_FRect testrect;
// Zeroed for the same reason as the fixtures in
// test_akgl_collide_point_rectangle_nullpointers: the last case here is a
// real call, and feeding it stack garbage is noise under `memcheck`.
SDL_FRect testrect = {.x = 0, .y = 0, .w = 0, .h = 0};
PREPARE_ERROR(errctx);
memset((void *)&points, 0x00, sizeof(RectanglePoints));
ATTEMPT {
CATCH(errctx, akgl_rectangle_points(NULL, NULL));
FAIL_BREAK(errctx, AKGL_ERR_BEHAVIOR, "akgl_rectangle_points fails to FAIL with all NULL pointers");
@@ -102,12 +108,18 @@ akerr_ErrorContext *test_akgl_rectangle_points_math(void)
akerr_ErrorContext *test_akgl_collide_point_rectangle_nullpointers(void)
{
point testpoint;
// Zeroed rather than left as whatever the stack held. The last case in this
// function is a real call with real arguments, and reading uninitialised
// floats out of it is sixteen findings under `memcheck` for a test that is
// not about coordinates at all.
point testpoint = { .x = 0, .y = 0 };
RectanglePoints testrectpoints;
bool testcollide;
bool testcollide = false;
PREPARE_ERROR(errctx);
memset(&testrectpoints, 0x00, sizeof(RectanglePoints));
ATTEMPT {
CATCH(errctx, akgl_collide_point_rectangle(&testpoint, &testrectpoints, NULL));
FAIL_BREAK(errctx, AKGL_ERR_BEHAVIOR, "akgl_collide_point_rectangle(*, *, NULL) failed");
@@ -182,9 +194,9 @@ akerr_ErrorContext *test_akgl_collide_point_rectangle_logic(void)
akerr_ErrorContext *test_akgl_collide_rectangles_nullpointers(void)
{
SDL_FRect testrect1;
SDL_FRect testrect2;
bool testcollide;
SDL_FRect testrect1 = {.x = 0, .y = 0, .w = 0, .h = 0};
SDL_FRect testrect2 = {.x = 0, .y = 0, .w = 0, .h = 0};
bool testcollide = false;
PREPARE_ERROR(errctx);