From 42b60f725dbbe8450ee07995bb3eb175adc88e4f Mon Sep 17 00:00:00 2001 From: Andrew Kesterson Date: Fri, 31 Jul 2026 07:30:18 -0400 Subject: [PATCH] Close the test gaps mutation testing found in audio and draw A mutation smoke run scored src/audio.c at 50% and src/draw.c at 70% and named three real holes: - Nothing asserted that an unconfigured voice is audible, which is the entire reason the voice table defaults to a square wave at full level instead of a zeroed struct. Deleting the defaults survived. - No envelope test used a non-zero attack and decay together, so the decay measuring from the start of the note rather than from the end of the attack survived. - The circle was only checked at its four axis points, which a mis-signed octant reflection survives. It now checks that every plotted pixel has a mirror in the other three quadrants. Also adds a backend that exists but was never given an SDL_Renderer, which is the state a host is in between allocating one and initializing it; every draw entry point has to report it rather than dereference it. audio 50% -> 75%, draw 70% -> 90%. The survivors are recorded in TODO.md and are honestly untestable from here. Co-Authored-By: Claude Opus 5 (1M context) --- TODO.md | 35 ++++++++++++++++++++- tests/audio.c | 49 +++++++++++++++++++++++++++-- tests/draw.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 166 insertions(+), 3 deletions(-) diff --git a/TODO.md b/TODO.md index 44087aa..0bb7ae1 100644 --- a/TODO.md +++ b/TODO.md @@ -407,12 +407,39 @@ came from elsewhere in the tree and this note was never updated.) | `src/util.c` | 121/131 (92%) | 7/8 | | `src/version.c` | 2/2 (100%) | 1/1 | -Branch coverage reads 20.8% and should not be used as a target. The akerror +Branch coverage reads 21.2% and should not be used as a target. The akerror control-flow macros (`ATTEMPT`/`CATCH`/`PROCESS`/`FINISH`, `FAIL_*_RETURN`) expand into large branch trees per call site, most of them unreachable in normal operation — `src/game.c` reports over 1700 branches across 230 lines. Track line and function coverage; treat branch coverage as a relative signal within a file. +### Mutation testing + +`scripts/mutation_test.py` was run over the three new files as a smoke check +(`--max-mutants 8` to 10 each, so these are samples rather than exhaustive +scores): + +| File | Score | Surviving mutants | +|---|---|---| +| `src/draw.c` | 90% | Deleting the `FAIL_ZERO_BREAK` on `SDL_CreateTextureFromSurface` in `akgl_draw_flood_fill` | +| `src/audio.c` | 75% | Deleting `SUCCEED_RETURN` from the static `check_voice`; deleting `spec.freq` before opening a device | +| `src/text.c` | 50% | Three in `akgl_text_rendertextat`, which has no test yet, plus one `SUCCEED_RETURN` deletion | + +The first pass over `src/audio.c` scored 50% and named two real gaps, both of +which are now tested: nothing asserted that an *unconfigured* voice is audible +(the whole reason the table defaults to a square wave at full level rather than +a zeroed struct), and no envelope test used a non-zero attack *and* decay +together, so the decay measuring from the wrong origin survived. `src/draw.c` +scored 70% first and named one: the circle was only checked at its four axis +points, which a mis-signed octant reflection survives, so it now checks that +every plotted pixel has a mirror in the other three quadrants. + +What is left is honestly untestable from here. Deleting a `SUCCEED_RETURN` +leaves a non-void function falling off its end, which is undefined rather than +observably wrong, and the surviving SDL branches are allocation failures the +suite has no way to provoke. The `src/text.c` survivors go away with the +offscreen harness. + ### Suites Every suite is registered through the `AKGL_TEST_SUITES` list in @@ -634,6 +661,12 @@ Each was found by a test written to assert correct behavior. The `build*/` entry in `.gitignore` hides these trees from `git status`, which makes the state easier to get into and no easier to notice. + A second, smaller version of the same thing: rebuilding an existing + coverage tree after editing a test leaves `.gcda` files describing the old + object layout, and `coverage_reset` -- whose whole job is to delete them -- + fails with `GCOV returncode was 5` before it gets the chance. `find + build-coverage -name '*.gcda' -delete` clears it. Observed with gcovr 7.0. + Fix: pass the build tree to gcovr as an explicit search path instead of letting it default to `--root`, so only the tree under measurement is considered. Touches the two `add_test` blocks at `CMakeLists.txt:204-211` diff --git a/tests/audio.c b/tests/audio.c index 685bf4c..2f0e28b 100644 --- a/tests/audio.c +++ b/tests/audio.c @@ -23,8 +23,13 @@ #include "testutil.h" -/** @brief Frames of output most tests generate at a time. */ -#define TEST_MIX_FRAMES 512 +/** + * @brief Frames of output most tests generate at a time. + * + * Enough to hold a 10 ms attack and a 10 ms decay back to back at 44100 frames + * per second, which is the longest single stretch any test here inspects. + */ +#define TEST_MIX_FRAMES 1024 /** @brief 441 Hz at 44100 frames per second is exactly 100 frames per cycle. */ #define TEST_TONE_HZ 441.0f @@ -67,6 +72,30 @@ static float32_t peak_of(int frames) return peak; } +akerr_ErrorContext *test_audio_defaults(void) +{ + PREPARE_ERROR(errctx); + + ATTEMPT { + // Deliberately the first thing this file does, and deliberately without + // a call to reset_audio(): a voice nobody has configured has to make a + // sound. A zeroed voice would have a sustain of 0.0 and be silent, with + // no error to say why, which is exactly the trap this defends against. + CATCH(errctx, akgl_audio_tone(0, TEST_TONE_HZ, 1000)); + CATCH(errctx, akgl_audio_mix(samples, TEST_MIX_FRAMES)); + TEST_ASSERT_FEQ(errctx, samples[0], 1.0f, + "an unconfigured voice mixed to %f, expected a full-level square wave", + samples[0]); + TEST_ASSERT_FEQ(errctx, samples[50], -1.0f, + "an unconfigured voice is not a square wave (%f half a cycle in)", + samples[50]); + } CLEANUP { + IGNORE(reset_audio()); + } PROCESS(errctx) { + } FINISH(errctx, true); + SUCCEED_RETURN(errctx); +} + akerr_ErrorContext *test_audio_silence(void) { PREPARE_ERROR(errctx); @@ -254,6 +283,21 @@ akerr_ErrorContext *test_audio_envelope(void) TEST_ASSERT_FEQ(errctx, samples[450], -0.5f, "after the decay the level is %f, expected the 0.5 sustain", samples[450]); + // Attack and decay together, which is the case where the decay has to + // measure from the end of the attack rather than from the start of the + // note. 10 ms of each is 441 frames of each at 44100. + CATCH(errctx, reset_audio()); + CATCH(errctx, akgl_audio_envelope(0, 10, 10, 0.5f, 0)); + CATCH(errctx, akgl_audio_tone(0, TEST_TONE_HZ, 1000)); + CATCH(errctx, akgl_audio_mix(samples, TEST_MIX_FRAMES)); + TEST_ASSERT_FEQ(errctx, samples[441], 1.0f, + "at the end of the attack the level is %f, expected full", samples[441]); + TEST_ASSERT_FEQ(errctx, samples[661], -(1.0f - (0.5f * (220.0f / 441.0f))), + "halfway through the decay the level is %f, expected the halfway ramp", + samples[661]); + TEST_ASSERT_FEQ(errctx, samples[900], 0.5f, + "after the decay the level is %f, expected the 0.5 sustain", samples[900]); + TEST_EXPECT_STATUS(errctx, AKERR_OUTOFBOUNDS, akgl_audio_envelope(0, 1, 1, -0.1f, 1), "setting a negative sustain level"); TEST_EXPECT_STATUS(errctx, AKERR_OUTOFBOUNDS, akgl_audio_envelope(0, 1, 1, 1.5f, 1), @@ -432,6 +476,7 @@ int main(void) "Couldn't initialize SDL: %s", SDL_GetError()); + CATCH(errctx, test_audio_defaults()); CATCH(errctx, test_audio_silence()); CATCH(errctx, test_audio_square_tone()); CATCH(errctx, test_audio_waveforms()); diff --git a/tests/draw.c b/tests/draw.c index 5ea0b8f..4b6d18c 100644 --- a/tests/draw.c +++ b/tests/draw.c @@ -200,6 +200,9 @@ akerr_ErrorContext *test_draw_circle(void) { PREPARE_ERROR(errctx); SDL_Surface *shot = NULL; + bool symmetric = true; + int x = 0; + int y = 0; ATTEMPT { CATCH(errctx, clear_target()); @@ -218,6 +221,23 @@ akerr_ErrorContext *test_draw_circle(void) // ...and nothing lands outside the radius. TEST_ASSERT(errctx, pixel_is(shot, 43, 32, testred) == false, "the circle drew a pixel one past its radius"); + + // Every plotted pixel has a mirror in the other three quadrants. The + // circle is drawn one octant at a time and reflected seven ways, so a + // sign error in any single reflection breaks this and nothing else -- + // the four axis points above stay put either way. + for ( y = 22; y <= 42; y++ ) { + for ( x = 22; x <= 42; x++ ) { + if ( !pixel_is(shot, x, y, testred) ) { + continue; + } + TEST_ASSERT_FLAG(symmetric, pixel_is(shot, 64 - x, y, testred)); + TEST_ASSERT_FLAG(symmetric, pixel_is(shot, x, 64 - y, testred)); + TEST_ASSERT_FLAG(symmetric, pixel_is(shot, 64 - x, 64 - y, testred)); + } + } + TEST_ASSERT(errctx, symmetric == true, + "the circle is not symmetric about its center; an octant is reflected wrong"); SDL_DestroySurface(shot); shot = NULL; @@ -470,6 +490,70 @@ akerr_ErrorContext *test_draw_preserves_render_draw_color(void) SUCCEED_RETURN(errctx); } +akerr_ErrorContext *test_draw_backend_without_a_renderer(void) +{ + PREPARE_ERROR(errctx); + akgl_RenderBackend empty; + SDL_Surface *saved = NULL; + SDL_Surface *scratch = NULL; + SDL_FRect box; + SDL_Rect region; + + ATTEMPT { + // A backend that exists but was never given an SDL_Renderer. Every + // entry point has to say so rather than dereference it -- this is the + // state a host is in between allocating a backend and initializing it. + memset(&empty, 0x00, sizeof(akgl_RenderBackend)); + box.x = 0.0f; + box.y = 0.0f; + box.w = 4.0f; + box.h = 4.0f; + region.x = 0; + region.y = 0; + region.w = 4; + region.h = 4; + + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_point(&empty, 0.0f, 0.0f, testred), + "plotting through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_line(&empty, 0.0f, 0.0f, 1.0f, 1.0f, testred), + "drawing a line through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_rect(&empty, &box, testred), + "outlining through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_filled_rect(&empty, &box, testred), + "filling through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_circle(&empty, 4.0f, 4.0f, 2.0f, testred), + "drawing a circle through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_flood_fill(&empty, 0, 0, testred), + "flooding through an uninitialized backend"); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_copy_region(&empty, ®ion, &saved), + "saving through an uninitialized backend"); + TEST_ASSERT(errctx, saved == NULL, + "a refused save still wrote something to the destination"); + + scratch = SDL_CreateSurface(4, 4, SDL_PIXELFORMAT_RGBA32); + FAIL_ZERO_BREAK(errctx, scratch, AKGL_ERR_SDL, "%s", SDL_GetError()); + TEST_EXPECT_STATUS(errctx, AKERR_NULLPOINTER, + akgl_draw_paste_region(&empty, scratch, 0.0f, 0.0f), + "pasting through an uninitialized backend"); + } CLEANUP { + if ( saved != NULL ) { + SDL_DestroySurface(saved); + } + if ( scratch != NULL ) { + SDL_DestroySurface(scratch); + } + } PROCESS(errctx) { + } FINISH(errctx, true); + SUCCEED_RETURN(errctx); +} + int main(void) { PREPARE_ERROR(errctx); @@ -508,6 +592,7 @@ int main(void) CATCH(errctx, test_draw_flood_fill()); CATCH(errctx, test_draw_copy_and_paste_region()); CATCH(errctx, test_draw_preserves_render_draw_color()); + CATCH(errctx, test_draw_backend_without_a_renderer()); } CLEANUP { SDL_Quit(); } PROCESS(errctx) {