Assert the log line a failed report leaves behind
Some checks failed
akbasic CI Build / cmake_build (push) Failing after 3m21s
akbasic CI Build / sanitizers (push) Failing after 4m32s
akbasic CI Build / coverage (push) Failing after 3m39s
akbasic CI Build / akgl_build (push) Failing after 21s
akbasic CI Build / mutation_test (push) Failing after 3m29s

A mutation run over src/runtime.c found that deleting the
`LOG_ERROR_WITH_MESSAGE` in `report_and_reraise()` left the whole suite green.
That line *is* the visible half of the "a failed report must not swallow the
program's own error" fix -- without it the secondary failure goes nowhere and
the only evidence is the original error still being the one raised. Nothing
looked at it, so nothing noticed.

`tests/trap_verbs.c` now redirects `akerr_log_method` into a buffer and drives
the path with a sink that refuses every write, which is the only way left to
make reporting fail now that the `TRAP` dispatch no longer allocates. It asserts
the log line and that `errclass` still records the program's own error, and it
kills the mutant.

TODO.md gains a section for what the run found and what it did not: it was cut
off at 551 of 997 mutants after ninety minutes, so 446 are unexamined and a full
run belongs in the release workflow rather than here. The three regions this
work touched were all covered, and the survivors in the skipped-block guard are
named and assessed -- three of the four are equivalent for any program that can
actually be written, and the one that is not would need a test with a loop on
line 1.

Also recorded: src/variable.c at 68.2% with every mutant on the inline-storage
lines killed, and src/runtime_trap.c at 82.5% with no survivors in
`set_error_variables()`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-02 02:17:17 -04:00
parent ad76889292
commit 89fca8007b
2 changed files with 128 additions and 0 deletions

33
TODO.md
View File

@@ -1858,6 +1858,39 @@ the reference's semantics reproduced faithfully; the third is the port's own.
reentrant" and "do not grow the runtime by a third for a scratch buffer" are both right
and picking between them is a decision.
### Mutation-testing gaps in the files the game work touched
Recorded the way §8's CI note records `src/audio_tables.c`: a real test gap rather than a
reason to avoid the file.
34. **`src/runtime.c` has never had a complete mutation run, and the one attempted here
was cut off at 551 of 997 mutants** after ninety minutes. What it did cover included
all three regions the value-pool, trap and skipped-block work touched, and the
survivors there are worth naming:
- `report_and_reraise()`: **closed.** Deleting the `LOG_ERROR_WITH_MESSAGE` survived,
because nothing looked at the log -- and that line *is* the visible half of item
33's second fix. `tests/trap_verbs.c` now redirects `akerr_log_method` into a buffer
and drives the path with a sink that refuses every write, which kills it.
- The skipped-block guard (`src/runtime.c`, the `BEND` test): four survivors, and
three of them are equivalent for any program that can be written -- `loopFirstLine
!= 0` mutated to `!= 1` differs only for a loop on line 1, `waitingForCommand[0]`
to `[1]` only for a one-character wait, and flipping `strcmp(...) == 0` to `!= 0`
moves the release from the `FOR` to the `NEXT` that must follow it. A test with a
loop on line 1 would kill the first. The other two are noted rather than chased.
- A NULL-argument guard in `akbasic_runtime_reserve_globals()`, which is the same
survivor class every file in this tree has: nothing calls these with NULL.
The remaining 446 mutants are unexamined. A full run is a release-workflow job, not a
per-commit one -- `.gitea/workflows/release.yaml` already mutates the whole tree -- and
what this entry is for is that the *partial* result has been read rather than filed
unopened.
Measured alongside: `src/variable.c` scores 68.2% with **every mutant on the inline-
storage lines killed**, and `src/runtime_trap.c` 82.5% with **no survivors** in
`set_error_variables()`. Both files' remaining survivors are pre-existing and are
mostly NULL guards and `MAX - 1` arithmetic in string helpers.
### Found while writing a game against the interpreter
A complete Breakout, sprites and text grid and keyboard, running for minutes at a time. Nothing