`copy_tree()` excluded `*.png` from the scratch tree, which was right when the
only PNGs anywhere were libpng's and SDL_image's test corpora -- 39 MB that
nothing in this build reads. `docs/images/` changed that: `docs_examples`
asserts that every `screenshot=` block has its figure, so a copy with no images
fails thirteen blocks, the *baseline* comes back red, and the run aborts with
"Fix the suite before mutation testing" before mutating anything.
Mutation testing has therefore been unrunnable since the figures landed, and it
fails in the one way this repository's own notes warn about: looking like the
thing it measures is broken.
Keep the project's PNGs, still drop the dependencies'. The pattern list cannot
express that -- `ignore_patterns` matches basenames -- so it becomes a function
that adds the `*.png` rule only under `deps/`.
Verified: `--target src/symtab.c --max-mutants 2` now reports "Baseline OK." and
scores both mutants, where every invocation before this aborted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All 41 .bas/.txt pairs copied byte for byte from basicinterpreter@d76162c into
tests/reference/, plus the Commodore font into assets/fonts/ -- the two things
that tied the build to deps/basicinterpret. Both were verified cmp-identical
to the submodule copies.
This reverses a decision that was deliberate and correct at the time: the
corpus was driven in place because copying a submodule's corpus guarantees
drift. Overruled on purpose -- the Go dependency is being deprecated, and a
build that cannot run its acceptance suite without cloning the implementation
it replaced is not finished. tests/reference/README.md records what the drift
now costs and that those expectations are never edited.
Checked rather than assumed: both configurations configure, build and pass
from scratch with deps/basicinterpret moved out of the tree entirely.
The submodule is kept as the behavioural spec, which is a real use. The font
came with an open licence question; assets/fonts/PROVENANCE.md states it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ports libakstdlib's mutation harness (the newest of the three) to scripts/
mutation_test.py, adds the namespaced `mutation` CMake target the sibling
libraries have, and splits CI in two.
.gitea/workflows/ci.yaml keeps the push path: suite, ASan+UBSan, coverage, and a
mutation run bounded to src/convert.c and src/symtab.c -- about four minutes,
scoring 77.8% against a gate of 65.
.gitea/workflows/release.yaml is new and manual (workflow_dispatch). It carries
the doxygen gate, moved out of ci.yaml, and a whole-tree mutation run: 3675
mutants and hours of runner time, which is a release cost rather than a
per-commit one. Both artifacts a release wants -- api-documentation and
mutation-report -- come out of it. Two optional inputs narrow the run or change
the threshold; they arrive through the environment rather than being
interpolated into the shell, because ${{ }} substitution happens before the
shell sees the line.
The harness paid for itself immediately, which is the point of it. Three real
gaps, each checked to be a genuine bug rather than an equivalent mutant:
- errno was never asserted clear before a strtoll. Confirmed with a standalone
probe that strtoll leaves a stale errno untouched on success, so without the
`errno = 0` a valid conversion raises ERANGE.
- Nothing exercised a maximum-length symbol-table key, so every MAX_KEY - 1
off-by-one in a strncpy and its NUL terminator survived. The same hole exists
for strings in src/value.c and is filed.
- Nothing asserted a freshly initialised table was actually zeroed.
Closing the first two took the measured score from 73.1% to 77.8%.
I set the push-path threshold to 75 first, on an estimate. Measuring gave 73.1%
and the job would have failed on its first run -- the earlier per-file figure was
too high because the captured output had been truncated to its last lines and I
counted fewer survivors than there were. It is 65 now, and the gate was run as
written and confirmed to exit 0.
src/value.c is the file most worth mutating and is deliberately off the push
path: 368 mutants at ~11s each is about 70 minutes, because almost everything
links against it. A partial run over it found the same maximum-length-string
hole plus two genuinely equivalent mutants that only exist because of reference
defect section 6 item 5 -- adding both of the right operand's numeric fields
works only while the unused one is zero, so + and - are interchangeable there.
Recorded in TODO.md.
ctest 61/61; doxygen exits 0; both workflows' steps were executed locally, both
input paths included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
