d2ae0d75de5e0600b3556776cdf4d55ab499e64e
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
506af093a1
|
Add mutation testing, and split the release gates into their own workflow
Ports libakstdlib's mutation harness (the newest of the three) to scripts/
mutation_test.py, adds the namespaced `mutation` CMake target the sibling
libraries have, and splits CI in two.
.gitea/workflows/ci.yaml keeps the push path: suite, ASan+UBSan, coverage, and a
mutation run bounded to src/convert.c and src/symtab.c -- about four minutes,
scoring 77.8% against a gate of 65.
.gitea/workflows/release.yaml is new and manual (workflow_dispatch). It carries
the doxygen gate, moved out of ci.yaml, and a whole-tree mutation run: 3675
mutants and hours of runner time, which is a release cost rather than a
per-commit one. Both artifacts a release wants -- api-documentation and
mutation-report -- come out of it. Two optional inputs narrow the run or change
the threshold; they arrive through the environment rather than being
interpolated into the shell, because ${{ }} substitution happens before the
shell sees the line.
The harness paid for itself immediately, which is the point of it. Three real
gaps, each checked to be a genuine bug rather than an equivalent mutant:
- errno was never asserted clear before a strtoll. Confirmed with a standalone
probe that strtoll leaves a stale errno untouched on success, so without the
`errno = 0` a valid conversion raises ERANGE.
- Nothing exercised a maximum-length symbol-table key, so every MAX_KEY - 1
off-by-one in a strncpy and its NUL terminator survived. The same hole exists
for strings in src/value.c and is filed.
- Nothing asserted a freshly initialised table was actually zeroed.
Closing the first two took the measured score from 73.1% to 77.8%.
I set the push-path threshold to 75 first, on an estimate. Measuring gave 73.1%
and the job would have failed on its first run -- the earlier per-file figure was
too high because the captured output had been truncated to its last lines and I
counted fewer survivors than there were. It is 65 now, and the gate was run as
written and confirmed to exit 0.
src/value.c is the file most worth mutating and is deliberately off the push
path: 368 mutants at ~11s each is about 70 minutes, because almost everything
links against it. A partial run over it found the same maximum-length-string
hole plus two genuinely equivalent mutants that only exist because of reference
defect section 6 item 5 -- adding both of the right operand's numeric fields
works only while the unused one is zero, so + and - are interchangeable there.
Recorded in TODO.md.
ctest 61/61; doxygen exits 0; both workflows' steps were executed locally, both
input paths included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|||
|
f8c15c2f2c
|
Port the BASIC interpreter from Go to C
Reproduces deps/basicinterpret in C, in the idiom of the ak* libraries. All 41
.bas files in the reference's corpus produce byte-identical stdout, including
the trailing double newline on an error line -- that comes from basicError
building a string ending in \n and handing it to Println, and
array_outofbounds.txt encodes it.
The corpus is driven in place from the submodule as 41 individual CTest cases
rather than copied, so it cannot drift from upstream. Eighteen unit tests cover
what the corpus cannot reach.
Three structural changes carry most of the work. Go's three reflection lookups
(Command*, Function*, ParseCommand*) become one sorted dispatch table in
src/verbs.c searched with bsearch; adding a verb is a row and two functions. The
five Go maps become one fixed open-addressed table over aksl_strhash_djb2. And
run(), which owned the process until MODE_QUIT, splits into step() plus a
bounded run() -- goal 3 requires a host game to be able to bound execution, and
nothing in the library now terminates the process or touches SDL.
Output goes through an akbasic_TextSink vtable. src/sink_stdio.c is what makes
the corpus runnable with no SDL present; the akgl-backed sink is still to come
and is blocked on libakgl having no text-measurement call.
src/convert.c exists because libakstdlib's aksl_ato* family cannot report a
conversion failure (its TODO.md 2.1.5). The reference checks strconv's error at
four sites and turns it into a BASIC error; routing those through aksl_atoi
would have turned four diagnosable errors into wrong answers, with VAL("garbage")
quietly returning 0. TODO.md 1.9 records which libakstdlib calls are cleared for
use here and which are not.
Reference defects are reproduced, not fixed: the golden files encode the observed
behaviour and a silent correction is a behaviour change. TODO.md section 6 lists
sixteen, and tests/known_reference_defects.c asserts the *correct* contract for
six of them under AKBASIC_KNOWN_FAILING_TESTS, so a fix shows up as
"unexpectedly passed". Five of the sixteen were found by this port and are new:
subtraction stops after one operator so 1-2-3 computes 1-2 and abandons the rest
of the line (a wrong answer, not a refused one); a unary-minus argument inflates
a function's arity so ABS(-9) is rejected; a comparison operator in a line's
final column is dropped; hex literals never survive the scanner; and the
"Reserved word in variable name" check is dead code.
Where the reference reaches undefined behaviour by a route that is defined in Go
-- an out-of-range shift, a negative string multiplier, integer division by zero
-- this raises instead of inheriting the UB. No golden case exercises any of
them.
The top-level CMakeLists shadows add_test, set_tests_properties and
add_custom_target around all three add_subdirectory calls. Without it libakerror's
tests land in our suite as Not Run, and its un-namespaced `coverage` target stops
a coverage build from configuring at all. Test targets are akbasic_test_<name>:
bare test_<name> collides with libakstdlib's, which is what broke libakgl's
configure in c2b16d3.
ctest 59/59; ASan+UBSan 59/59; 92.3% line and 96.9% function coverage; no
warnings under -Wall -Wextra. Branch coverage is not a target, for the reason
libakstdlib and libakgl both record: the akerror macros expand into large branch
trees at every call site.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
