Move BASIC fixtures into the editable language corpus
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s
Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths. Co-authored-by: andrew <andrew@aklabs.net>
This commit is contained in:
37
TODO.md
37
TODO.md
@@ -38,7 +38,7 @@ What it changes:
|
||||
- **§1.8's message-text contract is now a convention.** Improving a message is allowed; it costs
|
||||
a golden file, which is a cost rather than a veto.
|
||||
- **§5's bar drops** from "defensible against the golden suite" to defensible on its own merits.
|
||||
- **`tests/reference/` becomes a regression suite rather than a specification.** Diverging from
|
||||
- **`tests/language/` is the editable language corpus rather than a protected specification.** Diverging from
|
||||
it is allowed and must be deliberate and recorded — see its README.
|
||||
|
||||
What it does **not** change:
|
||||
@@ -306,13 +306,13 @@ to `Println`, which adds another.
|
||||
**This used to be a hard contract and is now a default.** The Go implementation is deprecated
|
||||
and will not be updated, so the two projects are no longer required to match — see §0.1. What
|
||||
survives is the practical half: these strings and this newline behaviour are what every
|
||||
expectation in `tests/reference/` was written against, so changing one means changing golden
|
||||
expectation in `tests/language/` was written against, so changing one means changing the paired
|
||||
files, and that is worth doing on purpose rather than by accident. A message that reads
|
||||
awkwardly *may* now be improved; do it deliberately, move the expectations in the same commit,
|
||||
and add a line to §5.
|
||||
|
||||
Numeric formatting still matches the reference: integers via `%" PRId64 "`, floats via `%f`
|
||||
(Go's `%f` and C's `%f` both give six decimals — `tests/reference/language/arithmetic/float.txt`
|
||||
(Go's `%f` and C's `%f` both give six decimals — `tests/language/arithmetic/float.txt`
|
||||
confirms). No reason to change it, which is different from not being allowed to.
|
||||
|
||||
### 1.9 Which `libakstdlib` calls are cleared for use — **the bans are lifted**
|
||||
@@ -375,7 +375,7 @@ Phases 0 through 6 of the original plan are done. The interpreter builds clean u
|
||||
|
||||
It *did* reproduce the reference byte for byte, and that claim is retired rather than broken:
|
||||
§0.1 released it, and one case has since diverged deliberately (§6 item 16, listed in
|
||||
`tests/reference/README.md`). Everything else still matches, which is worth knowing but is no
|
||||
`tests/language/README.md`). Everything else still matches, which is worth knowing but is no
|
||||
longer a gate.
|
||||
|
||||
```sh
|
||||
@@ -408,17 +408,16 @@ and the only path that existed — `AKBASIC_MODE_RUNSTREAM` reading through the
|
||||
is the code the README quotes, built by every build and registered as a CTest case so a
|
||||
signature change breaks the build rather than rotting the document.
|
||||
|
||||
**The acceptance suite is the reference's own corpus, checked in at `tests/reference/`.** All
|
||||
41 `.bas` files are registered as individual CTest cases and byte-compared against their `.txt`
|
||||
**The acceptance suite is the editable language corpus, checked in at `tests/language/`.** All
|
||||
65 `.bas` files are registered as individual CTest cases and compared against their `.txt`
|
||||
— including the trailing double newline on an error line (§1.8).
|
||||
|
||||
It was driven *in place* out of `deps/basicinterpret` until 2026-07-31, on the reasoning that
|
||||
copying a submodule's corpus guarantees drift. That reasoning was sound and was overruled
|
||||
deliberately: the Go dependency is being deprecated, and a build that cannot run its own
|
||||
acceptance suite without cloning the implementation it replaced is not finished. The copy is
|
||||
byte-identical to `basicinterpreter@d76162c`, and `tests/reference/README.md` records the
|
||||
provenance, the cost of the drift nobody is watching for now, and the rule that those
|
||||
expectations are never edited to suit this interpreter.
|
||||
originally byte-identical to `basicinterpreter@d76162c`, and `tests/language/README.md` records the
|
||||
provenance and the rule that programs and expectations are edited together deliberately.
|
||||
|
||||
**Nothing in the build or the suite needs `deps/basicinterpret` any more**, and that is checked
|
||||
rather than assumed — both configurations were configured, built and run from scratch with the
|
||||
@@ -1026,7 +1025,7 @@ deviations from the reference's *program*: `main.go` and the SDL half of
|
||||
|
||||
**What it costs a program:** a listing that used `INPUT$`, `LEN#` or `GOTO%` as a variable
|
||||
stops parsing, and the fix is to rename the variable. One case in the reference's own corpus
|
||||
did exactly that; see `tests/reference/README.md`.
|
||||
did exactly that; see `tests/language/examples/strreverse.bas`.
|
||||
|
||||
### Deviations in statement separation
|
||||
|
||||
@@ -1391,10 +1390,10 @@ deviations from the reference's *program*: `main.go` and the SDL half of
|
||||
**This one moved a golden file.** A line with no number used to be filed under the loader's
|
||||
cursor unchanged — that is, on top of the line before it — so two unnumbered lines in a row
|
||||
silently lost the first, and a *blank* line erased whatever preceded it. The reference does
|
||||
the same, and `tests/reference/language/arithmetic/integer.bas` is the proof: four `PRINT`
|
||||
the same, and `tests/language/arithmetic/integer.bas` is the proof: four `PRINT`
|
||||
statements, an expectation with three values, and a trailing blank line that erased
|
||||
`40 PRINT 4 - 2` before the program ran. The expectation is now `4 4 2 2` and
|
||||
`tests/reference/README.md` records it.
|
||||
`tests/language/README.md` records it.
|
||||
|
||||
In its place: `akbasic_runtime_file_line()` (`src/runtime.c`) is the one implementation of
|
||||
the rule, shared by `akbasic_runtime_load()`, RUNSTREAM and `DLOAD`. A numbered line is
|
||||
@@ -1586,9 +1585,9 @@ be reproduced before it can be fixed.
|
||||
|
||||
**It cost one golden case, exactly as predicted, and the cost turned out to be nothing.**
|
||||
The reference's `examples/strreverse.bas` names a variable `INPUT$`; the variable is renamed
|
||||
to `SOURCE$` in `tests/reference/`, the expectation is byte-for-byte unchanged — the program
|
||||
to `SOURCE$` in `tests/language/examples/strreverse.bas`, the expectation is byte-for-byte unchanged — the program
|
||||
still prints `REVERSED: OLLEH` — and the case keeps every bit of its coverage. Recorded in
|
||||
`tests/reference/README.md`'s divergence table, which this is the first entry in.
|
||||
`tests/language/README.md` records the corpus editing rule.
|
||||
|
||||
This item sat parked because the corpus was the acceptance contract and lived in a submodule
|
||||
this repository could not edit. Both premises are gone: the corpus is checked in, and
|
||||
@@ -2232,10 +2231,10 @@ update --init --recursive` gets them.
|
||||
|
||||
## 8. Status
|
||||
|
||||
**The port is done.** The C interpreter passes the Go reference's entire corpus and passes clean
|
||||
**The port is done.** The C interpreter passes the language corpus and passes clean
|
||||
under ASan and UBSan. It reproduced that corpus byte for byte until §0.1 retired the
|
||||
requirement; two cases have diverged on purpose since, and
|
||||
`tests/reference/README.md` lists them.
|
||||
`tests/language/README.md` lists them.
|
||||
|
||||
| Gate | Result |
|
||||
|---|---|
|
||||
@@ -2243,7 +2242,7 @@ requirement; two cases have diverged on purpose since, and
|
||||
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 112/112 headless, with `akgl_typing` skipping itself. The same set minus the four `no_device` cases the SDL driver contradicts, plus `akgl_backends`, `akgl_frontend`, `docs_screenshots` and `akgl_typing` — the last of which is the skip, and the `akgl_build` CI job is where it skips |
|
||||
| `docs_examples` | Every fenced block in `README.md`, `MAINTENANCE.md` and `docs/` executed and byte-compared: 71 programs, 9 transcripts, 79 output comparisons, 4 C snippets, 2 excerpts, 2 shell blocks and 19 figures in the default build. The C-snippet count reads 0 when the harness is run by hand without `--cflags-file`; CTest passes it. `MAINTENANCE.md` documents the fence-tag convention |
|
||||
| `docs_screenshots` | 19/19 figures re-rendered and byte-identical to the checked-in PNGs. AKGL build only — rendering a picture needs the SDL half |
|
||||
| Golden corpus | 41/41 byte-exact from `tests/reference/` — **and 41/41 again through the SDL binary**, which is most of what proves the frontend changes no output |
|
||||
| Language corpus | 65/65 paired expectations — **and 65/65 again through the SDL binary**, which is most of what proves the frontend changes no output |
|
||||
| ASan + UBSan | 112/112 |
|
||||
| Line coverage | 94.1% (7227/7681) — above the 90% gate |
|
||||
| Function coverage | 97.9% (474/484) |
|
||||
@@ -2553,7 +2552,7 @@ What remains, in priority order:
|
||||
this whole arrangement exists against.
|
||||
|
||||
The byte comparison is a deliberate bet that the dummy driver and the software renderer
|
||||
are reproducible, which is the same bet `tests/reference/` already makes about golden
|
||||
are reproducible, which is the same bet `tests/language/` makes about golden
|
||||
output. Verified run-to-run and build-to-build here; what is untested is an SDL upgrade
|
||||
that moves one pixel of a diagonal. **If that happens, regenerate the figures in the same
|
||||
commit as the bump** — do not weaken the test to a size check, which would pass for every
|
||||
@@ -2636,7 +2635,7 @@ reduced against `build/basic`, the stdio build, unless it says otherwise.
|
||||
|
||||
**It is narrower than it looks, and the golden corpus is why.** The first attempt
|
||||
released the scope on *any* skip, which broke
|
||||
`tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas`: a zero-
|
||||
`tests/language/flowcontrol/nestedforloopwaitingforcommand.bas`: a zero-
|
||||
iteration `FOR` skips its body by the same mechanism, and there the orphan is
|
||||
load-bearing -- it is what absorbs the inner `NEXT` so the outer one still finds its
|
||||
`FOR`. Releasing it turns that case into "NEXT outside the context of FOR". So the
|
||||
|
||||
Reference in New Issue
Block a user