Move BASIC fixtures into the editable language corpus
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s

Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths.

Co-authored-by: andrew <andrew@aklabs.net>
This commit is contained in:
2026-08-04 16:21:59 -04:00
parent fac84acdaa
commit 8a02674af5
96 changed files with 69 additions and 168 deletions

View File

@@ -81,7 +81,7 @@ static void test_counter_survives_the_loop(void)
* `FOR I = 1 TO 1` executes the body one time on every BASIC there has ever
* been. Here the entry test treats "the counter has reached the limit" as
* "do not enter", so the body is skipped entirely -- and
* tests/reference/language/flowcontrol/forloopwaitingforcommand.bas pins that,
* tests/language/flowcontrol/forloopwaitingforcommand.bas pins that,
* which is why this cannot simply be corrected.
*/
static void test_single_iteration_loop(void)

16
tests/language/README.md Normal file
View File

@@ -0,0 +1,16 @@
# The language test corpus
The `.bas` files in this directory and their sibling `.txt` files are the editable language
corpus. CMake registers every program as an individual `local_*` CTest case and compares its
output with the sibling expectation.
The corpus includes cases carried over from the deprecated Go implementation as well as cases
written for this interpreter. Their provenance is useful when investigating a regression, but it
does not make any case immutable or turn the old implementation into a specification.
**Change a program and its expectation deliberately and in the same commit.** If the changed
output represents an intentional language decision, record the reason in `TODO.md` or the
relevant documentation. If it is an accidental change, fix the interpreter instead.
The `examples/` subdirectory contains complete BASIC programs used as worked examples. It is
part of the same corpus and follows the same `.bas`/`.txt` pairing rule.

View File

@@ -1,63 +0,0 @@
# The reference interpreter's own test corpus
These 41 `.bas` files and their 41 `.txt` expectations are **not this project's tests.** They
are a byte-for-byte copy of the acceptance corpus belonging to the Go implementation this
interpreter was ported from:
| | |
|---|---|
| Source | `https://source.starfort.tech/andrew/basicinterpreter`, `tests/` |
| Copied at | commit `d76162cb37eeaccddae2cf6cab7af7bc8175d6d2` ("Update README.md") |
| Copied on | 2026-07-31 |
| Verified | every file `cmp`-identical to the submodule at the time of the copy |
They are registered as the `golden_*` CTest cases and run against `basic` on every build, in
both the default and the `-DAKBASIC_WITH_AKGL=ON` configurations.
## Why they are a copy, when they deliberately were not
Until 2026-07-31 these were driven *in place* out of `deps/basicinterpret`, and the reasoning
was written down in `CMakeLists.txt`: the corpus is a submodule, and copying a submodule's
corpus guarantees drift.
That reasoning was sound and it stopped applying. The Go project is deprecated and will not be
updated, so there is nothing left to drift *from* — and a build that cannot run its own
acceptance suite without cloning the implementation it replaced is not finished.
## What these are for now
**A regression suite, not a specification.** That distinction is the whole of how to treat
them, and it changed on 2026-07-31 (`TODO.md` §0.1): matching the Go implementation is no
longer a goal, so a difference between it and this interpreter is not automatically a bug in
this one.
What they are still good for is unchanged and is worth a lot: forty-one real BASIC programs
with known-good output, covering arithmetic, arrays, flow control, every builtin function,
`READ`/`DATA`, labels and three worked examples. That catches regressions whatever its
provenance.
## The rule for editing these
**Diverge deliberately, never by accident.**
A failure here means this interpreter's behaviour changed. Nine times in ten that is a bug you
just introduced and the answer is to fix the code. The tenth time it is an improvement on the
reference, and then the answer is:
1. Change the expectation in the same commit as the code.
2. Say why in the commit message.
3. Add a line to `TODO.md` §5, which is the list of deliberate deviations.
What is not acceptable is editing a `.txt` to make a red suite green without deciding which of
the two you are doing.
New cases for this project's own behaviour go in `tests/language/`, registered under `local_*`.
The prefix says which corpus a failure came from, and a diff touching `tests/reference/` stands
out as the thing it is.
## Divergences so far
| Case | Change | Why |
|---|---|---|
| `language/arithmetic/integer.txt` | Expectation grew from `4 4 2` to `4 4 2 2` | The program has **four** `PRINT` statements and the expectation had three values. The file ends in a blank line, and both interpreters filed a blank line under the loader's cursor — which for a line with no number of its own is the number of the line before it. So `40 PRINT 4 - 2` was erased before the program ran. The reference lost it the same way, which is why the expectation was written short. Fixed by skipping blank lines in RUNSTREAM, as `akbasic_runtime_load()` and `DLOAD` already did. `TODO.md` §5. |
| `examples/strreverse.bas` | Variable `INPUT$` renamed to `SOURCE$` | `INPUT$` is a reserved word with a type suffix, which a real C128 refuses and which this interpreter now refuses too (`TODO.md` §6 item 16). The reference accepted it only because its own reserved-word check never fired. The expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — so the case keeps all of its value. |

View File

@@ -358,7 +358,7 @@ static void test_skipped_block_releases_its_loop_scope(void)
* inner `NEXT`, so the outer `NEXT` still finds its own `FOR`. Releasing it
* turns this into "NEXT outside the context of FOR".
*
* tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas is the
* tests/language/flowcontrol/nestedforloopwaitingforcommand.bas is the
* golden case that caught exactly that, and this is the same shape asserted
* where a reader of this file will see it.
*/

View File

@@ -182,7 +182,7 @@ static void test_runstream_assigns_the_same_slots(void)
/**
* @brief A file ending in a blank line keeps its last line.
*
* `tests/reference/language/arithmetic/integer.bas` is this shape, and both this
* `tests/language/arithmetic/integer.bas` is this shape, and both this
* interpreter and the reference used to erase the last line before running it --
* the blank line was filed under the cursor, which for a line with no number of
* its own is the number of the line before it.