Move BASIC fixtures into the editable language corpus
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s
Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths. Co-authored-by: andrew <andrew@aklabs.net>
This commit is contained in:
@@ -81,7 +81,7 @@ static void test_counter_survives_the_loop(void)
|
||||
* `FOR I = 1 TO 1` executes the body one time on every BASIC there has ever
|
||||
* been. Here the entry test treats "the counter has reached the limit" as
|
||||
* "do not enter", so the body is skipped entirely -- and
|
||||
* tests/reference/language/flowcontrol/forloopwaitingforcommand.bas pins that,
|
||||
* tests/language/flowcontrol/forloopwaitingforcommand.bas pins that,
|
||||
* which is why this cannot simply be corrected.
|
||||
*/
|
||||
static void test_single_iteration_loop(void)
|
||||
|
||||
16
tests/language/README.md
Normal file
16
tests/language/README.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# The language test corpus
|
||||
|
||||
The `.bas` files in this directory and their sibling `.txt` files are the editable language
|
||||
corpus. CMake registers every program as an individual `local_*` CTest case and compares its
|
||||
output with the sibling expectation.
|
||||
|
||||
The corpus includes cases carried over from the deprecated Go implementation as well as cases
|
||||
written for this interpreter. Their provenance is useful when investigating a regression, but it
|
||||
does not make any case immutable or turn the old implementation into a specification.
|
||||
|
||||
**Change a program and its expectation deliberately and in the same commit.** If the changed
|
||||
output represents an intentional language decision, record the reason in `TODO.md` or the
|
||||
relevant documentation. If it is an accidental change, fix the interpreter instead.
|
||||
|
||||
The `examples/` subdirectory contains complete BASIC programs used as worked examples. It is
|
||||
part of the same corpus and follows the same `.bas`/`.txt` pairing rule.
|
||||
@@ -1,63 +0,0 @@
|
||||
# The reference interpreter's own test corpus
|
||||
|
||||
These 41 `.bas` files and their 41 `.txt` expectations are **not this project's tests.** They
|
||||
are a byte-for-byte copy of the acceptance corpus belonging to the Go implementation this
|
||||
interpreter was ported from:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Source | `https://source.starfort.tech/andrew/basicinterpreter`, `tests/` |
|
||||
| Copied at | commit `d76162cb37eeaccddae2cf6cab7af7bc8175d6d2` ("Update README.md") |
|
||||
| Copied on | 2026-07-31 |
|
||||
| Verified | every file `cmp`-identical to the submodule at the time of the copy |
|
||||
|
||||
They are registered as the `golden_*` CTest cases and run against `basic` on every build, in
|
||||
both the default and the `-DAKBASIC_WITH_AKGL=ON` configurations.
|
||||
|
||||
## Why they are a copy, when they deliberately were not
|
||||
|
||||
Until 2026-07-31 these were driven *in place* out of `deps/basicinterpret`, and the reasoning
|
||||
was written down in `CMakeLists.txt`: the corpus is a submodule, and copying a submodule's
|
||||
corpus guarantees drift.
|
||||
|
||||
That reasoning was sound and it stopped applying. The Go project is deprecated and will not be
|
||||
updated, so there is nothing left to drift *from* — and a build that cannot run its own
|
||||
acceptance suite without cloning the implementation it replaced is not finished.
|
||||
|
||||
## What these are for now
|
||||
|
||||
**A regression suite, not a specification.** That distinction is the whole of how to treat
|
||||
them, and it changed on 2026-07-31 (`TODO.md` §0.1): matching the Go implementation is no
|
||||
longer a goal, so a difference between it and this interpreter is not automatically a bug in
|
||||
this one.
|
||||
|
||||
What they are still good for is unchanged and is worth a lot: forty-one real BASIC programs
|
||||
with known-good output, covering arithmetic, arrays, flow control, every builtin function,
|
||||
`READ`/`DATA`, labels and three worked examples. That catches regressions whatever its
|
||||
provenance.
|
||||
|
||||
## The rule for editing these
|
||||
|
||||
**Diverge deliberately, never by accident.**
|
||||
|
||||
A failure here means this interpreter's behaviour changed. Nine times in ten that is a bug you
|
||||
just introduced and the answer is to fix the code. The tenth time it is an improvement on the
|
||||
reference, and then the answer is:
|
||||
|
||||
1. Change the expectation in the same commit as the code.
|
||||
2. Say why in the commit message.
|
||||
3. Add a line to `TODO.md` §5, which is the list of deliberate deviations.
|
||||
|
||||
What is not acceptable is editing a `.txt` to make a red suite green without deciding which of
|
||||
the two you are doing.
|
||||
|
||||
New cases for this project's own behaviour go in `tests/language/`, registered under `local_*`.
|
||||
The prefix says which corpus a failure came from, and a diff touching `tests/reference/` stands
|
||||
out as the thing it is.
|
||||
|
||||
## Divergences so far
|
||||
|
||||
| Case | Change | Why |
|
||||
|---|---|---|
|
||||
| `language/arithmetic/integer.txt` | Expectation grew from `4 4 2` to `4 4 2 2` | The program has **four** `PRINT` statements and the expectation had three values. The file ends in a blank line, and both interpreters filed a blank line under the loader's cursor — which for a line with no number of its own is the number of the line before it. So `40 PRINT 4 - 2` was erased before the program ran. The reference lost it the same way, which is why the expectation was written short. Fixed by skipping blank lines in RUNSTREAM, as `akbasic_runtime_load()` and `DLOAD` already did. `TODO.md` §5. |
|
||||
| `examples/strreverse.bas` | Variable `INPUT$` renamed to `SOURCE$` | `INPUT$` is a reserved word with a type suffix, which a real C128 refuses and which this interpreter now refuses too (`TODO.md` §6 item 16). The reference accepted it only because its own reserved-word check never fired. The expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — so the case keeps all of its value. |
|
||||
@@ -358,7 +358,7 @@ static void test_skipped_block_releases_its_loop_scope(void)
|
||||
* inner `NEXT`, so the outer `NEXT` still finds its own `FOR`. Releasing it
|
||||
* turns this into "NEXT outside the context of FOR".
|
||||
*
|
||||
* tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas is the
|
||||
* tests/language/flowcontrol/nestedforloopwaitingforcommand.bas is the
|
||||
* golden case that caught exactly that, and this is the same shape asserted
|
||||
* where a reader of this file will see it.
|
||||
*/
|
||||
|
||||
@@ -182,7 +182,7 @@ static void test_runstream_assigns_the_same_slots(void)
|
||||
/**
|
||||
* @brief A file ending in a blank line keeps its last line.
|
||||
*
|
||||
* `tests/reference/language/arithmetic/integer.bas` is this shape, and both this
|
||||
* `tests/language/arithmetic/integer.bas` is this shape, and both this
|
||||
* interpreter and the reference used to erase the last line before running it --
|
||||
* the blank line was filed under the cursor, which for a line with no number of
|
||||
* its own is the number of the line before it.
|
||||
|
||||
Reference in New Issue
Block a user