From 8a02674af5c07ce148a528427e32f2f14845e3d1 Mon Sep 17 00:00:00 2001 From: Logikoma Date: Tue, 4 Aug 2026 16:21:59 -0400 Subject: [PATCH] Move BASIC fixtures into the editable language corpus Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths. Co-authored-by: andrew --- .gitea/workflows/ci.yaml | 10 ++- CLAUDE.md | 6 +- CMakeLists.txt | 65 ++++--------------- MAINTENANCE.md | 15 ++--- README.md | 6 +- TODO.md | 37 +++++------ docs/14-architecture.md | 4 +- scripts/mutation_test.py | 5 +- src/runtime.c | 4 +- tests/for_semantics.c | 2 +- tests/language/README.md | 16 +++++ .../language/arithmetic/float.bas | 0 .../language/arithmetic/float.txt | 0 .../language/arithmetic/integer.bas | 0 .../language/arithmetic/integer.txt | 0 .../language/array_multidimensional.bas | 0 .../language/array_multidimensional.txt | 0 .../language/array_outofbounds.bas | 0 .../language/array_outofbounds.txt | 0 tests/{reference => }/language/arrays.bas | 0 tests/{reference => }/language/arrays.txt | 0 .../caseinsensitivefunctionsandcommands.bas | 0 .../caseinsensitivefunctionsandcommands.txt | 0 .../examples/bubblesort.bas | 0 .../examples/bubblesort.txt | 0 .../examples/factorial.bas | 0 .../examples/factorial.txt | 0 .../examples/strreverse.bas | 0 .../examples/strreverse.txt | 0 .../language/flowcontrol/forloop.bas | 0 .../language/flowcontrol/forloop.txt | 0 .../flowcontrol/forloopwaitingforcommand.bas | 0 .../flowcontrol/forloopwaitingforcommand.txt | 0 .../language/flowcontrol/goto.bas | 0 .../language/flowcontrol/goto.txt | 0 .../language/flowcontrol/ifthenelse.bas | 0 .../language/flowcontrol/ifthenelse.txt | 0 .../language/flowcontrol/nestedforloops.bas | 0 .../language/flowcontrol/nestedforloops.txt | 0 .../nestedforloopwaitingforcommand.bas | 0 .../nestedforloopwaitingforcommand.txt | 0 tests/{reference => }/language/functions.bas | 0 tests/{reference => }/language/functions.txt | 0 .../language/functions/atn.bas | 0 .../language/functions/atn.txt | 0 .../language/functions/chr.bas | 0 .../language/functions/chr.txt | 0 .../language/functions/cos.bas | 0 .../language/functions/cos.txt | 0 .../language/functions/hex.bas | 0 .../language/functions/hex.txt | 0 .../language/functions/instr.bas | 0 .../language/functions/instr.txt | 0 .../language/functions/left.bas | 0 .../language/functions/left.txt | 0 .../language/functions/len.bas | 0 .../language/functions/len.txt | 0 .../language/functions/log.bas | 0 .../language/functions/log.txt | 0 .../language/functions/mid.bas | 0 .../language/functions/mid.txt | 0 .../language/functions/mod.bas | 0 .../language/functions/mod.txt | 0 .../language/functions/pointer.bas | 0 .../language/functions/pointer.txt | 0 .../language/functions/rad.bas | 0 .../language/functions/rad.txt | 0 .../language/functions/right.bas | 0 .../language/functions/right.txt | 0 .../language/functions/sgn.bas | 0 .../language/functions/sgn.txt | 0 .../language/functions/shl.bas | 0 .../language/functions/shl.txt | 0 .../language/functions/shr.bas | 0 .../language/functions/shr.txt | 0 .../language/functions/sin.bas | 0 .../language/functions/sin.txt | 0 .../language/functions/spc.bas | 0 .../language/functions/spc.txt | 0 .../language/functions/str.bas | 0 .../language/functions/str.txt | 0 .../language/functions/tan.bas | 0 .../language/functions/tan.txt | 0 .../language/functions/val.bas | 0 .../language/functions/val.txt | 0 .../language/functions/xor.bas | 0 .../language/functions/xor.txt | 0 .../language/functions_multiline.bas | 0 .../language/functions_multiline.txt | 0 tests/{reference => }/language/label.bas | 0 tests/{reference => }/language/label.txt | 0 tests/{reference => }/language/read_data.bas | 0 tests/{reference => }/language/read_data.txt | 0 tests/reference/README.md | 63 ------------------ tests/structure_verbs.c | 2 +- tests/unnumbered.c | 2 +- 96 files changed, 69 insertions(+), 168 deletions(-) create mode 100644 tests/language/README.md rename tests/{reference => }/language/arithmetic/float.bas (100%) rename tests/{reference => }/language/arithmetic/float.txt (100%) rename tests/{reference => }/language/arithmetic/integer.bas (100%) rename tests/{reference => }/language/arithmetic/integer.txt (100%) rename tests/{reference => }/language/array_multidimensional.bas (100%) rename tests/{reference => }/language/array_multidimensional.txt (100%) rename tests/{reference => }/language/array_outofbounds.bas (100%) rename tests/{reference => }/language/array_outofbounds.txt (100%) rename tests/{reference => }/language/arrays.bas (100%) rename tests/{reference => }/language/arrays.txt (100%) rename tests/{reference => }/language/caseinsensitivefunctionsandcommands.bas (100%) rename tests/{reference => }/language/caseinsensitivefunctionsandcommands.txt (100%) rename tests/{reference => language}/examples/bubblesort.bas (100%) rename tests/{reference => language}/examples/bubblesort.txt (100%) rename tests/{reference => language}/examples/factorial.bas (100%) rename tests/{reference => language}/examples/factorial.txt (100%) rename tests/{reference => language}/examples/strreverse.bas (100%) rename tests/{reference => language}/examples/strreverse.txt (100%) rename tests/{reference => }/language/flowcontrol/forloop.bas (100%) rename tests/{reference => }/language/flowcontrol/forloop.txt (100%) rename tests/{reference => }/language/flowcontrol/forloopwaitingforcommand.bas (100%) rename tests/{reference => }/language/flowcontrol/forloopwaitingforcommand.txt (100%) rename tests/{reference => }/language/flowcontrol/goto.bas (100%) rename tests/{reference => }/language/flowcontrol/goto.txt (100%) rename tests/{reference => }/language/flowcontrol/ifthenelse.bas (100%) rename tests/{reference => }/language/flowcontrol/ifthenelse.txt (100%) rename tests/{reference => }/language/flowcontrol/nestedforloops.bas (100%) rename tests/{reference => }/language/flowcontrol/nestedforloops.txt (100%) rename tests/{reference => }/language/flowcontrol/nestedforloopwaitingforcommand.bas (100%) rename tests/{reference => }/language/flowcontrol/nestedforloopwaitingforcommand.txt (100%) rename tests/{reference => }/language/functions.bas (100%) rename tests/{reference => }/language/functions.txt (100%) rename tests/{reference => }/language/functions/atn.bas (100%) rename tests/{reference => }/language/functions/atn.txt (100%) rename tests/{reference => }/language/functions/chr.bas (100%) rename tests/{reference => }/language/functions/chr.txt (100%) rename tests/{reference => }/language/functions/cos.bas (100%) rename tests/{reference => }/language/functions/cos.txt (100%) rename tests/{reference => }/language/functions/hex.bas (100%) rename tests/{reference => }/language/functions/hex.txt (100%) rename tests/{reference => }/language/functions/instr.bas (100%) rename tests/{reference => }/language/functions/instr.txt (100%) rename tests/{reference => }/language/functions/left.bas (100%) rename tests/{reference => }/language/functions/left.txt (100%) rename tests/{reference => }/language/functions/len.bas (100%) rename tests/{reference => }/language/functions/len.txt (100%) rename tests/{reference => }/language/functions/log.bas (100%) rename tests/{reference => }/language/functions/log.txt (100%) rename tests/{reference => }/language/functions/mid.bas (100%) rename tests/{reference => }/language/functions/mid.txt (100%) rename tests/{reference => }/language/functions/mod.bas (100%) rename tests/{reference => }/language/functions/mod.txt (100%) rename tests/{reference => }/language/functions/pointer.bas (100%) rename tests/{reference => }/language/functions/pointer.txt (100%) rename tests/{reference => }/language/functions/rad.bas (100%) rename tests/{reference => }/language/functions/rad.txt (100%) rename tests/{reference => }/language/functions/right.bas (100%) rename tests/{reference => }/language/functions/right.txt (100%) rename tests/{reference => }/language/functions/sgn.bas (100%) rename tests/{reference => }/language/functions/sgn.txt (100%) rename tests/{reference => }/language/functions/shl.bas (100%) rename tests/{reference => }/language/functions/shl.txt (100%) rename tests/{reference => }/language/functions/shr.bas (100%) rename tests/{reference => }/language/functions/shr.txt (100%) rename tests/{reference => }/language/functions/sin.bas (100%) rename tests/{reference => }/language/functions/sin.txt (100%) rename tests/{reference => }/language/functions/spc.bas (100%) rename tests/{reference => }/language/functions/spc.txt (100%) rename tests/{reference => }/language/functions/str.bas (100%) rename tests/{reference => }/language/functions/str.txt (100%) rename tests/{reference => }/language/functions/tan.bas (100%) rename tests/{reference => }/language/functions/tan.txt (100%) rename tests/{reference => }/language/functions/val.bas (100%) rename tests/{reference => }/language/functions/val.txt (100%) rename tests/{reference => }/language/functions/xor.bas (100%) rename tests/{reference => }/language/functions/xor.txt (100%) rename tests/{reference => }/language/functions_multiline.bas (100%) rename tests/{reference => }/language/functions_multiline.txt (100%) rename tests/{reference => }/language/label.bas (100%) rename tests/{reference => }/language/label.txt (100%) rename tests/{reference => }/language/read_data.bas (100%) rename tests/{reference => }/language/read_data.txt (100%) delete mode 100644 tests/reference/README.md diff --git a/.gitea/workflows/ci.yaml b/.gitea/workflows/ci.yaml index 7c71781..9b9a3dc 100644 --- a/.gitea/workflows/ci.yaml +++ b/.gitea/workflows/ci.yaml @@ -17,7 +17,7 @@ jobs: # Not recursive, deliberately. The top-level build needs # deps/libakerror and deps/libakstdlib, via add_subdirectory, and # nothing else: the golden corpus and the Commodore font now live in - # this repository (tests/reference/ and assets/fonts/), so + # this repository (tests/language/ and assets/fonts/), so # deps/basicinterpret is no longer a build dependency at all. # It does *not* need deps/libakgl, which is guarded behind # AKBASIC_WITH_AKGL and defaults OFF -- and recursing into it would @@ -62,11 +62,9 @@ jobs: run: | cmake -S . -B build cmake --build build --parallel 2 - # The suite is 78 cases: 41 golden files byte-compared against the Go - # reference's own corpus (checked in at tests/reference/, see its README), - # 9 local golden cases for verbs the reference never implemented, 25 unit - # tests, 2 embedding examples, and 1 known-failing test that asserts the - # *correct* contract for defects carried over from the reference (TODO.md + # The suite is 112 cases: 65 language files with sibling expectations, + # 43 unit tests, 3 embedding examples, and docs_examples. + # Some unit tests assert the *correct* contract for known defects (TODO.md # section 6). A green run therefore does not mean defect-free -- see # AKBASIC_KNOWN_FAILING_TESTS. # diff --git a/CLAUDE.md b/CLAUDE.md index 7e6eaa1..99a43df 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,9 +57,9 @@ repeating where you will see them: `include/akgl/SDL_GameControllerDB.h`. Change the template or the generator script. - **Do not reformat code you are not otherwise changing.** Several files mix tabs and spaces and there is no repo-wide formatter; style conversions get their own commit. -- **Do not edit `tests/reference/`.** Those expectations came from the Go implementation and - are never edited to suit this interpreter. A deliberate divergence goes in - `tests/reference/README.md`'s divergence table and `docs/13-differences.md`. +- **Keep `tests/language/` editable.** Its `.bas` programs and sibling `.txt` expectations + are changed together when behavior changes. Record deliberate language decisions in + `TODO.md` or `docs/13-differences.md`. - **Open an issue for outstanding work; do not add it to `TODO.md`.** , or `tea issues create --repo andrew/akbasic`. Name the file and line, the functional consequence, and what closing it would diff --git a/CMakeLists.txt b/CMakeLists.txt index 21229cf..7f4a3b6 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -448,7 +448,7 @@ if(AKBASIC_WITH_AKGL) # against. # # **A byte comparison of a rendered PNG is a deliberate bet**, the same bet - # tests/reference/ already makes about golden output: that the dummy video + # the language corpus makes about golden output: that the dummy video # driver and the software renderer are reproducible. They are, run to run and # build to build. What is untested is an SDL upgrade that shifts one pixel of # a diagonal, and the answer to that is to regenerate the figures in the same @@ -531,63 +531,20 @@ if(AKBASIC_WILL_FAIL_TESTS OR AKBASIC_KNOWN_FAILING_TESTS) ) endif() -# The reference's own corpus, byte-compared against the sibling .txt. One CTest -# case per .bas so a failure names the file. +# The editable language corpus. One CTest case per .bas so a failure names the +# file. Each program is paired with a sibling .txt expectation; see +# tests/language/README.md for the editing rule. # -# It used to be driven in place out of deps/basicinterpret, on the reasoning that -# copying a submodule's corpus guarantees drift. That reasoning was sound and it -# has been overruled deliberately: the Go dependency is being deprecated, and a -# build that cannot run its acceptance suite without cloning the implementation -# it replaced is not finished. The copy is byte-identical to -# basicinterpreter@d76162c and tests/reference/README.md records -# where it came from and what the drift now costs. -file(GLOB_RECURSE AKBASIC_GOLDEN_CASES - RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}/tests/reference" - "${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/*.bas" -) - -foreach(_case IN LISTS AKBASIC_GOLDEN_CASES) - string(REGEX REPLACE "^tests/" "" _name "${_case}") - string(REGEX REPLACE "\\.bas$" "" _name "${_name}") - string(REPLACE "/" "_" _name "${_name}") - _add_test( - NAME golden_${_name} - COMMAND ${CMAKE_COMMAND} - -DBASIC=$ - -DCASE=${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/${_case} - -P ${CMAKE_CURRENT_SOURCE_DIR}/tests/golden.cmake - ) -endforeach() - -if(AKBASIC_GOLDEN_CASES) - set(AKBASIC_GOLDEN_NAMES) - foreach(_case IN LISTS AKBASIC_GOLDEN_CASES) - string(REGEX REPLACE "^tests/" "" _name "${_case}") - string(REGEX REPLACE "\\.bas$" "" _name "${_name}") - string(REPLACE "/" "_" _name "${_name}") - list(APPEND AKBASIC_GOLDEN_NAMES golden_${_name}) - endforeach() - _set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES TIMEOUT 30) - # An AKGL build of `basic` opens a window, and forty-one of them is not what - # anybody running the suite wanted. The dummy driver produces the same stdout, - # which is the only thing a golden case compares. - if(AKBASIC_WITH_AKGL) - _set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES - ENVIRONMENT "SDL_VIDEODRIVER=dummy;SDL_AUDIODRIVER=dummy;SDL_RENDER_DRIVER=software") - endif() -endif() - -# The local golden corpus, for verbs the reference never implemented. +# The corpus includes programs carried over from the deprecated Go implementation +# and cases written for this interpreter. Their provenance is useful when +# investigating a regression, but it does not make any case immutable. # -# Still separate now that the reference's corpus lives in this repository too, -# and the reason changed rather than went away: tests/reference/ is a *record* of -# what the Go implementation did and nothing in it should ever be edited to suit -# this one, while tests/language/ is ours to change. Registered under local_ so a -# failure says at a glance which of the two it came from -- and so a diff that -# touches tests/reference/ stands out as the thing it is. +# Programs carried over from the deprecated Go implementation and cases written +# for this interpreter use the same editable `.bas`/`.txt` contract. Registered +# under local_ so every failure identifies the program that produced it. # # Note what this can and cannot cover. The graphics and sound verbs draw and play -# rather than print, so what a golden file sees of them is their *refusals* and +# rather than print, so what a golden file sees of them is their refusals and # whatever a program can PRINT about the state they changed. The behaviour that # reaches a device is asserted against tests/mockdevice.h instead. file(GLOB_RECURSE AKBASIC_LOCAL_CASES diff --git a/MAINTENANCE.md b/MAINTENANCE.md index b632f54..dd55a0e 100644 --- a/MAINTENANCE.md +++ b/MAINTENANCE.md @@ -88,7 +88,7 @@ source stays readable as documentation of what the original did. Neither is bind **It is not a build or test dependency.** Both configurations have been configured, built and run from scratch with it moved out of the tree. Its acceptance corpus is checked in at -`tests/reference/` and its Commodore font at `assets/fonts/`. +`tests/language/` and its Commodore font at `assets/fonts/`. ```sh norun cd deps/basicinterpret @@ -351,15 +351,10 @@ name. That is not cosmetic: `add_executable` creates a dependency's targets even ### The golden corpora -`tests/reference/` is the Go implementation's own acceptance suite, byte-compared. -**Nothing in it is ever edited to suit this interpreter.** If a case fails, either this -interpreter is wrong or the divergence is deliberate — and a deliberate one goes in -`tests/reference/README.md`'s divergence table and `docs/13-differences.md`, not into the -expectation file. `tests/reference/README.md` -says the same thing at more length. - -`tests/language/` is ours and may be changed freely. A new language feature needs a -`.bas`/`.txt` pair there as well as unit tests. +`tests/language/` is the editable language corpus. It includes cases carried over from the +deprecated Go implementation as well as cases written for this interpreter. Every `.bas` file +has a sibling `.txt` expectation, and a new language feature needs that pair as well as unit +tests. Change both deliberately in the same commit; provenance does not make a case immutable. ### Mutation-check a fix before you believe it diff --git a/README.md b/README.md index 6dce36a..47103c1 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ implementation that started from the Java Lox instructions in [craftinginterpreters.com](https://craftinginterpreters.com) and then struck off on its own. That project is deprecated. It is vendored here as the behavioural spec to read when a question about semantics comes up, and its acceptance corpus is checked in at -[`tests/reference/`](tests/reference/README.md) and runs on every build — so nothing about +[`tests/language/`](tests/language/README.md) and runs on every build — so nothing about building or testing this project needs it. ## Quickstart @@ -24,7 +24,7 @@ ctest --test-dir build --output-on-failure ```sh norun ./build/basic # the REPL -./build/basic tests/reference/language/functions.bas # run a program +./build/basic tests/language/functions.bas # run a program ``` ```basic @@ -129,7 +129,7 @@ version are catalogued in [`TODO.md`](TODO.md) and summarised for a BASIC progra | [`docs/`](docs/README.md) | The guide: eighteen chapters, the language then each hardware area then a reference section for every verb and function, [Chapter 14](docs/14-architecture.md) on the interpreter's own architecture, [Chapter 15](docs/15-error-codes.md) listing every error code, and [Chapters 17](docs/17-tutorial-breakout.md) and [18](docs/18-tutorial-breakout-artwork.md) building a whole game twice | | [`MAINTENANCE.md`](MAINTENANCE.md) | For contributors and maintainers: the documentation-example harness, the three test lists, mutation testing, error-code allocation, style | | [`TODO.md`](TODO.md) | Outstanding defects, with file, line and consequence | -| [`tests/reference/README.md`](tests/reference/README.md) | Where the golden corpus came from, and the rule for changing it | +| [`tests/language/README.md`](tests/language/README.md) | The editable language corpus and the rule for changing it | API documentation builds with `doxygen Doxyfile`, into `build/docs/html`. diff --git a/TODO.md b/TODO.md index ed0f753..8b2a855 100644 --- a/TODO.md +++ b/TODO.md @@ -38,7 +38,7 @@ What it changes: - **§1.8's message-text contract is now a convention.** Improving a message is allowed; it costs a golden file, which is a cost rather than a veto. - **§5's bar drops** from "defensible against the golden suite" to defensible on its own merits. -- **`tests/reference/` becomes a regression suite rather than a specification.** Diverging from +- **`tests/language/` is the editable language corpus rather than a protected specification.** Diverging from it is allowed and must be deliberate and recorded — see its README. What it does **not** change: @@ -306,13 +306,13 @@ to `Println`, which adds another. **This used to be a hard contract and is now a default.** The Go implementation is deprecated and will not be updated, so the two projects are no longer required to match — see §0.1. What survives is the practical half: these strings and this newline behaviour are what every -expectation in `tests/reference/` was written against, so changing one means changing golden +expectation in `tests/language/` was written against, so changing one means changing the paired files, and that is worth doing on purpose rather than by accident. A message that reads awkwardly *may* now be improved; do it deliberately, move the expectations in the same commit, and add a line to §5. Numeric formatting still matches the reference: integers via `%" PRId64 "`, floats via `%f` -(Go's `%f` and C's `%f` both give six decimals — `tests/reference/language/arithmetic/float.txt` +(Go's `%f` and C's `%f` both give six decimals — `tests/language/arithmetic/float.txt` confirms). No reason to change it, which is different from not being allowed to. ### 1.9 Which `libakstdlib` calls are cleared for use — **the bans are lifted** @@ -375,7 +375,7 @@ Phases 0 through 6 of the original plan are done. The interpreter builds clean u It *did* reproduce the reference byte for byte, and that claim is retired rather than broken: §0.1 released it, and one case has since diverged deliberately (§6 item 16, listed in -`tests/reference/README.md`). Everything else still matches, which is worth knowing but is no +`tests/language/README.md`). Everything else still matches, which is worth knowing but is no longer a gate. ```sh @@ -408,17 +408,16 @@ and the only path that existed — `AKBASIC_MODE_RUNSTREAM` reading through the is the code the README quotes, built by every build and registered as a CTest case so a signature change breaks the build rather than rotting the document. -**The acceptance suite is the reference's own corpus, checked in at `tests/reference/`.** All -41 `.bas` files are registered as individual CTest cases and byte-compared against their `.txt` +**The acceptance suite is the editable language corpus, checked in at `tests/language/`.** All +65 `.bas` files are registered as individual CTest cases and compared against their `.txt` — including the trailing double newline on an error line (§1.8). It was driven *in place* out of `deps/basicinterpret` until 2026-07-31, on the reasoning that copying a submodule's corpus guarantees drift. That reasoning was sound and was overruled deliberately: the Go dependency is being deprecated, and a build that cannot run its own acceptance suite without cloning the implementation it replaced is not finished. The copy is -byte-identical to `basicinterpreter@d76162c`, and `tests/reference/README.md` records the -provenance, the cost of the drift nobody is watching for now, and the rule that those -expectations are never edited to suit this interpreter. +originally byte-identical to `basicinterpreter@d76162c`, and `tests/language/README.md` records the +provenance and the rule that programs and expectations are edited together deliberately. **Nothing in the build or the suite needs `deps/basicinterpret` any more**, and that is checked rather than assumed — both configurations were configured, built and run from scratch with the @@ -1026,7 +1025,7 @@ deviations from the reference's *program*: `main.go` and the SDL half of **What it costs a program:** a listing that used `INPUT$`, `LEN#` or `GOTO%` as a variable stops parsing, and the fix is to rename the variable. One case in the reference's own corpus - did exactly that; see `tests/reference/README.md`. + did exactly that; see `tests/language/examples/strreverse.bas`. ### Deviations in statement separation @@ -1391,10 +1390,10 @@ deviations from the reference's *program*: `main.go` and the SDL half of **This one moved a golden file.** A line with no number used to be filed under the loader's cursor unchanged — that is, on top of the line before it — so two unnumbered lines in a row silently lost the first, and a *blank* line erased whatever preceded it. The reference does - the same, and `tests/reference/language/arithmetic/integer.bas` is the proof: four `PRINT` + the same, and `tests/language/arithmetic/integer.bas` is the proof: four `PRINT` statements, an expectation with three values, and a trailing blank line that erased `40 PRINT 4 - 2` before the program ran. The expectation is now `4 4 2 2` and - `tests/reference/README.md` records it. + `tests/language/README.md` records it. In its place: `akbasic_runtime_file_line()` (`src/runtime.c`) is the one implementation of the rule, shared by `akbasic_runtime_load()`, RUNSTREAM and `DLOAD`. A numbered line is @@ -1586,9 +1585,9 @@ be reproduced before it can be fixed. **It cost one golden case, exactly as predicted, and the cost turned out to be nothing.** The reference's `examples/strreverse.bas` names a variable `INPUT$`; the variable is renamed - to `SOURCE$` in `tests/reference/`, the expectation is byte-for-byte unchanged — the program + to `SOURCE$` in `tests/language/examples/strreverse.bas`, the expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — and the case keeps every bit of its coverage. Recorded in - `tests/reference/README.md`'s divergence table, which this is the first entry in. + `tests/language/README.md` records the corpus editing rule. This item sat parked because the corpus was the acceptance contract and lived in a submodule this repository could not edit. Both premises are gone: the corpus is checked in, and @@ -2232,10 +2231,10 @@ update --init --recursive` gets them. ## 8. Status -**The port is done.** The C interpreter passes the Go reference's entire corpus and passes clean +**The port is done.** The C interpreter passes the language corpus and passes clean under ASan and UBSan. It reproduced that corpus byte for byte until §0.1 retired the requirement; two cases have diverged on purpose since, and -`tests/reference/README.md` lists them. +`tests/language/README.md` lists them. | Gate | Result | |---|---| @@ -2243,7 +2242,7 @@ requirement; two cases have diverged on purpose since, and | `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 112/112 headless, with `akgl_typing` skipping itself. The same set minus the four `no_device` cases the SDL driver contradicts, plus `akgl_backends`, `akgl_frontend`, `docs_screenshots` and `akgl_typing` — the last of which is the skip, and the `akgl_build` CI job is where it skips | | `docs_examples` | Every fenced block in `README.md`, `MAINTENANCE.md` and `docs/` executed and byte-compared: 71 programs, 9 transcripts, 79 output comparisons, 4 C snippets, 2 excerpts, 2 shell blocks and 19 figures in the default build. The C-snippet count reads 0 when the harness is run by hand without `--cflags-file`; CTest passes it. `MAINTENANCE.md` documents the fence-tag convention | | `docs_screenshots` | 19/19 figures re-rendered and byte-identical to the checked-in PNGs. AKGL build only — rendering a picture needs the SDL half | -| Golden corpus | 41/41 byte-exact from `tests/reference/` — **and 41/41 again through the SDL binary**, which is most of what proves the frontend changes no output | +| Language corpus | 65/65 paired expectations — **and 65/65 again through the SDL binary**, which is most of what proves the frontend changes no output | | ASan + UBSan | 112/112 | | Line coverage | 94.1% (7227/7681) — above the 90% gate | | Function coverage | 97.9% (474/484) | @@ -2553,7 +2552,7 @@ What remains, in priority order: this whole arrangement exists against. The byte comparison is a deliberate bet that the dummy driver and the software renderer - are reproducible, which is the same bet `tests/reference/` already makes about golden + are reproducible, which is the same bet `tests/language/` makes about golden output. Verified run-to-run and build-to-build here; what is untested is an SDL upgrade that moves one pixel of a diagonal. **If that happens, regenerate the figures in the same commit as the bump** — do not weaken the test to a size check, which would pass for every @@ -2636,7 +2635,7 @@ reduced against `build/basic`, the stdio build, unless it says otherwise. **It is narrower than it looks, and the golden corpus is why.** The first attempt released the scope on *any* skip, which broke - `tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas`: a zero- + `tests/language/flowcontrol/nestedforloopwaitingforcommand.bas`: a zero- iteration `FOR` skips its body by the same mechanism, and there the orphan is load-bearing -- it is what absorbs the inner `NEXT` so the outer one still finds its `FOR`. Releasing it turns that case into "NEXT outside the context of FOR". So the diff --git a/docs/14-architecture.md b/docs/14-architecture.md index 721f09b..e1b39a0 100644 --- a/docs/14-architecture.md +++ b/docs/14-architecture.md @@ -743,8 +743,8 @@ way to see that something pushed a scope and never popped it. ```sh norun ctest --test-dir build --output-on-failure -R for_next # one unit test -ctest --test-dir build --output-on-failure -R golden_ # the reference corpus -./build/basic tests/reference/language/functions.bas | diff - tests/reference/language/functions.txt +ctest --test-dir build --output-on-failure -R local_ # the language corpus +./build/basic tests/language/functions.bas | diff - tests/language/functions.txt ./tests/docs_examples.sh --root . --basic ./build/basic \ --cflags-file build/docs_cflags.txt docs/14-architecture.md ``` diff --git a/scripts/mutation_test.py b/scripts/mutation_test.py index 7d49dbb..d63ac10 100755 --- a/scripts/mutation_test.py +++ b/scripts/mutation_test.py @@ -282,9 +282,8 @@ def copy_tree(src, dst): # and a coverage tree drags along every .gcno/.gcda as well. # # deps/ is copied wholesale and has to be: the build pulls libakerror and - # libakstdlib in with add_subdirectory. The golden corpus used to be the - # other reason -- it was driven in place out of deps/basicinterpret -- and - # now lives in tests/reference/, which comes along with the rest of the tree. + # libakstdlib in with add_subdirectory. The language corpus is the other + # reason it comes along with the rest of the tree. # Only vendored Windows DLLs are dead weight, hence "*.dll". # # **PNGs are ours or theirs, and the difference matters.** This used to drop diff --git a/src/runtime.c b/src/runtime.c index 69983ae..27a0a29 100644 --- a/src/runtime.c +++ b/src/runtime.c @@ -893,7 +893,7 @@ akerr_ErrorContext *akbasic_runtime_interpret(akbasic_Runtime *obj, akbasic_ASTL * * **Only a block skip**, which is what the `BEND` test is for. A * zero-iteration `FOR` skips its body the same way, and there the - * orphan is load-bearing: `tests/reference/.../nestedforloopwaiting + * orphan is load-bearing: `tests/language/.../nestedforloopwaiting * forcommand.bas` nests a loop inside one that runs zero times, and * the inner scope is what absorbs the inner `NEXT` so the outer `NEXT` * still finds its `FOR`. Popping it there turns that case into "NEXT @@ -1262,7 +1262,7 @@ akerr_ErrorContext *akbasic_runtime_process_line_runstream(akbasic_Runtime *obj) * This mode used to file it like any other, under the cursor -- which for a * blank line is the number of the line *before* it. A file ending in a blank * line therefore had its last line erased before it ever ran, silently. The - * reference did the same, and `tests/reference/language/arithmetic/integer.bas` + * reference did the same, and `tests/language/arithmetic/integer.bas` * has an expectation with three values for four PRINT statements to prove it. */ if ( buffer[0] == '\0' ) { diff --git a/tests/for_semantics.c b/tests/for_semantics.c index 1d2572c..d5377bf 100644 --- a/tests/for_semantics.c +++ b/tests/for_semantics.c @@ -81,7 +81,7 @@ static void test_counter_survives_the_loop(void) * `FOR I = 1 TO 1` executes the body one time on every BASIC there has ever * been. Here the entry test treats "the counter has reached the limit" as * "do not enter", so the body is skipped entirely -- and - * tests/reference/language/flowcontrol/forloopwaitingforcommand.bas pins that, + * tests/language/flowcontrol/forloopwaitingforcommand.bas pins that, * which is why this cannot simply be corrected. */ static void test_single_iteration_loop(void) diff --git a/tests/language/README.md b/tests/language/README.md new file mode 100644 index 0000000..b677db1 --- /dev/null +++ b/tests/language/README.md @@ -0,0 +1,16 @@ +# The language test corpus + +The `.bas` files in this directory and their sibling `.txt` files are the editable language +corpus. CMake registers every program as an individual `local_*` CTest case and compares its +output with the sibling expectation. + +The corpus includes cases carried over from the deprecated Go implementation as well as cases +written for this interpreter. Their provenance is useful when investigating a regression, but it +does not make any case immutable or turn the old implementation into a specification. + +**Change a program and its expectation deliberately and in the same commit.** If the changed +output represents an intentional language decision, record the reason in `TODO.md` or the +relevant documentation. If it is an accidental change, fix the interpreter instead. + +The `examples/` subdirectory contains complete BASIC programs used as worked examples. It is +part of the same corpus and follows the same `.bas`/`.txt` pairing rule. diff --git a/tests/reference/language/arithmetic/float.bas b/tests/language/arithmetic/float.bas similarity index 100% rename from tests/reference/language/arithmetic/float.bas rename to tests/language/arithmetic/float.bas diff --git a/tests/reference/language/arithmetic/float.txt b/tests/language/arithmetic/float.txt similarity index 100% rename from tests/reference/language/arithmetic/float.txt rename to tests/language/arithmetic/float.txt diff --git a/tests/reference/language/arithmetic/integer.bas b/tests/language/arithmetic/integer.bas similarity index 100% rename from tests/reference/language/arithmetic/integer.bas rename to tests/language/arithmetic/integer.bas diff --git a/tests/reference/language/arithmetic/integer.txt b/tests/language/arithmetic/integer.txt similarity index 100% rename from tests/reference/language/arithmetic/integer.txt rename to tests/language/arithmetic/integer.txt diff --git a/tests/reference/language/array_multidimensional.bas b/tests/language/array_multidimensional.bas similarity index 100% rename from tests/reference/language/array_multidimensional.bas rename to tests/language/array_multidimensional.bas diff --git a/tests/reference/language/array_multidimensional.txt b/tests/language/array_multidimensional.txt similarity index 100% rename from tests/reference/language/array_multidimensional.txt rename to tests/language/array_multidimensional.txt diff --git a/tests/reference/language/array_outofbounds.bas b/tests/language/array_outofbounds.bas similarity index 100% rename from tests/reference/language/array_outofbounds.bas rename to tests/language/array_outofbounds.bas diff --git a/tests/reference/language/array_outofbounds.txt b/tests/language/array_outofbounds.txt similarity index 100% rename from tests/reference/language/array_outofbounds.txt rename to tests/language/array_outofbounds.txt diff --git a/tests/reference/language/arrays.bas b/tests/language/arrays.bas similarity index 100% rename from tests/reference/language/arrays.bas rename to tests/language/arrays.bas diff --git a/tests/reference/language/arrays.txt b/tests/language/arrays.txt similarity index 100% rename from tests/reference/language/arrays.txt rename to tests/language/arrays.txt diff --git a/tests/reference/language/caseinsensitivefunctionsandcommands.bas b/tests/language/caseinsensitivefunctionsandcommands.bas similarity index 100% rename from tests/reference/language/caseinsensitivefunctionsandcommands.bas rename to tests/language/caseinsensitivefunctionsandcommands.bas diff --git a/tests/reference/language/caseinsensitivefunctionsandcommands.txt b/tests/language/caseinsensitivefunctionsandcommands.txt similarity index 100% rename from tests/reference/language/caseinsensitivefunctionsandcommands.txt rename to tests/language/caseinsensitivefunctionsandcommands.txt diff --git a/tests/reference/examples/bubblesort.bas b/tests/language/examples/bubblesort.bas similarity index 100% rename from tests/reference/examples/bubblesort.bas rename to tests/language/examples/bubblesort.bas diff --git a/tests/reference/examples/bubblesort.txt b/tests/language/examples/bubblesort.txt similarity index 100% rename from tests/reference/examples/bubblesort.txt rename to tests/language/examples/bubblesort.txt diff --git a/tests/reference/examples/factorial.bas b/tests/language/examples/factorial.bas similarity index 100% rename from tests/reference/examples/factorial.bas rename to tests/language/examples/factorial.bas diff --git a/tests/reference/examples/factorial.txt b/tests/language/examples/factorial.txt similarity index 100% rename from tests/reference/examples/factorial.txt rename to tests/language/examples/factorial.txt diff --git a/tests/reference/examples/strreverse.bas b/tests/language/examples/strreverse.bas similarity index 100% rename from tests/reference/examples/strreverse.bas rename to tests/language/examples/strreverse.bas diff --git a/tests/reference/examples/strreverse.txt b/tests/language/examples/strreverse.txt similarity index 100% rename from tests/reference/examples/strreverse.txt rename to tests/language/examples/strreverse.txt diff --git a/tests/reference/language/flowcontrol/forloop.bas b/tests/language/flowcontrol/forloop.bas similarity index 100% rename from tests/reference/language/flowcontrol/forloop.bas rename to tests/language/flowcontrol/forloop.bas diff --git a/tests/reference/language/flowcontrol/forloop.txt b/tests/language/flowcontrol/forloop.txt similarity index 100% rename from tests/reference/language/flowcontrol/forloop.txt rename to tests/language/flowcontrol/forloop.txt diff --git a/tests/reference/language/flowcontrol/forloopwaitingforcommand.bas b/tests/language/flowcontrol/forloopwaitingforcommand.bas similarity index 100% rename from tests/reference/language/flowcontrol/forloopwaitingforcommand.bas rename to tests/language/flowcontrol/forloopwaitingforcommand.bas diff --git a/tests/reference/language/flowcontrol/forloopwaitingforcommand.txt b/tests/language/flowcontrol/forloopwaitingforcommand.txt similarity index 100% rename from tests/reference/language/flowcontrol/forloopwaitingforcommand.txt rename to tests/language/flowcontrol/forloopwaitingforcommand.txt diff --git a/tests/reference/language/flowcontrol/goto.bas b/tests/language/flowcontrol/goto.bas similarity index 100% rename from tests/reference/language/flowcontrol/goto.bas rename to tests/language/flowcontrol/goto.bas diff --git a/tests/reference/language/flowcontrol/goto.txt b/tests/language/flowcontrol/goto.txt similarity index 100% rename from tests/reference/language/flowcontrol/goto.txt rename to tests/language/flowcontrol/goto.txt diff --git a/tests/reference/language/flowcontrol/ifthenelse.bas b/tests/language/flowcontrol/ifthenelse.bas similarity index 100% rename from tests/reference/language/flowcontrol/ifthenelse.bas rename to tests/language/flowcontrol/ifthenelse.bas diff --git a/tests/reference/language/flowcontrol/ifthenelse.txt b/tests/language/flowcontrol/ifthenelse.txt similarity index 100% rename from tests/reference/language/flowcontrol/ifthenelse.txt rename to tests/language/flowcontrol/ifthenelse.txt diff --git a/tests/reference/language/flowcontrol/nestedforloops.bas b/tests/language/flowcontrol/nestedforloops.bas similarity index 100% rename from tests/reference/language/flowcontrol/nestedforloops.bas rename to tests/language/flowcontrol/nestedforloops.bas diff --git a/tests/reference/language/flowcontrol/nestedforloops.txt b/tests/language/flowcontrol/nestedforloops.txt similarity index 100% rename from tests/reference/language/flowcontrol/nestedforloops.txt rename to tests/language/flowcontrol/nestedforloops.txt diff --git a/tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas b/tests/language/flowcontrol/nestedforloopwaitingforcommand.bas similarity index 100% rename from tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas rename to tests/language/flowcontrol/nestedforloopwaitingforcommand.bas diff --git a/tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.txt b/tests/language/flowcontrol/nestedforloopwaitingforcommand.txt similarity index 100% rename from tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.txt rename to tests/language/flowcontrol/nestedforloopwaitingforcommand.txt diff --git a/tests/reference/language/functions.bas b/tests/language/functions.bas similarity index 100% rename from tests/reference/language/functions.bas rename to tests/language/functions.bas diff --git a/tests/reference/language/functions.txt b/tests/language/functions.txt similarity index 100% rename from tests/reference/language/functions.txt rename to tests/language/functions.txt diff --git a/tests/reference/language/functions/atn.bas b/tests/language/functions/atn.bas similarity index 100% rename from tests/reference/language/functions/atn.bas rename to tests/language/functions/atn.bas diff --git a/tests/reference/language/functions/atn.txt b/tests/language/functions/atn.txt similarity index 100% rename from tests/reference/language/functions/atn.txt rename to tests/language/functions/atn.txt diff --git a/tests/reference/language/functions/chr.bas b/tests/language/functions/chr.bas similarity index 100% rename from tests/reference/language/functions/chr.bas rename to tests/language/functions/chr.bas diff --git a/tests/reference/language/functions/chr.txt b/tests/language/functions/chr.txt similarity index 100% rename from tests/reference/language/functions/chr.txt rename to tests/language/functions/chr.txt diff --git a/tests/reference/language/functions/cos.bas b/tests/language/functions/cos.bas similarity index 100% rename from tests/reference/language/functions/cos.bas rename to tests/language/functions/cos.bas diff --git a/tests/reference/language/functions/cos.txt b/tests/language/functions/cos.txt similarity index 100% rename from tests/reference/language/functions/cos.txt rename to tests/language/functions/cos.txt diff --git a/tests/reference/language/functions/hex.bas b/tests/language/functions/hex.bas similarity index 100% rename from tests/reference/language/functions/hex.bas rename to tests/language/functions/hex.bas diff --git a/tests/reference/language/functions/hex.txt b/tests/language/functions/hex.txt similarity index 100% rename from tests/reference/language/functions/hex.txt rename to tests/language/functions/hex.txt diff --git a/tests/reference/language/functions/instr.bas b/tests/language/functions/instr.bas similarity index 100% rename from tests/reference/language/functions/instr.bas rename to tests/language/functions/instr.bas diff --git a/tests/reference/language/functions/instr.txt b/tests/language/functions/instr.txt similarity index 100% rename from tests/reference/language/functions/instr.txt rename to tests/language/functions/instr.txt diff --git a/tests/reference/language/functions/left.bas b/tests/language/functions/left.bas similarity index 100% rename from tests/reference/language/functions/left.bas rename to tests/language/functions/left.bas diff --git a/tests/reference/language/functions/left.txt b/tests/language/functions/left.txt similarity index 100% rename from tests/reference/language/functions/left.txt rename to tests/language/functions/left.txt diff --git a/tests/reference/language/functions/len.bas b/tests/language/functions/len.bas similarity index 100% rename from tests/reference/language/functions/len.bas rename to tests/language/functions/len.bas diff --git a/tests/reference/language/functions/len.txt b/tests/language/functions/len.txt similarity index 100% rename from tests/reference/language/functions/len.txt rename to tests/language/functions/len.txt diff --git a/tests/reference/language/functions/log.bas b/tests/language/functions/log.bas similarity index 100% rename from tests/reference/language/functions/log.bas rename to tests/language/functions/log.bas diff --git a/tests/reference/language/functions/log.txt b/tests/language/functions/log.txt similarity index 100% rename from tests/reference/language/functions/log.txt rename to tests/language/functions/log.txt diff --git a/tests/reference/language/functions/mid.bas b/tests/language/functions/mid.bas similarity index 100% rename from tests/reference/language/functions/mid.bas rename to tests/language/functions/mid.bas diff --git a/tests/reference/language/functions/mid.txt b/tests/language/functions/mid.txt similarity index 100% rename from tests/reference/language/functions/mid.txt rename to tests/language/functions/mid.txt diff --git a/tests/reference/language/functions/mod.bas b/tests/language/functions/mod.bas similarity index 100% rename from tests/reference/language/functions/mod.bas rename to tests/language/functions/mod.bas diff --git a/tests/reference/language/functions/mod.txt b/tests/language/functions/mod.txt similarity index 100% rename from tests/reference/language/functions/mod.txt rename to tests/language/functions/mod.txt diff --git a/tests/reference/language/functions/pointer.bas b/tests/language/functions/pointer.bas similarity index 100% rename from tests/reference/language/functions/pointer.bas rename to tests/language/functions/pointer.bas diff --git a/tests/reference/language/functions/pointer.txt b/tests/language/functions/pointer.txt similarity index 100% rename from tests/reference/language/functions/pointer.txt rename to tests/language/functions/pointer.txt diff --git a/tests/reference/language/functions/rad.bas b/tests/language/functions/rad.bas similarity index 100% rename from tests/reference/language/functions/rad.bas rename to tests/language/functions/rad.bas diff --git a/tests/reference/language/functions/rad.txt b/tests/language/functions/rad.txt similarity index 100% rename from tests/reference/language/functions/rad.txt rename to tests/language/functions/rad.txt diff --git a/tests/reference/language/functions/right.bas b/tests/language/functions/right.bas similarity index 100% rename from tests/reference/language/functions/right.bas rename to tests/language/functions/right.bas diff --git a/tests/reference/language/functions/right.txt b/tests/language/functions/right.txt similarity index 100% rename from tests/reference/language/functions/right.txt rename to tests/language/functions/right.txt diff --git a/tests/reference/language/functions/sgn.bas b/tests/language/functions/sgn.bas similarity index 100% rename from tests/reference/language/functions/sgn.bas rename to tests/language/functions/sgn.bas diff --git a/tests/reference/language/functions/sgn.txt b/tests/language/functions/sgn.txt similarity index 100% rename from tests/reference/language/functions/sgn.txt rename to tests/language/functions/sgn.txt diff --git a/tests/reference/language/functions/shl.bas b/tests/language/functions/shl.bas similarity index 100% rename from tests/reference/language/functions/shl.bas rename to tests/language/functions/shl.bas diff --git a/tests/reference/language/functions/shl.txt b/tests/language/functions/shl.txt similarity index 100% rename from tests/reference/language/functions/shl.txt rename to tests/language/functions/shl.txt diff --git a/tests/reference/language/functions/shr.bas b/tests/language/functions/shr.bas similarity index 100% rename from tests/reference/language/functions/shr.bas rename to tests/language/functions/shr.bas diff --git a/tests/reference/language/functions/shr.txt b/tests/language/functions/shr.txt similarity index 100% rename from tests/reference/language/functions/shr.txt rename to tests/language/functions/shr.txt diff --git a/tests/reference/language/functions/sin.bas b/tests/language/functions/sin.bas similarity index 100% rename from tests/reference/language/functions/sin.bas rename to tests/language/functions/sin.bas diff --git a/tests/reference/language/functions/sin.txt b/tests/language/functions/sin.txt similarity index 100% rename from tests/reference/language/functions/sin.txt rename to tests/language/functions/sin.txt diff --git a/tests/reference/language/functions/spc.bas b/tests/language/functions/spc.bas similarity index 100% rename from tests/reference/language/functions/spc.bas rename to tests/language/functions/spc.bas diff --git a/tests/reference/language/functions/spc.txt b/tests/language/functions/spc.txt similarity index 100% rename from tests/reference/language/functions/spc.txt rename to tests/language/functions/spc.txt diff --git a/tests/reference/language/functions/str.bas b/tests/language/functions/str.bas similarity index 100% rename from tests/reference/language/functions/str.bas rename to tests/language/functions/str.bas diff --git a/tests/reference/language/functions/str.txt b/tests/language/functions/str.txt similarity index 100% rename from tests/reference/language/functions/str.txt rename to tests/language/functions/str.txt diff --git a/tests/reference/language/functions/tan.bas b/tests/language/functions/tan.bas similarity index 100% rename from tests/reference/language/functions/tan.bas rename to tests/language/functions/tan.bas diff --git a/tests/reference/language/functions/tan.txt b/tests/language/functions/tan.txt similarity index 100% rename from tests/reference/language/functions/tan.txt rename to tests/language/functions/tan.txt diff --git a/tests/reference/language/functions/val.bas b/tests/language/functions/val.bas similarity index 100% rename from tests/reference/language/functions/val.bas rename to tests/language/functions/val.bas diff --git a/tests/reference/language/functions/val.txt b/tests/language/functions/val.txt similarity index 100% rename from tests/reference/language/functions/val.txt rename to tests/language/functions/val.txt diff --git a/tests/reference/language/functions/xor.bas b/tests/language/functions/xor.bas similarity index 100% rename from tests/reference/language/functions/xor.bas rename to tests/language/functions/xor.bas diff --git a/tests/reference/language/functions/xor.txt b/tests/language/functions/xor.txt similarity index 100% rename from tests/reference/language/functions/xor.txt rename to tests/language/functions/xor.txt diff --git a/tests/reference/language/functions_multiline.bas b/tests/language/functions_multiline.bas similarity index 100% rename from tests/reference/language/functions_multiline.bas rename to tests/language/functions_multiline.bas diff --git a/tests/reference/language/functions_multiline.txt b/tests/language/functions_multiline.txt similarity index 100% rename from tests/reference/language/functions_multiline.txt rename to tests/language/functions_multiline.txt diff --git a/tests/reference/language/label.bas b/tests/language/label.bas similarity index 100% rename from tests/reference/language/label.bas rename to tests/language/label.bas diff --git a/tests/reference/language/label.txt b/tests/language/label.txt similarity index 100% rename from tests/reference/language/label.txt rename to tests/language/label.txt diff --git a/tests/reference/language/read_data.bas b/tests/language/read_data.bas similarity index 100% rename from tests/reference/language/read_data.bas rename to tests/language/read_data.bas diff --git a/tests/reference/language/read_data.txt b/tests/language/read_data.txt similarity index 100% rename from tests/reference/language/read_data.txt rename to tests/language/read_data.txt diff --git a/tests/reference/README.md b/tests/reference/README.md deleted file mode 100644 index 1b050c6..0000000 --- a/tests/reference/README.md +++ /dev/null @@ -1,63 +0,0 @@ -# The reference interpreter's own test corpus - -These 41 `.bas` files and their 41 `.txt` expectations are **not this project's tests.** They -are a byte-for-byte copy of the acceptance corpus belonging to the Go implementation this -interpreter was ported from: - -| | | -|---|---| -| Source | `https://source.starfort.tech/andrew/basicinterpreter`, `tests/` | -| Copied at | commit `d76162cb37eeaccddae2cf6cab7af7bc8175d6d2` ("Update README.md") | -| Copied on | 2026-07-31 | -| Verified | every file `cmp`-identical to the submodule at the time of the copy | - -They are registered as the `golden_*` CTest cases and run against `basic` on every build, in -both the default and the `-DAKBASIC_WITH_AKGL=ON` configurations. - -## Why they are a copy, when they deliberately were not - -Until 2026-07-31 these were driven *in place* out of `deps/basicinterpret`, and the reasoning -was written down in `CMakeLists.txt`: the corpus is a submodule, and copying a submodule's -corpus guarantees drift. - -That reasoning was sound and it stopped applying. The Go project is deprecated and will not be -updated, so there is nothing left to drift *from* — and a build that cannot run its own -acceptance suite without cloning the implementation it replaced is not finished. - -## What these are for now - -**A regression suite, not a specification.** That distinction is the whole of how to treat -them, and it changed on 2026-07-31 (`TODO.md` §0.1): matching the Go implementation is no -longer a goal, so a difference between it and this interpreter is not automatically a bug in -this one. - -What they are still good for is unchanged and is worth a lot: forty-one real BASIC programs -with known-good output, covering arithmetic, arrays, flow control, every builtin function, -`READ`/`DATA`, labels and three worked examples. That catches regressions whatever its -provenance. - -## The rule for editing these - -**Diverge deliberately, never by accident.** - -A failure here means this interpreter's behaviour changed. Nine times in ten that is a bug you -just introduced and the answer is to fix the code. The tenth time it is an improvement on the -reference, and then the answer is: - -1. Change the expectation in the same commit as the code. -2. Say why in the commit message. -3. Add a line to `TODO.md` §5, which is the list of deliberate deviations. - -What is not acceptable is editing a `.txt` to make a red suite green without deciding which of -the two you are doing. - -New cases for this project's own behaviour go in `tests/language/`, registered under `local_*`. -The prefix says which corpus a failure came from, and a diff touching `tests/reference/` stands -out as the thing it is. - -## Divergences so far - -| Case | Change | Why | -|---|---|---| -| `language/arithmetic/integer.txt` | Expectation grew from `4 4 2` to `4 4 2 2` | The program has **four** `PRINT` statements and the expectation had three values. The file ends in a blank line, and both interpreters filed a blank line under the loader's cursor — which for a line with no number of its own is the number of the line before it. So `40 PRINT 4 - 2` was erased before the program ran. The reference lost it the same way, which is why the expectation was written short. Fixed by skipping blank lines in RUNSTREAM, as `akbasic_runtime_load()` and `DLOAD` already did. `TODO.md` §5. | -| `examples/strreverse.bas` | Variable `INPUT$` renamed to `SOURCE$` | `INPUT$` is a reserved word with a type suffix, which a real C128 refuses and which this interpreter now refuses too (`TODO.md` §6 item 16). The reference accepted it only because its own reserved-word check never fired. The expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — so the case keeps all of its value. | diff --git a/tests/structure_verbs.c b/tests/structure_verbs.c index 9508131..e0fb49b 100644 --- a/tests/structure_verbs.c +++ b/tests/structure_verbs.c @@ -358,7 +358,7 @@ static void test_skipped_block_releases_its_loop_scope(void) * inner `NEXT`, so the outer `NEXT` still finds its own `FOR`. Releasing it * turns this into "NEXT outside the context of FOR". * - * tests/reference/language/flowcontrol/nestedforloopwaitingforcommand.bas is the + * tests/language/flowcontrol/nestedforloopwaitingforcommand.bas is the * golden case that caught exactly that, and this is the same shape asserted * where a reader of this file will see it. */ diff --git a/tests/unnumbered.c b/tests/unnumbered.c index 37e5b9f..e122cd9 100644 --- a/tests/unnumbered.c +++ b/tests/unnumbered.c @@ -182,7 +182,7 @@ static void test_runstream_assigns_the_same_slots(void) /** * @brief A file ending in a blank line keeps its last line. * - * `tests/reference/language/arithmetic/integer.bas` is this shape, and both this + * `tests/language/arithmetic/integer.bas` is this shape, and both this * interpreter and the reference used to erase the last line before running it -- * the blank line was filed under the cursor, which for a line with no number of * its own is the number of the line before it.