Andrew Kesterson
583c0abbd2
The Go reference is deprecated and will not be updated, so the two projects are no longer required to match. Recorded as TODO.md section 0.1, first thing in the file, because it silently reverses the premise several later sections were written on -- an agent reading section 6 without it will park work that is no longer blocked. Section 6 item 16 was the item waiting on exactly this and is now fixed: the keyword tables are searched on the base name with any type suffix stripped, so PRINT$, LEN# and GOTO% are refused as variable names and the reference's own diagnostic stops being dead code. It cost the one golden case predicted, and the cost was nothing -- renaming a variable in strreverse.bas left its expected output byte-for-byte unchanged. AKBASIC_KNOWN_FAILING_TESTS is empty as a result and known_reference_defects.c is gone. Section 6 items 1-9 and 11 are reopened as an ordinary defect list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
63 lines
3.1 KiB
Markdown
63 lines
3.1 KiB
Markdown
# The reference interpreter's own test corpus
|
|
|
|
These 41 `.bas` files and their 41 `.txt` expectations are **not this project's tests.** They
|
|
are a byte-for-byte copy of the acceptance corpus belonging to the Go implementation this
|
|
interpreter was ported from:
|
|
|
|
| | |
|
|
|---|---|
|
|
| Source | `https://source.starfort.tech/andrew/basicinterpreter`, `tests/` |
|
|
| Copied at | commit `d76162cb37eeaccddae2cf6cab7af7bc8175d6d2` ("Update README.md") |
|
|
| Copied on | 2026-07-31 |
|
|
| Verified | every file `cmp`-identical to the submodule at the time of the copy |
|
|
|
|
They are registered as the `golden_*` CTest cases and run against `basic` on every build, in
|
|
both the default and the `-DAKBASIC_WITH_AKGL=ON` configurations.
|
|
|
|
## Why they are a copy, when they deliberately were not
|
|
|
|
Until 2026-07-31 these were driven *in place* out of `deps/basicinterpret`, and the reasoning
|
|
was written down in `CMakeLists.txt`: the corpus is a submodule, and copying a submodule's
|
|
corpus guarantees drift.
|
|
|
|
That reasoning was sound and it stopped applying. The Go project is deprecated and will not be
|
|
updated, so there is nothing left to drift *from* — and a build that cannot run its own
|
|
acceptance suite without cloning the implementation it replaced is not finished.
|
|
|
|
## What these are for now
|
|
|
|
**A regression suite, not a specification.** That distinction is the whole of how to treat
|
|
them, and it changed on 2026-07-31 (`TODO.md` §0.1): matching the Go implementation is no
|
|
longer a goal, so a difference between it and this interpreter is not automatically a bug in
|
|
this one.
|
|
|
|
What they are still good for is unchanged and is worth a lot: forty-one real BASIC programs
|
|
with known-good output, covering arithmetic, arrays, flow control, every builtin function,
|
|
`READ`/`DATA`, labels and three worked examples. That catches regressions whatever its
|
|
provenance.
|
|
|
|
## The rule for editing these
|
|
|
|
**Diverge deliberately, never by accident.**
|
|
|
|
A failure here means this interpreter's behaviour changed. Nine times in ten that is a bug you
|
|
just introduced and the answer is to fix the code. The tenth time it is an improvement on the
|
|
reference, and then the answer is:
|
|
|
|
1. Change the expectation in the same commit as the code.
|
|
2. Say why in the commit message.
|
|
3. Add a line to `TODO.md` §5, which is the list of deliberate deviations.
|
|
|
|
What is not acceptable is editing a `.txt` to make a red suite green without deciding which of
|
|
the two you are doing.
|
|
|
|
New cases for this project's own behaviour go in `tests/language/`, registered under `local_*`.
|
|
The prefix says which corpus a failure came from, and a diff touching `tests/reference/` stands
|
|
out as the thing it is.
|
|
|
|
## Divergences so far
|
|
|
|
| Case | Change | Why |
|
|
|---|---|---|
|
|
| `examples/strreverse.bas` | Variable `INPUT$` renamed to `SOURCE$` | `INPUT$` is a reserved word with a type suffix, which a real C128 refuses and which this interpreter now refuses too (`TODO.md` §6 item 16). The reference accepted it only because its own reserved-word check never fired. The expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — so the case keeps all of its value. |
|