andrew/akbasic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Retire the byte-for-byte fidelity constraint, and fix what it was blocking

Browse Source 
The Go reference is deprecated and will not be updated, so the two projects
are no longer required to match. Recorded as TODO.md section 0.1, first thing
in the file, because it silently reverses the premise several later sections
were written on -- an agent reading section 6 without it will park work that
is no longer blocked.

Section 6 item 16 was the item waiting on exactly this and is now fixed: the
keyword tables are searched on the base name with any type suffix stripped, so
PRINT$, LEN# and GOTO% are refused as variable names and the reference's own
diagnostic stops being dead code. It cost the one golden case predicted, and
the cost was nothing -- renaming a variable in strreverse.bas left its expected
output byte-for-byte unchanged.

AKBASIC_KNOWN_FAILING_TESTS is empty as a result and known_reference_defects.c
is gone. Section 6 items 1-9 and 11 are reopened as an ordinary defect list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Kesterson
2026-07-31 12:52:00 -04:00
parent eb5374d212
commit 583c0abbd2
9 changed files with 205 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
tests/known_reference_defects.c
View File

@@ -1,46 +0,0 @@
/**
* @file known_reference_defects.c
* @brief Asserts the *correct* contract for defects carried over from the reference.
*
* Registered in AKBASIC_KNOWN_FAILING_TESTS, so CTest expects it to fail. Every
* assertion here states what the interpreter *should* do, not what it currently
* does -- pinning current-but-wrong behaviour would turn the eventual fix into a
* test failure.
*
* When one of these is fixed, this test starts passing and CTest reports
* "unexpectedly passed". That is the cue to split the fixed assertion out into
* the corresponding suite in AKBASIC_TESTS and strike the item from TODO.md
* section 6.
*
* Five of the six this file was written with are gone that way. What is left is
* the one whose fix is not a coding question but a decision, and it is described
* where it is asserted.
*/
#include "harness.h"
int main(void)
{
TEST_REQUIRE_OK(harness_start(NULL));
/*
* 6.16 -- "Reserved word in variable name" never fires, because the lexeme
* still carries its type suffix when the verb table is searched, so PRINT$
* misses every table and becomes an ordinary string variable.
*
* Still here rather than fixed, and the reason is worth knowing before
* anybody "just fixes it": stripping the suffix before the lookup is four
* lines and it works, and it costs a golden case. The reference's own
* examples/strreverse.bas names a variable INPUT$ -- which a real C128 would
* refuse too, so the reference program is the wrong one. But the corpus is
* the acceptance contract and it lives in a submodule this repository may
* not edit, so the diagnostic and the corpus cannot both be right. TODO.md
* section 6 records the trade rather than making it silently.
*/
HARNESS_RUNTIME.hasError = false;
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT$ = 1", NULL, 0));
TEST_REQUIRE(HARNESS_RUNTIME.hasError, "PRINT$ should be refused as a variable name");
harness_stop();
return akbasic_test_failures;
}

55
tests/reference/README.md
View File

@@ -14,36 +14,49 @@ interpreter was ported from:
They are registered as the `golden_*` CTest cases and run against `basic` on every build, in
both the default and the `-DAKBASIC_WITH_AKGL=ON` configurations.
## Why they are a copy now, when they deliberately were not
## Why they are a copy, when they deliberately were not
Until 2026-07-31 these were driven *in place* out of `deps/basicinterpret`, and the reasoning
was written down in `CMakeLists.txt`: the corpus is a submodule, and copying a submodule's
corpus guarantees drift.
That reasoning was sound. It was overruled on purpose, because the Go dependency is being
deprecated and a build that cannot run its own acceptance suite without cloning the
implementation it replaced is not finished.
That reasoning was sound and it stopped applying. The Go project is deprecated and will not be
updated, so there is nothing left to drift *from* — and a build that cannot run its own
acceptance suite without cloning the implementation it replaced is not finished.
**What the change costs, stated plainly so nobody has to rediscover it:** if the Go
implementation ever gains a test, this directory will not. There is no longer anything
watching for that. The trade is deliberate — the Go project is the thing being retired, so a
corpus that stops tracking it is a corpus that stopped tracking something that stopped moving.
## What these are for now
**A regression suite, not a specification.** That distinction is the whole of how to treat
them, and it changed on 2026-07-31 (`TODO.md` §0.1): matching the Go implementation is no
longer a goal, so a difference between it and this interpreter is not automatically a bug in
this one.
What they are still good for is unchanged and is worth a lot: forty-one real BASIC programs
with known-good output, covering arithmetic, arrays, flow control, every builtin function,
`READ`/`DATA`, labels and three worked examples. That catches regressions whatever its
provenance.
## The rule for editing these
**Don't.**
**Diverge deliberately, never by accident.**
This directory is a *record of what the Go implementation did*, and it is what makes the claim
"reproduces the reference byte for byte" checkable at all. A failure here means this
interpreter's behaviour changed, and the answer is to change this interpreter or to record a
deliberate deviation in `TODO.md` §5 — never to edit the expectation to match.
A failure here means this interpreter's behaviour changed. Nine times in ten that is a bug you
just introduced and the answer is to fix the code. The tenth time it is an improvement on the
reference, and then the answer is:
New cases for this project's own behaviour go in `tests/language/`, which is registered under
`local_*` for exactly this reason: the prefix says which corpus a failure came from, and a diff
that touches `tests/reference/` stands out as the thing it is.
1. Change the expectation in the same commit as the code.
2. Say why in the commit message.
3. Add a line to `TODO.md` §5, which is the list of deliberate deviations.
There is one case worth knowing about before anybody "fixes" it.
`examples/strreverse.bas` names a variable `INPUT$`, which a real Commodore 128 would refuse
and which this interpreter accepts only because the reference's reserved-word check never
fires. That is `TODO.md` §6 item 16, and it is the reason that item is filed as a decision
rather than a defect: making the check work costs this case.
What is not acceptable is editing a `.txt` to make a red suite green without deciding which of
the two you are doing.
New cases for this project's own behaviour go in `tests/language/`, registered under `local_*`.
The prefix says which corpus a failure came from, and a diff touching `tests/reference/` stands
out as the thing it is.
## Divergences so far
| Case | Change | Why |
|---|---|---|
| `examples/strreverse.bas` | Variable `INPUT$` renamed to `SOURCE$` | `INPUT$` is a reserved word with a type suffix, which a real C128 refuses and which this interpreter now refuses too (`TODO.md` §6 item 16). The reference accepted it only because its own reserved-word check never fired. The expectation is byte-for-byte unchanged — the program still prints `REVERSED: OLLEH` — so the case keeps all of its value. |

6
tests/reference/examples/strreverse.bas
View File

@@ -1,7 +1,7 @@
10 INPUT$ = "HELLO"
20 STRLEN# = LEN(INPUT$) - 1
10 SOURCE$ = "HELLO"
20 STRLEN# = LEN(SOURCE$) - 1
30 REV$ = ""
40 FOR I# = STRLEN# TO 0 STEP -1
50 REV$ = REV$ + MID(INPUT$, I#, 1)
50 REV$ = REV$ + MID(SOURCE$, I#, 1)
60 NEXT I#
70 PRINT "REVERSED: " + REV$

31
tests/scanner_tokens.c
View File

@@ -150,15 +150,32 @@ int main(void)
TEST_REQUIRE_INT(env()->lineno, 10);
/*
* The "Reserved word in variable name" check never fires: the lexeme still
* carries its suffix when the verb table is searched, so "PRINT$" misses and
* becomes an ordinary string variable. Pinned as it stands -- TODO.md
* section 12 item 16.
* A verb name with a type suffix stuck on it is refused as a variable name.
* The reference searches the keyword tables with the suffix still attached
* (basicscanner.go:349), so "PRINT$" misses every table, its own "Reserved
* word in variable name" diagnostic is dead code, and `PRINT$ = 1` is
* quietly accepted. TODO.md section 6 item 16; this is its regression test.
*/
HARNESS_RUNTIME.hasError = false;
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT$ = 1", NULL, 0));
TEST_REQUIRE(!HARNESS_RUNTIME.hasError,
"PRINT$ is currently accepted as a variable name; see TODO.md 12.16");
expect_token(0, AKBASIC_TOK_IDENTIFIER_STRING, "PRINT$");
TEST_REQUIRE(HARNESS_RUNTIME.hasError, "PRINT$ must be refused as a variable name");
/* The same for a function name, and for the other two suffixes. */
HARNESS_RUNTIME.hasError = false;
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "LEN# = 1", NULL, 0));
TEST_REQUIRE(HARNESS_RUNTIME.hasError, "LEN# must be refused as a variable name");
HARNESS_RUNTIME.hasError = false;
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "GOTO% = 1", NULL, 0));
TEST_REQUIRE(HARNESS_RUNTIME.hasError, "GOTO%% must be refused as a variable name");
/*
* And an ordinary name that merely *contains* a verb is still fine, which is
* what says the check compares the whole base name rather than a prefix.
*/
HARNESS_RUNTIME.hasError = false;
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINTER$ = 1", NULL, 0));
TEST_REQUIRE(!HARNESS_RUNTIME.hasError, "PRINTER$ is not a reserved word");
expect_token(0, AKBASIC_TOK_IDENTIFIER_STRING, "PRINTER$");
/* An unknown character is diagnosed. */
HARNESS_RUNTIME.hasError = false;