Consume the COLON token: a line can hold several statements

The token has existed since the scanner was written and nothing read it, so
10 PRINT A$ : REM ... was a parse error. Leading separators are consumed
before each statement and an empty statement yields a NULL leaf rather than
an error, so a trailing colon and a run of them are both legal.

BASIC 7.0 scopes everything after THEN to the condition, which the reference
had no opinion about because it never got here. The rule is not "skip when
false": the rest of the line belongs to whichever arm was written last.

A whole FOR/NEXT on one line still does not loop -- block skipping works by
source line. Recorded in TODO.md as what group A has to fix first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 11:46:10 -04:00
parent 0f3d8a0ac6
commit a31058cf37
8 changed files with 256 additions and 18 deletions

74
TODO.md
View File

@@ -565,11 +565,30 @@ control), `MONITOR` (no machine-language monitor).
Also on the queue, from the reference's own defect list:
- **Multiple statements per line** (`10 PRINT A$ : REM ...`). The `COLON` token exists
(`basicscanner.go:40`) and nothing consumes it. This changes the parser's statement loop and
should be done before group A, because `DO`/`LOOP` bodies read badly without it.
- **Array references in parameter lists** (`READ A$(0), B#`) currently fail to parse. Fix in
`argumentList`.
- ~~**Multiple statements per line**~~ **Done.** `akbasic_parser_parse()` consumes a run of
`COLON` tokens before each statement and hands the caller a NULL leaf when nothing followed
them, which is how an empty statement — a trailing separator, or `::` — stays legal rather
than becoming an error. `tests/parser_commands.c` counts statements per line;
`tests/language/statements/multiple_per_line.bas` asserts what actually runs.
**One thing about it needed a decision rather than a transcription**, and it is recorded as
deviation 29 in §5: BASIC 7.0 scopes everything after `THEN` to the condition, so
`IF C THEN A : B` must run neither `A` nor `B` when `C` is false. The parser takes exactly one
statement per arm, so the rest of the line reaches the statement loop as ordinary statements
and the branch has to tell the loop whether to run them.
**A known limitation, not a defect:** a whole `FOR`/`NEXT` on one line
(`FOR I# = 1 TO 3 : PRINT I# : NEXT I#`) does not loop. The block structure is the
reference's `waitingForCommand` model (§1.6), which skips forward *by source line* to the verb
it is waiting for, so a `NEXT` on the same line as its `FOR` is never reached. Fixing it means
restructuring control flow to work on statements rather than lines, which is a deliberate
piece of work and wants its own commit. Group A's `DO`/`LOOP` will meet the same wall.
- **Array references in parameter lists** (`READ A$(0), B#`) currently fail to parse. This is
the *same collision* §6 item 13 was, one field along: an identifier keeps its subscript list
on `.right`, which is also where an argument list chains its arguments. That item was fixed by
moving a unary leaf's operand to `.left`; the same move very likely works here, since an
identifier leaf has no other use for the field either. Fix in `akbasic_parser_primary` and
`akbasic_leaf_first_subscript`.
---
@@ -727,6 +746,29 @@ deviations from the reference's *program*: `main.go` and the SDL half of
driven through the SDL binary, which is where most of the frontend's real coverage comes
from.
### Deviations in statement separation
29. **A branch decides who owns the rest of its line.** BASIC 7.0 scopes every statement after
`THEN` to the condition, and the reference has no opinion on the matter because it never
consumed the `COLON` token at all. The parser here takes exactly one statement for each arm,
so the rest of the line arrives at the statement loop as ordinary top-level statements and
something has to say whether to run them. The `BRANCH` case in
`akbasic_runtime_evaluate()` sets `runtime->skiprestofline`, and the two statement loops
stop on it.
**The rule is not "skip when false"**, which is the reason this is written down:
| Line | Condition | What runs |
|---|---|---|
| `IF C THEN A : B` | true | `A`, `B` |
| `IF C THEN A : B` | false | nothing |
| `IF C THEN A ELSE B : D` | true | `A` |
| `IF C THEN A ELSE B : D` | false | `B`, `D` |
The remainder always belongs to whichever arm was written *last*, so it is skipped exactly
when that arm is the one not taken — with an `ELSE` the last arm is `ELSE`, without one it
is `THEN`. That is one line of code and it reads as an oddity without the table above it.
---
## 6. Reference defects — ported faithfully, fix them deliberately
@@ -977,10 +1019,10 @@ entire test corpus and passes clean under ASan and UBSan.
| Gate | Result |
|---|---|
| `ctest` | 72/72 — 41 upstream golden cases, 5 local ones, 22 unit tests, 2 embedding examples, 1 known-failing, 1 version check |
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 72/72 — the same, minus the three `no_device` cases the SDL driver contradicts, plus `akgl_backends` and `akgl_frontend`; the `akgl_build` CI job |
| `ctest` | 74/74 — 41 upstream golden cases, 8 local ones, 22 unit tests, 2 embedding examples, 1 known-failing |
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 73/73 — the same, minus the three `no_device` cases the SDL driver contradicts, plus `akgl_backends` and `akgl_frontend`; the `akgl_build` CI job |
| Golden corpus | 41/41 byte-exact, driven in place from the submodule — **and 41/41 again through the SDL binary**, which is most of what proves the frontend changes no output |
| ASan + UBSan | 72/72 |
| ASan + UBSan | 74/74 |
| Line coverage | 93.6% (3618/3867) — above the 90% gate |
| Function coverage | 97.8% (267/273) |
| Warnings | none under `-Wall -Wextra` |
@@ -1066,10 +1108,18 @@ What remains, in priority order:
2. ~~**A line editor for the akgl sink.**~~ **Done**`readline` in `src/sink_akgl.c`, over
the keystroke ring, borrowing frames from the host through an `akbasic_AkglPump`. It cannot
type a shifted character, which is `libakgl` item 10 rather than work outstanding here.
3. **§4 — the language completion work queue.** Groups G and I and the `GET`/`GETKEY`/`SCNCLR`
part of E are done. Of what is left, **multiple statements per line** should come first:
the `COLON` token exists and nothing consumes it, and `DO`/`LOOP` in group A reads badly
without it. Then groups A, B, D, F and J, none of which need anything from `libakgl`.
3. **§4 — the language completion work queue.** Groups G and I, the `GET`/`GETKEY`/`SCNCLR`
part of E, and **multiple statements per line** are done. What is left is groups A, B, D, F
and J, none of which need anything from `libakgl`.
**Do group A next, and expect it to cost more than the table suggests.** Multiple statements
per line turned up the wall it will hit: the `waitingForCommand` model skips forward *by
source line*, so a whole loop written on one line
(`FOR I# = 1 TO 3 : PRINT I# : NEXT I#`) never reaches its `NEXT`. §4 records it as a known
limitation rather than a defect, because it is inherited structure and not a slip — but
`DO`/`LOOP`/`WHILE`/`UNTIL` are exactly the verbs people write on one line, so group A
probably has to move block skipping from lines to statements before it starts. That is a
deliberate restructure and wants its own commit and its own test, ahead of any verb.
4. ~~**CI does not cover `-DAKBASIC_WITH_AKGL=ON`.**~~ **Done** — the `akgl_build` job in
`.gitea/workflows/ci.yaml`. It turned out much cheaper than the deferral assumed, and the
two reasons are worth keeping because both were guesses that measurement corrected: