Reset the scanner's leftover token type between lines

The REM early-exit leaves tokentype holding AKBASIC_TOK_REM, and the scan
loop's post-switch check reads it before the next line's first character
has assigned anything. A line opening with whitespace then re-triggered
the REM break and scanned to nothing: every indented line after a REM was
silently skipped. Numbered programs never saw it -- the line number is the
first token and overwrites the leftover -- which is why the whole golden
corpus missed it and the unnumbered, indented galaga.bas found it.

Co-authored-by: andrew <andrew@aklabs.net>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XiGgpHuXUm2mR4Wzndw3dc
This commit is contained in:
2026-08-04 08:46:51 -04:00
parent 17af2d406c
commit dde1d91c6e
3 changed files with 28 additions and 0 deletions

View File

@@ -23,6 +23,7 @@ akerr_ErrorContext *akbasic_scanner_zero(akbasic_Runtime *obj)
obj->current = 0;
obj->start = 0;
obj->hasError = false;
obj->tokentype = AKBASIC_TOK_UNDEFINED;
SUCCEED_RETURN(errctx);
}
@@ -409,6 +410,16 @@ akerr_ErrorContext *akbasic_scanner_scan(akbasic_Runtime *obj, const char *line,
obj->current = 0;
obj->start = 0;
obj->hasError = false;
/*
* The `REM` early-exit below leaves `tokentype` holding AKBASIC_TOK_REM,
* and the loop's post-switch check reads it before the first character of
* the *next* line has assigned anything. A line whose first character
* carries no token of its own -- leading whitespace -- then re-triggered
* the REM break and scanned to nothing: every indented line after a REM
* was silently skipped. A numbered program never saw it, because the line
* number is the first token and overwrites the leftover.
*/
obj->tokentype = AKBASIC_TOK_UNDEFINED;
/*
* Cleared here rather than by each caller, so the flag always describes the
* line this call just scanned. It used to be cleared only in