Reset the scanner's leftover token type between lines

The REM early-exit leaves tokentype holding AKBASIC_TOK_REM, and the scan
loop's post-switch check reads it before the next line's first character
has assigned anything. A line opening with whitespace then re-triggered
the REM break and scanned to nothing: every indented line after a REM was
silently skipped. Numbered programs never saw it -- the line number is the
first token and overwrites the leftover -- which is why the whole golden
corpus missed it and the unnumbered, indented galaga.bas found it.

Co-authored-by: andrew <andrew@aklabs.net>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XiGgpHuXUm2mR4Wzndw3dc
This commit is contained in:
2026-08-04 08:46:51 -04:00
parent 17af2d406c
commit dde1d91c6e
3 changed files with 28 additions and 0 deletions

View File

@@ -0,0 +1,14 @@
REM The line after this comment is indented, and it must still run: the
REM scanner's REM early-exit used to leave TOK_REM armed, and the next
REM line's leading whitespace re-triggered it -- every indented line
REM after a REM was silently skipped. Unnumbered on purpose: a numbered
REM line's first token overwrites the leftover and hides the defect.
PRINT 1
REM an indented statement follows
PRINT 2
REM an indented multi-line DEF body, the shape that found it
DEF F(N#)
REM a comment inside the body
RETURN N# + 5
PRINT F(10)
END

View File

@@ -0,0 +1,3 @@
1
2
15