andrew/akbasic
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Fix five defects the port uncovered in the reference parser and scanner

Browse Source 
Chained subtraction and exponentiation fold left instead of abandoning the
rest of the line -- `1 - 2 - 3` gave -1 and dropped the `- 3`. A unary leaf's
operand moves to .left, so a negative literal is one argument again and a
second argument no longer overwrites it. An operator in a line's final column
survives. Hex literals reach the parser whole. A leading zero is padding, not
a radix.

Item 16 stays pinned: the fix works and costs an upstream golden case, which
is a decision rather than a defect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Kesterson
2026-07-31 11:35:14 -04:00
parent f24201fdef
commit 1eb8f9ffaa
14 changed files with 272 additions and 167 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
tests/scanner_tokens.c
View File

@@ -43,13 +43,7 @@ int main(void)
expect_token(6, AKBASIC_TOK_RIGHT_PAREN, ")");
expect_token(7, AKBASIC_TOK_COMMA, ",");
/*
* Two-character comparisons, and the one-character fallbacks. The trailing
* space is not decoration: matchNextChar cannot peek past the end of the
* line, and when it fails it returns without setting a token type, so a
* comparison operator in the final column is silently dropped. TODO.md
* section 12 item 14.
*/
/* Two-character comparisons, and the one-character fallbacks. */
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "<= <> >= < > == = ", NULL, 0));
TEST_REQUIRE_INT(env()->nexttoken, 7);
expect_token(0, AKBASIC_TOK_LESS_THAN_EQUAL, NULL);
@@ -60,9 +54,25 @@ int main(void)
expect_token(5, AKBASIC_TOK_EQUAL, NULL);
expect_token(6, AKBASIC_TOK_ASSIGNMENT, NULL);
/* Without the trailing space the final operator vanishes. */
/*
* An operator in the final column survives, with no trailing space to help
* it. matchNextChar cannot peek past the end of the line, and the reference
* returns from that without setting a token type (basicscanner.go:272), so
* `A# =` produced one token and the parse error that followed pointed
* somewhere else. TODO.md section 6 item 14; this is its regression test.
*/
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "A# =", NULL, 0));
TEST_REQUIRE_INT(env()->nexttoken, 1);
TEST_REQUIRE_INT(env()->nexttoken, 2);
expect_token(0, AKBASIC_TOK_IDENTIFIER_INT, "A#");
expect_token(1, AKBASIC_TOK_ASSIGNMENT, "=");
/* The same for the other two, each alone in the final column. */
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "A# <", NULL, 0));
TEST_REQUIRE_INT(env()->nexttoken, 2);
expect_token(1, AKBASIC_TOK_LESS_THAN, "<");
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "A# >", NULL, 0));
TEST_REQUIRE_INT(env()->nexttoken, 2);
expect_token(1, AKBASIC_TOK_GREATER_THAN, ">");
/* Literals. */
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT 1.5", NULL, 0));
@@ -70,13 +80,25 @@ int main(void)
expect_token(1, AKBASIC_TOK_LITERAL_FLOAT, "1.5");
/*
* A hex literal does not survive the scanner: matchNumber allows 'x' through
* but not the hex digits after it, so "0xff" lexes as "0x" and the "ff" is
* scanned as an identifier. The parser's base-16 branch is therefore
* unreachable. TODO.md section 12 item 15.
* A hex literal survives the scanner whole. The reference lets 'x' through
* but stops at the hex digits after it (basicscanner.go:318), so "0xff"
* lexed as "0x" plus an identifier "ff", the parser's base-16 branch was
* unreachable and the README's hex support did not exist. TODO.md section 6
* item 15; this is its regression test.
*/
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT 0xff", NULL, 0));
expect_token(1, AKBASIC_TOK_LITERAL_INT, "0x");
TEST_REQUIRE_INT(env()->nexttoken, 2);
expect_token(1, AKBASIC_TOK_LITERAL_INT, "0xff");
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT 0xDEAD", NULL, 0));
expect_token(1, AKBASIC_TOK_LITERAL_INT, "0xDEAD");
/* And the hex run ends where it should, rather than eating what follows. */
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT 0x10 + 1", NULL, 0));
TEST_REQUIRE_INT(env()->nexttoken, 4);
expect_token(1, AKBASIC_TOK_LITERAL_INT, "0x10");
expect_token(2, AKBASIC_TOK_PLUS, "+");
expect_token(3, AKBASIC_TOK_LITERAL_INT, "1");
TEST_REQUIRE_OK(akbasic_scanner_scan(&HARNESS_RUNTIME, "PRINT \"HELLO\"", NULL, 0));
expect_token(1, AKBASIC_TOK_LITERAL_STRING, "HELLO");