andrew/akbasic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Port the BASIC interpreter from Go to C

Browse Source 
Reproduces deps/basicinterpret in C, in the idiom of the ak* libraries. All 41
.bas files in the reference's corpus produce byte-identical stdout, including
the trailing double newline on an error line -- that comes from basicError
building a string ending in \n and handing it to Println, and
array_outofbounds.txt encodes it.

The corpus is driven in place from the submodule as 41 individual CTest cases
rather than copied, so it cannot drift from upstream. Eighteen unit tests cover
what the corpus cannot reach.

Three structural changes carry most of the work. Go's three reflection lookups
(Command*, Function*, ParseCommand*) become one sorted dispatch table in
src/verbs.c searched with bsearch; adding a verb is a row and two functions. The
five Go maps become one fixed open-addressed table over aksl_strhash_djb2. And
run(), which owned the process until MODE_QUIT, splits into step() plus a
bounded run() -- goal 3 requires a host game to be able to bound execution, and
nothing in the library now terminates the process or touches SDL.

Output goes through an akbasic_TextSink vtable. src/sink_stdio.c is what makes
the corpus runnable with no SDL present; the akgl-backed sink is still to come
and is blocked on libakgl having no text-measurement call.

src/convert.c exists because libakstdlib's aksl_ato* family cannot report a
conversion failure (its TODO.md 2.1.5). The reference checks strconv's error at
four sites and turns it into a BASIC error; routing those through aksl_atoi
would have turned four diagnosable errors into wrong answers, with VAL("garbage")
quietly returning 0. TODO.md 1.9 records which libakstdlib calls are cleared for
use here and which are not.

Reference defects are reproduced, not fixed: the golden files encode the observed
behaviour and a silent correction is a behaviour change. TODO.md section 6 lists
sixteen, and tests/known_reference_defects.c asserts the *correct* contract for
six of them under AKBASIC_KNOWN_FAILING_TESTS, so a fix shows up as
"unexpectedly passed". Five of the sixteen were found by this port and are new:
subtraction stops after one operator so 1-2-3 computes 1-2 and abandons the rest
of the line (a wrong answer, not a refused one); a unary-minus argument inflates
a function's arity so ABS(-9) is rejected; a comparison operator in a line's
final column is dropped; hex literals never survive the scanner; and the
"Reserved word in variable name" check is dead code.

Where the reference reaches undefined behaviour by a route that is defined in Go
-- an out-of-range shift, a negative string multiplier, integer division by zero
-- this raises instead of inheriting the UB. No golden case exercises any of
them.

The top-level CMakeLists shadows add_test, set_tests_properties and
add_custom_target around all three add_subdirectory calls. Without it libakerror's
tests land in our suite as Not Run, and its un-namespaced `coverage` target stops
a coverage build from configuring at all. Test targets are akbasic_test_<name>:
bare test_<name> collides with libakstdlib's, which is what broke libakgl's
configure in c2b16d3.

ctest 59/59; ASan+UBSan 59/59; 92.3% line and 96.9% function coverage; no
warnings under -Wall -Wextra. Branch coverage is not a target, for the reason
libakstdlib and libakgl both record: the akerror macros expand into large branch
trees at every call site.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Kesterson
2026-07-30 23:53:56 -04:00
commit f8c15c2f2c
60 changed files with 10192 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

376
src/scanner.c Normal file
View File

@@ -0,0 +1,376 @@
/**
* @file scanner.c
* @brief Implements the line tokenizer.
*/
#include <ctype.h>
#include <inttypes.h>
#include <stdio.h>
#include <string.h>
#include <akerror.h>
#include <akbasic/convert.h>
#include <akbasic/error.h>
#include <akbasic/scanner.h>
#include <akbasic/verbs.h>
akerr_ErrorContext *akbasic_scanner_zero(akbasic_Runtime *obj)
{
PREPARE_ERROR(errctx);
FAIL_ZERO_RETURN(errctx, (obj != NULL), AKERR_NULLPOINTER, "NULL runtime in scanner zero");
obj->current = 0;
obj->start = 0;
obj->hasError = false;
SUCCEED_RETURN(errctx);
}
static bool is_at_end(akbasic_Runtime *obj)
{
return (obj->current >= (int)strlen(obj->line));
}
static bool peek(akbasic_Runtime *obj, char *dest)
{
if ( is_at_end(obj) ) {
return false;
}
*dest = obj->line[obj->current];
return true;
}
static bool peek_next(akbasic_Runtime *obj, char *dest)
{
if ( (obj->current + 1) >= (int)strlen(obj->line) ) {
return false;
}
*dest = obj->line[obj->current + 1];
return true;
}
/*
* The lexeme is the span [start, current), with two special cases the reference
* relies on: at end of line it runs to the end, and a zero-width span yields the
* single character at `start` -- unless we are closing a string literal, where
* an empty span really is the empty string.
*/
static akerr_ErrorContext *get_lexeme(akbasic_Runtime *obj, char *dest, size_t len)
{
PREPARE_ERROR(errctx);
int linelen = (int)strlen(obj->line);
int span = 0;
if ( obj->current == linelen ) {
span = linelen - obj->start;
} else if ( obj->start == obj->current ) {
if ( obj->tokentype == AKBASIC_TOK_LITERAL_STRING ) {
dest[0] = '\0';
SUCCEED_RETURN(errctx);
}
span = 1;
} else {
span = obj->current - obj->start;
}
FAIL_ZERO_RETURN(errctx, (span >= 0 && (size_t)span < len), AKBASIC_ERR_BOUNDS,
"Lexeme of %d characters exceeds the %zu character limit", span, len - 1);
memcpy(dest, obj->line + obj->start, (size_t)span);
dest[span] = '\0';
SUCCEED_RETURN(errctx);
}
static akerr_ErrorContext *add_token(akbasic_Runtime *obj, akbasic_TokenType token, const char *lexeme)
{
PREPARE_ERROR(errctx);
akbasic_Environment *env = obj->environment;
FAIL_ZERO_RETURN(errctx, (env->nexttoken < AKBASIC_MAX_TOKENS), AKBASIC_ERR_BOUNDS,
"Line %" PRId64 " has more than %d tokens",
env->lineno, AKBASIC_MAX_TOKENS);
FAIL_ZERO_RETURN(errctx, (strlen(lexeme) < AKBASIC_MAX_LINE_LENGTH), AKBASIC_ERR_BOUNDS,
"Token lexeme exceeds the %d character limit", AKBASIC_MAX_LINE_LENGTH - 1);
env->tokens[env->nexttoken].tokentype = token;
env->tokens[env->nexttoken].lineno = env->lineno;
strncpy(env->tokens[env->nexttoken].lexeme, lexeme, AKBASIC_MAX_LINE_LENGTH - 1);
env->tokens[env->nexttoken].lexeme[AKBASIC_MAX_LINE_LENGTH - 1] = '\0';
env->nexttoken += 1;
SUCCEED_RETURN(errctx);
}
/* Consume one more character when it matches, choosing between two token types. */
static bool match_next_char(akbasic_Runtime *obj, char cm, akbasic_TokenType truetype, akbasic_TokenType falsetype)
{
char nc = '\0';
if ( !peek(obj, &nc) ) {
return false;
}
if ( nc == cm ) {
obj->current += 1;
obj->tokentype = truetype;
return true;
}
obj->tokentype = falsetype;
return false;
}
static akerr_ErrorContext *match_string(akbasic_Runtime *obj)
{
PREPARE_ERROR(errctx);
char c = '\0';
while ( !is_at_end(obj) ) {
if ( !peek(obj, &c) ) {
PASS(errctx, akbasic_runtime_error(obj, AKBASIC_ERRCLASS_PARSE,
"UNTERMINATED STRING LITERAL\n"));
obj->hasError = true;
SUCCEED_RETURN(errctx);
}
if ( c == '"' ) {
break;
}
obj->current += 1;
}
obj->tokentype = AKBASIC_TOK_LITERAL_STRING;
SUCCEED_RETURN(errctx);
}
static akerr_ErrorContext *match_number(akbasic_Runtime *obj)
{
PREPARE_ERROR(errctx);
bool linenumber = (obj->environment->nexttoken == 0);
char lexeme[AKBASIC_MAX_LINE_LENGTH];
char c = '\0';
char nc = '\0';
int64_t lineno = 0;
obj->tokentype = AKBASIC_TOK_LITERAL_INT;
while ( !is_at_end(obj) ) {
(void)peek(obj, &c);
if ( c == '.' ) {
if ( !peek_next(obj, &nc) || !isdigit((unsigned char)nc) ) {
PASS(errctx, akbasic_runtime_error(obj, AKBASIC_ERRCLASS_PARSE,
"INVALID FLOATING POINT LITERAL\n"));
obj->hasError = true;
SUCCEED_RETURN(errctx);
}
obj->tokentype = AKBASIC_TOK_LITERAL_FLOAT;
} else if ( !isdigit((unsigned char)c) && c != 'x' ) {
/* 'x' is allowed through so 0x-prefixed hex reaches the parser. */
break;
}
obj->current += 1;
}
if ( obj->tokentype == AKBASIC_TOK_LITERAL_INT && linenumber ) {
PASS(errctx, get_lexeme(obj, lexeme, sizeof(lexeme)));
ATTEMPT {
CATCH(errctx, akbasic_str_to_int64(lexeme, 10, &lineno));
} CLEANUP {
} PROCESS(errctx) {
} HANDLE_DEFAULT(errctx) {
char message[AKBASIC_MAX_LINE_LENGTH + 32];
snprintf(message, sizeof(message), "INTEGER CONVERSION ON '%s'", lexeme);
/*
* Reporting can itself fail if the sink is broken. Nothing useful
* remains to be done about that here, so record the flag and let the
* next operation surface it.
*/
IGNORE(akbasic_runtime_error(obj, AKBASIC_ERRCLASS_PARSE, message));
obj->hasError = true;
} FINISH(errctx, false);
if ( obj->hasError ) {
SUCCEED_RETURN(errctx);
}
obj->environment->lineno = lineno;
obj->tokentype = AKBASIC_TOK_LINE_NUMBER;
}
SUCCEED_RETURN(errctx);
}
static akerr_ErrorContext *match_identifier(akbasic_Runtime *obj)
{
PREPARE_ERROR(errctx);
char lexeme[AKBASIC_MAX_LINE_LENGTH];
const akbasic_Verb *verb = NULL;
void *fndef = NULL;
bool userfunction = false;
char c = '\0';
obj->tokentype = AKBASIC_TOK_IDENTIFIER;
while ( !is_at_end(obj) ) {
(void)peek(obj, &c);
if ( isdigit((unsigned char)c) || isalpha((unsigned char)c) ) {
obj->current += 1;
continue;
}
switch ( c ) {
case '@':
obj->tokentype = AKBASIC_TOK_IDENTIFIER_STRUCT;
obj->current += 1;
break;
case '$':
obj->tokentype = AKBASIC_TOK_IDENTIFIER_STRING;
obj->current += 1;
break;
case '%':
obj->tokentype = AKBASIC_TOK_IDENTIFIER_FLOAT;
obj->current += 1;
break;
case '#':
obj->tokentype = AKBASIC_TOK_IDENTIFIER_INT;
obj->current += 1;
break;
default:
break;
}
break;
}
PASS(errctx, get_lexeme(obj, lexeme, sizeof(lexeme)));
PASS(errctx, akbasic_verb_lookup(lexeme, &verb));
ATTEMPT {
CATCH(errctx, akbasic_environment_get_function(obj->environment, lexeme, &fndef));
userfunction = true;
} CLEANUP {
} PROCESS(errctx) {
} HANDLE(errctx, AKERR_KEY) {
userfunction = false;
} FINISH(errctx, true);
if ( obj->tokentype == AKBASIC_TOK_IDENTIFIER ) {
if ( verb != NULL ) {
obj->tokentype = verb->tokentype;
} else if ( userfunction ) {
obj->tokentype = AKBASIC_TOK_FUNCTION;
}
} else if ( verb != NULL ) {
/*
* A suffixed identifier that collides with a verb or function name.
* PRINT$ is not a variable.
*/
PASS(errctx, akbasic_runtime_error(obj, AKBASIC_ERRCLASS_SYNTAX,
"Reserved word in variable name\n"));
obj->hasError = true;
}
SUCCEED_RETURN(errctx);
}
akerr_ErrorContext *akbasic_scanner_scan(akbasic_Runtime *obj, const char *line, char *dest, size_t len)
{
PREPARE_ERROR(errctx);
char lexeme[AKBASIC_MAX_LINE_LENGTH];
char c = '\0';
bool done = false;
FAIL_ZERO_RETURN(errctx, (obj != NULL), AKERR_NULLPOINTER, "NULL runtime in scan");
FAIL_ZERO_RETURN(errctx, (line != NULL), AKERR_NULLPOINTER, "NULL line in scan");
FAIL_ZERO_RETURN(errctx, (strlen(line) < AKBASIC_MAX_LINE_LENGTH), AKBASIC_ERR_BOUNDS,
"Source line of %zu characters exceeds the %d character limit",
strlen(line), AKBASIC_MAX_LINE_LENGTH - 1);
strncpy(obj->line, line, AKBASIC_MAX_LINE_LENGTH - 1);
obj->line[AKBASIC_MAX_LINE_LENGTH - 1] = '\0';
PASS(errctx, akbasic_environment_zero_parser(obj->environment));
obj->current = 0;
obj->start = 0;
obj->hasError = false;
while ( !is_at_end(obj) && !done ) {
c = obj->line[obj->current];
obj->current += 1;
switch ( c ) {
case '@': obj->tokentype = AKBASIC_TOK_ATSYMBOL; break;
case '^': obj->tokentype = AKBASIC_TOK_CARAT; break;
case '(': obj->tokentype = AKBASIC_TOK_LEFT_PAREN; break;
case ')': obj->tokentype = AKBASIC_TOK_RIGHT_PAREN; break;
case '+': obj->tokentype = AKBASIC_TOK_PLUS; break;
case '-': obj->tokentype = AKBASIC_TOK_MINUS; break;
case '/': obj->tokentype = AKBASIC_TOK_LEFT_SLASH; break;
case '*': obj->tokentype = AKBASIC_TOK_STAR; break;
case ',': obj->tokentype = AKBASIC_TOK_COMMA; break;
case ':': obj->tokentype = AKBASIC_TOK_COLON; break;
case '[': obj->tokentype = AKBASIC_TOK_LEFT_SQUAREBRACKET; break;
case ']': obj->tokentype = AKBASIC_TOK_RIGHT_SQUAREBRACKET; break;
case '=':
(void)match_next_char(obj, '=', AKBASIC_TOK_EQUAL, AKBASIC_TOK_ASSIGNMENT);
break;
case '<':
if ( !match_next_char(obj, '=', AKBASIC_TOK_LESS_THAN_EQUAL, AKBASIC_TOK_LESS_THAN) ) {
(void)match_next_char(obj, '>', AKBASIC_TOK_NOT_EQUAL, AKBASIC_TOK_LESS_THAN);
}
break;
case '>':
(void)match_next_char(obj, '=', AKBASIC_TOK_GREATER_THAN_EQUAL, AKBASIC_TOK_GREATER_THAN);
break;
case '"':
obj->start = obj->current;
PASS(errctx, match_string(obj));
break;
case '\t':
case ' ':
obj->start = obj->current;
break;
case '\r':
case '\n':
done = true;
break;
default:
if ( isdigit((unsigned char)c) ) {
PASS(errctx, match_number(obj));
} else if ( isalpha((unsigned char)c) ) {
PASS(errctx, match_identifier(obj));
} else {
char message[AKBASIC_MAX_LINE_LENGTH];
snprintf(message, sizeof(message), "UNKNOWN TOKEN %c\n", c);
PASS(errctx, akbasic_runtime_error(obj, AKBASIC_ERRCLASS_PARSE, message));
obj->hasError = true;
obj->start = obj->current;
}
break;
}
if ( done ) {
break;
}
if ( obj->tokentype != AKBASIC_TOK_UNDEFINED && !obj->hasError ) {
if ( obj->tokentype == AKBASIC_TOK_REM ) {
/* Everything after REM is a comment. Stop, keeping the line intact. */
break;
} else if ( obj->tokentype == AKBASIC_TOK_LINE_NUMBER ) {
/*
* The line number is not kept as a token. Rewrite the line to
* everything after it, minus leading spaces, and restart the
* cursor -- the REPL reads the rewritten line back out and
* stores *that* as the program text.
*/
int skip = obj->current;
while ( obj->line[skip] == ' ' ) {
skip += 1;
}
memmove(obj->line, obj->line + skip, strlen(obj->line + skip) + 1);
obj->current = 0;
} else {
PASS(errctx, get_lexeme(obj, lexeme, sizeof(lexeme)));
PASS(errctx, add_token(obj, obj->tokentype, lexeme));
if ( obj->tokentype == AKBASIC_TOK_LITERAL_STRING ) {
/* Scanning stopped on the closing quote; step past it. */
obj->current += 1;
}
}
obj->tokentype = AKBASIC_TOK_UNDEFINED;
obj->start = obj->current;
}
}
if ( dest != NULL ) {
FAIL_ZERO_RETURN(errctx, (strlen(obj->line) < len), AKBASIC_ERR_BOUNDS,
"Scanned line does not fit the caller's buffer");
strncpy(dest, obj->line, len - 1);
dest[len - 1] = '\0';
}
SUCCEED_RETURN(errctx);
}