Implement generators: GEN, EMIT, END GEN, FOR EACH and DO EACH (issue 57)

Adds generator support per the plan in issue 57:

- environment.h: isGenerator/generatorFn on a GEN call's own environment,
  isEachLoop and forGeneratorEnv on a FOR EACH/DO EACH loop's own
  environment.
- runtime.c: splits akbasic_runtime_prev_environment() into
  akbasic_runtime_detach_environment() (return to parent without releasing)
  and akbasic_runtime_release_environment() (give variables and the pool
  slot back, on any environment); prev_environment() is now the two in
  sequence. akbasic_runtime_call_function() refuses to call a GEN like an
  ordinary function.
- verbs.c/verbs.h/scanner: new keywords GEN, EMIT, EACH, IN and the compound
  verb "END GEN" (built by a new akbasic_parse_end(), the same trick
  akbasic_parse_print() uses for PRINT #).
- parser_commands.c: akbasic_parse_gen() (modelled on multi-line DEF),
  akbasic_parse_end(), and EACH branches in akbasic_parse_for()/
  akbasic_parse_do().
- runtime_generator.c (new): akbasic_cmd_gen, akbasic_cmd_emit,
  akbasic_cmd_end_gen, and the invoke/pump/release machinery FOR EACH, DO
  EACH, NEXT and LOOP share. EMIT walks up to the nearest isGenerator
  ancestor rather than assuming it is standing directly in the GEN's own
  call frame, because a GEN body may nest its own FOR/DO/GOSUB around an
  EMIT -- the issue's own ROOMOBJECTS example does exactly that.
- runtime_commands.c/runtime_structure.c: EACH branches in cmd_for/cmd_do,
  matching EACH branches in cmd_next/cmd_loop, and forGeneratorEnv release
  on every path that can abandon a live generator (EXIT, a NEXT that pops
  for a mismatched loop variable).

Deviates from the plan in one place: FunctionDef gained an isGenerator flag
(not in the plan's field list) because refusing a GEN called like a
function has to happen before anything is pushed. Relying on EMIT's own
isGenerator check for that case doesn't work: akbasic_runtime_call_function()
drives its own step loop the same way akbasic_runtime_pump_generator() does,
and a BASIC-level error inside that loop is swallowed by process_line_run()
as reported-but-not-propagated, so the call would silently "succeed" with a
meaningless return value instead of failing.

Also: a zero-argument parameter list is not supported by the DEF/GEN
parameter parser this reuses (a pre-existing limitation, not
generator-specific); every generator in the tests takes at least one
parameter as a result.

Tests: tests/generators.c (pool exhaustion under repeated EXIT, calling a
GEN like a function, EMIT outside a GEN, self-recursion, sibling/nested
invocations) and tests/language/flowcontrol/generators_*.bas -- the
issue's own ROOMOBJECTS example in both loop shapes, an empty generator,
non-numeric EMIT, nested/interleaved invocations, and three error-path
golden cases. Docs: control-flow chapter 4 gets a GEN/EMIT/FOR EACH/DO EACH
section, the verb reference gets GEN/EMIT/END GEN entries and updated
FOR/DO/NEXT/LOOP/EXIT rows, and architecture chapter 14 documents the
detach/release split and the two-environment generator invocation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

src/parser_commands.c

@@ -20,6 +20,9 @@
 
 #include "verbs.h"
 
+/* Shared by akbasic_parse_for() and akbasic_parse_do(); defined after akbasic_parse_def(). */
+static akerr_ErrorContext *parse_each_clause(akbasic_Parser *parser, akbasic_ASTLeaf **var, akbasic_ASTLeaf **callexpr);
+
 akerr_ErrorContext *akbasic_parse_arglist(akbasic_Parser *parser, akbasic_ASTLeaf **dest)
 {
     PREPARE_ERROR(errctx);
@@ -309,8 +312,45 @@ akerr_ErrorContext *akbasic_parse_do(akbasic_Parser *parser, akbasic_ASTLeaf **d
     akbasic_Environment *newenv = NULL;
     akbasic_ASTLeaf *expr = NULL;
     akbasic_ASTLeaf *condition = NULL;
+    akbasic_Token *peeked = NULL;
     int kind = AKBASIC_LOOPCOND_NONE;
     int64_t firstline = parent->lineno + 1;
+    int cmp = 0;
+
+    /*
+     * DO EACH <variable> IN <generator call> ... LOOP. Mutually exclusive with
+     * DO WHILE/UNTIL on the same DO, so this is checked first and returns
+     * before any of the WHILE/UNTIL machinery runs.
+     */
+    peeked = akbasic_parser_peek(parser);
+    if ( peeked != NULL && peeked->tokentype == AKBASIC_TOK_COMMAND ) {
+	PASS(errctx, aksl_strcmp(peeked->lexeme, "EACH", &cmp));
+	if ( cmp == 0 ) {
+	    akbasic_ASTLeaf *var = NULL;
+	    akbasic_ASTLeaf *callexpr = NULL;
+
+	    (void)akbasic_parser_match1(parser, AKBASIC_TOK_COMMAND);
+
+	    PASS(errctx, akbasic_runtime_new_environment(runtime));
+	    newenv = runtime->environment;
+	    runtime->environment = parent;
+
+	    PASS(errctx, parse_each_clause(parser, &var, &callexpr));
+
+	    newenv->isDoLoop = true;
+	    newenv->isEachLoop = true;
+	    newenv->loopFirstLine = firstline;
+	    /* See akbasic_parse_for()'s EACH branch for why this is not cloned. */
+	    newenv->forToLeaf = callexpr;
+
+	    PASS(errctx, akbasic_parser_new_leaf(parser, &expr));
+	    PASS(errctx, akbasic_leaf_new_command(expr, "DO", var));
+
+	    runtime->environment = newenv;
+	    *dest = expr;
+	    SUCCEED_RETURN(errctx);
+	}
+    }
 
     PASS(errctx, akbasic_runtime_new_environment(runtime));
     newenv = runtime->environment;
@@ -883,6 +923,134 @@ akerr_ErrorContext *akbasic_parse_def(akbasic_Parser *parser, akbasic_ASTLeaf **
     SUCCEED_RETURN(errctx);
 }
 
+/*
+ * GEN NAME(parameters) ... END GEN
+ *
+ * A DEF that yields more than once, in the same shape a multi-line DEF is:
+ * the header is parsed, the parameter list is read with parse_def_parameters()
+ * (GEN and DEF share the functions table -- TODO.md's namespace decision for
+ * this feature -- so a name cannot be both), and the body is skipped on this,
+ * the definitional pass, by arming akbasic_environment_wait_for_command() for
+ * "END GEN" instead of "RETURN". It only really executes when a FOR EACH/DO
+ * EACH invokes it through akbasic_runtime_generator_invoke(), which sets
+ * `nextline` to `fndef->lineno` directly and never runs this line again.
+ *
+ * There is no single-expression form: a GEN with nothing to loop over is just
+ * a DEF, and EMIT already needs a body to sit in.
+ */
+akerr_ErrorContext *akbasic_parse_gen(akbasic_Parser *parser, akbasic_ASTLeaf **dest)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_Runtime *runtime = parser->runtime;
+    akbasic_ASTLeaf *identifier = NULL;
+    akbasic_ASTLeaf *arglist = NULL;
+    akbasic_ASTLeaf *command = NULL;
+    akbasic_FunctionDef *fndef = NULL;
+    size_t namelen = 0;
+    size_t i = 0;
+
+    PASS(errctx, akbasic_parser_primary(parser, &identifier));
+    FAIL_ZERO_RETURN(errctx, (identifier->leaftype == AKBASIC_LEAF_IDENTIFIER),
+		     AKBASIC_ERR_SYNTAX, "Expected identifier");
+
+    PASS(errctx, parse_def_parameters(parser, &arglist));
+
+    PASS(errctx, akbasic_runtime_new_function(runtime, &fndef));
+
+    /* Uppercase the name: verbs, functions and generators are all case-insensitive. */
+    PASS(errctx, aksl_strlen(identifier->identifier, &namelen));
+    FAIL_ZERO_RETURN(errctx, (namelen < sizeof(fndef->name)),
+		     AKBASIC_ERR_BOUNDS, "Function name '%s' is too long", identifier->identifier);
+    for ( i = 0; i < namelen; i++ ) {
+	char c = identifier->identifier[i];
+	fndef->name[i] = (char)((c >= 'a' && c <= 'z') ? (c - 'a' + 'A') : c);
+    }
+    fndef->name[namelen] = '\0';
+
+    fndef->expression = NULL;
+    fndef->isGenerator = true;
+    PASS(errctx, akbasic_environment_wait_for_command(runtime->environment, "END GEN"));
+    PASS(errctx, akbasic_leaf_clone(arglist, &fndef->leafpool, &fndef->arglist));
+    fndef->lineno = runtime->environment->lineno + 1;
+
+    PASS(errctx, akbasic_symtab_set(&runtime->environment->functions, fndef->name, fndef, 0));
+
+    PASS(errctx, akbasic_parser_new_leaf(parser, &command));
+    PASS(errctx, akbasic_leaf_new_command(command, "GEN", NULL));
+    *dest = command;
+    SUCCEED_RETURN(errctx);
+}
+
+/*
+ * END [GEN]
+ *
+ * Bare END finishes the program, the default command path's job before this
+ * handler existed. `END GEN` is a GEN body's own closing verb, and it is
+ * never scanned as one token -- GEN follows END as an ordinary COMMAND token
+ * on the same line -- so this is what tells the two apart and builds the
+ * compound leaf akbasic_cmd_end_gen dispatches on, the same trick
+ * akbasic_parse_print() uses for `PRINT #`.
+ */
+akerr_ErrorContext *akbasic_parse_end(akbasic_Parser *parser, akbasic_ASTLeaf **dest)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_ASTLeaf *expr = NULL;
+    akbasic_ASTLeaf *right = NULL;
+    akbasic_Token *peeked = NULL;
+    int cmp = 0;
+
+    peeked = akbasic_parser_peek(parser);
+    if ( peeked != NULL && peeked->tokentype == AKBASIC_TOK_COMMAND ) {
+	PASS(errctx, aksl_strcmp(peeked->lexeme, "GEN", &cmp));
+	if ( cmp == 0 ) {
+	    (void)akbasic_parser_match1(parser, AKBASIC_TOK_COMMAND);
+	    PASS(errctx, akbasic_parser_new_leaf(parser, &expr));
+	    PASS(errctx, akbasic_leaf_new_command(expr, "END GEN", NULL));
+	    *dest = expr;
+	    SUCCEED_RETURN(errctx);
+	}
+    }
+
+    /* A plain END, matching what the default command path used to do. */
+    if ( peeked != NULL && peeked->tokentype != AKBASIC_TOK_UNDEFINED &&
+	 peeked->tokentype != AKBASIC_TOK_COLON ) {
+	PASS(errctx, akbasic_parser_expression(parser, &right));
+    }
+    PASS(errctx, akbasic_parser_new_leaf(parser, &expr));
+    PASS(errctx, akbasic_leaf_new_command(expr, "END", right));
+    *dest = expr;
+    SUCCEED_RETURN(errctx);
+}
+
+/*
+ * EACH <variable> IN <generator call>
+ *
+ * Shared by akbasic_parse_for() and akbasic_parse_do(), both of which consume
+ * the leading EACH themselves (it is what tells them this is an EACH loop
+ * rather than their ordinary form) before calling this for the rest.
+ */
+static akerr_ErrorContext *parse_each_clause(akbasic_Parser *parser, akbasic_ASTLeaf **var, akbasic_ASTLeaf **callexpr)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_Token *word = NULL;
+    int cmp = 0;
+
+    PASS(errctx, akbasic_parser_expression(parser, var));
+    FAIL_ZERO_RETURN(errctx, (*var != NULL && akbasic_leaf_is_identifier(*var)), AKBASIC_ERR_SYNTAX,
+		     "Expected EACH (variable) IN (generator call)");
+
+    FAIL_ZERO_RETURN(errctx, akbasic_parser_match1(parser, AKBASIC_TOK_COMMAND), AKBASIC_ERR_SYNTAX,
+		     "Expected IN after EACH (variable)");
+    PASS(errctx, akbasic_parser_previous(parser, &word));
+    PASS(errctx, aksl_strcmp(word->lexeme, "IN", &cmp));
+    FAIL_NONZERO_RETURN(errctx, cmp, AKBASIC_ERR_SYNTAX, "Expected IN after EACH (variable)");
+
+    PASS(errctx, akbasic_parser_expression(parser, callexpr));
+    FAIL_ZERO_RETURN(errctx, (*callexpr != NULL && (*callexpr)->leaftype == AKBASIC_LEAF_FUNCTION),
+		     AKBASIC_ERR_SYNTAX, "Expected a generator call after IN");
+    SUCCEED_RETURN(errctx);
+}
+
 /*
  * FOR     ...        TO ....        [STEP    ...]
  * COMMAND ASSIGNMENT    EXPRESSION  [COMMAND EXPRESSION]
@@ -898,11 +1066,57 @@ akerr_ErrorContext *akbasic_parse_for(akbasic_Parser *parser, akbasic_ASTLeaf **
     akbasic_ASTLeaf *assignment = NULL;
     akbasic_ASTLeaf *expr = NULL;
     akbasic_Token *operator_ = NULL;
+    akbasic_Token *peeked = NULL;
     akbasic_Environment *parent = runtime->environment;
     akbasic_Environment *newenv = NULL;
     int64_t firstline = 0;
     int cmp = 0;
 
+    /*
+     * FOR EACH <variable> IN <generator call>. Checked before the leaf right of
+     * FOR is required to be an assignment, because EACH is the one other thing
+     * that leaf is allowed to be.
+     */
+    peeked = akbasic_parser_peek(parser);
+    if ( peeked != NULL && peeked->tokentype == AKBASIC_TOK_COMMAND ) {
+	PASS(errctx, aksl_strcmp(peeked->lexeme, "EACH", &cmp));
+	if ( cmp == 0 ) {
+	    akbasic_ASTLeaf *var = NULL;
+	    akbasic_ASTLeaf *callexpr = NULL;
+
+	    (void)akbasic_parser_match1(parser, AKBASIC_TOK_COMMAND);
+	    firstline = parent->lineno + 1;
+
+	    /*
+	     * Pushed the same way a plain FOR's environment is: while the
+	     * parent is still active, so the call expression's identifiers
+	     * resolve in the caller's scope rather than the loop's own.
+	     */
+	    PASS(errctx, akbasic_runtime_new_environment(runtime));
+	    newenv = runtime->environment;
+	    runtime->environment = parent;
+
+	    PASS(errctx, parse_each_clause(parser, &var, &callexpr));
+
+	    newenv->isEachLoop = true;
+	    newenv->loopFirstLine = firstline;
+	    /*
+	     * Stashed on forToLeaf rather than cloned into a leaf pool: unlike
+	     * DO's condition, this expression is evaluated exactly once, by
+	     * akbasic_cmd_for() on this same pass before the per-line leaf
+	     * storage it lives in is reused -- see akbasic_runtime_generator_invoke().
+	     */
+	    newenv->forToLeaf = callexpr;
+
+	    PASS(errctx, akbasic_parser_new_leaf(parser, &expr));
+	    PASS(errctx, akbasic_leaf_new_command(expr, "FOR", var));
+
+	    runtime->environment = newenv;
+	    *dest = expr;
+	    SUCCEED_RETURN(errctx);
+	}
+    }
+
     PASS(errctx, akbasic_parser_assignment(parser, &assignment));
     FAIL_ZERO_RETURN(errctx, akbasic_parser_match1(parser, AKBASIC_TOK_COMMAND),
 		     AKBASIC_ERR_SYNTAX,