Implement generators: GEN, EMIT, END GEN, FOR EACH and DO EACH (issue 57)

Adds generator support per the plan in issue 57:

- environment.h: isGenerator/generatorFn on a GEN call's own environment,
  isEachLoop and forGeneratorEnv on a FOR EACH/DO EACH loop's own
  environment.
- runtime.c: splits akbasic_runtime_prev_environment() into
  akbasic_runtime_detach_environment() (return to parent without releasing)
  and akbasic_runtime_release_environment() (give variables and the pool
  slot back, on any environment); prev_environment() is now the two in
  sequence. akbasic_runtime_call_function() refuses to call a GEN like an
  ordinary function.
- verbs.c/verbs.h/scanner: new keywords GEN, EMIT, EACH, IN and the compound
  verb "END GEN" (built by a new akbasic_parse_end(), the same trick
  akbasic_parse_print() uses for PRINT #).
- parser_commands.c: akbasic_parse_gen() (modelled on multi-line DEF),
  akbasic_parse_end(), and EACH branches in akbasic_parse_for()/
  akbasic_parse_do().
- runtime_generator.c (new): akbasic_cmd_gen, akbasic_cmd_emit,
  akbasic_cmd_end_gen, and the invoke/pump/release machinery FOR EACH, DO
  EACH, NEXT and LOOP share. EMIT walks up to the nearest isGenerator
  ancestor rather than assuming it is standing directly in the GEN's own
  call frame, because a GEN body may nest its own FOR/DO/GOSUB around an
  EMIT -- the issue's own ROOMOBJECTS example does exactly that.
- runtime_commands.c/runtime_structure.c: EACH branches in cmd_for/cmd_do,
  matching EACH branches in cmd_next/cmd_loop, and forGeneratorEnv release
  on every path that can abandon a live generator (EXIT, a NEXT that pops
  for a mismatched loop variable).

Deviates from the plan in one place: FunctionDef gained an isGenerator flag
(not in the plan's field list) because refusing a GEN called like a
function has to happen before anything is pushed. Relying on EMIT's own
isGenerator check for that case doesn't work: akbasic_runtime_call_function()
drives its own step loop the same way akbasic_runtime_pump_generator() does,
and a BASIC-level error inside that loop is swallowed by process_line_run()
as reported-but-not-propagated, so the call would silently "succeed" with a
meaningless return value instead of failing.

Also: a zero-argument parameter list is not supported by the DEF/GEN
parameter parser this reuses (a pre-existing limitation, not
generator-specific); every generator in the tests takes at least one
parameter as a result.

Tests: tests/generators.c (pool exhaustion under repeated EXIT, calling a
GEN like a function, EMIT outside a GEN, self-recursion, sibling/nested
invocations) and tests/language/flowcontrol/generators_*.bas -- the
issue's own ROOMOBJECTS example in both loop shapes, an empty generator,
non-numeric EMIT, nested/interleaved invocations, and three error-path
golden cases. Docs: control-flow chapter 4 gets a GEN/EMIT/FOR EACH/DO EACH
section, the verb reference gets GEN/EMIT/END GEN entries and updated
FOR/DO/NEXT/LOOP/EXIT rows, and architecture chapter 14 documents the
detach/release split and the two-environment generator invocation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

src/runtime_generator.c

@@ -0,0 +1,297 @@
+/**
+ * @file runtime_generator.c
+ * @brief GEN, EMIT and END GEN, plus the machinery FOR EACH/DO EACH share.
+ *
+ * A GEN is a DEF that yields more than once. Its body is skipped on the
+ * definitional pass exactly as a multi-line DEF's is -- `akbasic_parse_gen()`
+ * arms `akbasic_environment_wait_for_command(env, "END GEN")` the same way
+ * `akbasic_parse_def()` arms one for `RETURN` -- and it only ever really runs
+ * when a `FOR EACH`/`DO EACH` invokes it.
+ *
+ * That invocation pushes one environment for the whole lifetime of the loop,
+ * exactly as a GOSUB or a function call does, except that `EMIT` does not pop
+ * it: it hands control back to the loop without releasing anything, so the
+ * environment EMIT actually ran in -- which may be nested several levels
+ * below the GEN's own call frame, inside a FOR/DO/GOSUB the body wrote --
+ * still says exactly where to resume when `NEXT`/`LOOP` calls back into
+ * akbasic_runtime_pump_generator(). Only `END GEN` reached for real --
+ * meaning `obj->environment->isGenerator` is true and nothing is skipping
+ * forward to it -- actually releases the call frame, the way `RETURN`
+ * releases a DEF's call environment.
+ */
+
+#include <akerror.h>
+#include <akstdlib.h>
+
+#include <akbasic/error.h>
+#include <akbasic/runtime.h>
+#include <akbasic/scanner.h>
+
+#include "verbs.h"
+
+/* Most verbs answer "did something happen"; this is that answer. */
+#define SUCCEED_TRUE(__obj, __dest)			\
+    do {						\
+	*(__dest) = &(__obj)->staticTrueValue;		\
+    } while ( 0 )
+
+akerr_ErrorContext *akbasic_runtime_pump_generator(akbasic_Runtime *obj, akbasic_Environment *loopenv)
+{
+    PREPARE_ERROR(errctx);
+
+    FAIL_ZERO_RETURN(errctx, (obj != NULL && loopenv != NULL), AKERR_NULLPOINTER,
+		     "NULL argument in pump_generator");
+    /*
+     * The same per-line prologue akbasic_runtime_step() and
+     * akbasic_runtime_call_function() run, driving process_line_run() directly
+     * rather than through the ordinary step loop: a generator body is not the
+     * top-level program and nothing else is going to advance it.
+     */
+    ATTEMPT {
+	while ( obj->environment != loopenv && obj->mode == AKBASIC_MODE_RUN ) {
+	    CATCH(errctx, akbasic_runtime_zero(obj));
+	    CATCH(errctx, akbasic_scanner_zero(obj));
+	    CATCH(errctx, akbasic_runtime_process_line_run(obj));
+	}
+    } CLEANUP {
+	/*
+	 * CLEANUP runs unconditionally -- it is not a `catch` -- so it is
+	 * guarded on the one thing that tells success and failure apart here:
+	 * whether `obj->environment` is still `loopenv`. On the ordinary
+	 * success path it already is (that is the ATTEMPT loop's own exit
+	 * condition), so this is a no-op there, exactly as it is meant to be.
+	 * Only a genuine C-level failure -- the scanner or parser raised, or a
+	 * runtime error escaped the swallow process_line_run() ordinarily does
+	 * for a BASIC-level one -- leaves scopes active between here and
+	 * loopenv, and only then does this force them back, taking
+	 * `forGeneratorEnv` down with them since whatever it pointed at is
+	 * among the scopes just released.
+	 */
+	if ( obj->environment != loopenv ) {
+	    while ( obj->environment != loopenv && obj->environment->parent != NULL ) {
+		IGNORE(akbasic_runtime_prev_environment(obj));
+	    }
+	    loopenv->forGeneratorEnv = NULL;
+	}
+    } PROCESS(errctx) {
+    } FINISH(errctx, true);
+    SUCCEED_RETURN(errctx);
+}
+
+/**
+ * @brief Release a live generator, however deep EMIT left it suspended.
+ *
+ * `loopenv->forGeneratorEnv` is the *resume point*, not necessarily the GEN's
+ * own call frame -- EMIT may have run several levels down, inside a FOR/DO/
+ * GOSUB the body wrote around it. Abandoning it (EXIT) has to give back every
+ * environment from there up through the call frame itself, or everything
+ * above the resume point leaks.
+ *
+ * @param obj Object to initialize, inspect, or modify.
+ * @param env The suspended resume point; walks up through its own parents.
+ * @return `NULL` on success, otherwise an error context owned by the caller.
+ */
+akerr_ErrorContext *akbasic_runtime_release_generator(akbasic_Runtime *obj, akbasic_Environment *env)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_Environment *walk = env;
+    akbasic_Environment *next = NULL;
+    bool isgen = false;
+
+    FAIL_ZERO_RETURN(errctx, (obj != NULL && env != NULL), AKERR_NULLPOINTER,
+		     "NULL argument in release_generator");
+    while ( walk != NULL ) {
+	isgen = walk->isGenerator;
+	next = walk->parent;
+	PASS(errctx, akbasic_runtime_release_environment(obj, walk));
+	if ( isgen ) {
+	    break;
+	}
+	walk = next;
+    }
+    SUCCEED_RETURN(errctx);
+}
+
+akerr_ErrorContext *akbasic_runtime_generator_invoke(akbasic_Runtime *obj, akbasic_Environment *loopenv, akbasic_ASTLeaf *callexpr)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_Environment *callenv = NULL;
+    akbasic_Environment *walk = NULL;
+    akbasic_FunctionDef *fndef = NULL;
+    akbasic_ASTLeaf *fnarg = NULL;
+    akbasic_ASTLeaf *paramleaf = NULL;
+    akbasic_Value *argvals[AKBASIC_MAX_CALL_ARGUMENTS];
+    akbasic_Value *unused = NULL;
+    void *fnptr = NULL;
+    int nargs = 0;
+    int i = 0;
+
+    FAIL_ZERO_RETURN(errctx, (obj != NULL && loopenv != NULL && callexpr != NULL), AKERR_NULLPOINTER,
+		     "NULL argument in generator_invoke");
+    FAIL_ZERO_RETURN(errctx, (callexpr->leaftype == AKBASIC_LEAF_FUNCTION), AKBASIC_ERR_SYNTAX,
+		     "Expected a generator call after IN");
+
+    /*
+     * GEN and DEF share the functions table (TODO.md's namespace decision for
+     * this feature), so this is the same lookup akbasic_runtime_call_function()
+     * does. The parser already proved the name resolves and the arity matches
+     * when it parsed `callexpr` -- akbasic_parser_expression() would not have
+     * produced an AKBASIC_LEAF_FUNCTION leaf otherwise -- so a miss here would
+     * mean the function table changed out from under a leaf built against it,
+     * which is not a case this needs its own message for.
+     */
+    PASS(errctx, akbasic_environment_get_function(loopenv, callexpr->identifier, &fnptr));
+    fndef = (akbasic_FunctionDef *)fnptr;
+    FAIL_ZERO_RETURN(errctx, fndef->isGenerator, AKBASIC_ERR_STATE,
+		     "%s is a DEF, not a GEN -- FOR EACH/DO EACH needs a generator",
+		     fndef->name);
+
+    /*
+     * Self-recursion: walk the *parent* chain, not the pool. An environment
+     * reachable only through some other loop's `forGeneratorEnv` is a sibling
+     * invocation sitting detached between its own iterations, not an ancestor
+     * of this call -- nothing points from here to it via `parent`, so it never
+     * matches and independent or nested FOR EACH/DO EACH over the same GEN
+     * (even the same GEN with different arguments) is unaffected.
+     */
+    for ( walk = loopenv; walk != NULL; walk = walk->parent ) {
+	if ( walk->isGenerator && walk->generatorFn == (void *)fndef ) {
+	    FAIL_RETURN(errctx, AKBASIC_ERR_STATE,
+			"GEN %s cannot FOR EACH/DO EACH over itself from its own body",
+			fndef->name);
+	}
+    }
+
+    /*
+     * Evaluated in the caller's own scope, before anything is pushed -- the
+     * same reason akbasic_runtime_user_function() evaluates every argument
+     * before binding the first one: a later argument must not see an earlier
+     * one already sitting in the callee's scope.
+     */
+    fnarg = akbasic_leaf_first_argument(callexpr);
+    for ( ; fnarg != NULL; fnarg = fnarg->next ) {
+	FAIL_ZERO_RETURN(errctx, (nargs < AKBASIC_MAX_CALL_ARGUMENTS), AKBASIC_ERR_BOUNDS,
+			 "%s was called with more than %d arguments",
+			 callexpr->identifier, AKBASIC_MAX_CALL_ARGUMENTS);
+	PASS(errctx, akbasic_runtime_evaluate(obj, fnarg, &argvals[nargs]));
+	nargs += 1;
+    }
+
+    /*
+     * One environment for the whole lifetime of the loop, exactly as GOSUB and
+     * a function call take one from the pool -- and unlike either, this one
+     * survives past the verb that pushed it, held alive by `loopenv`'s own
+     * reference until NEXT/LOOP exhausts or EXIT abandons it.
+     */
+    PASS(errctx, akbasic_runtime_new_environment(obj));
+    callenv = obj->environment;
+    callenv->isGenerator = true;
+    callenv->generatorFn = (void *)fndef;
+    callenv->nextline = fndef->lineno;
+    loopenv->forGeneratorEnv = callenv;
+
+    paramleaf = (fndef->arglist != NULL ? fndef->arglist->right : NULL);
+    for ( i = 0; i < nargs && paramleaf != NULL; i++ ) {
+	PASS(errctx, akbasic_environment_assign(callenv, paramleaf, argvals[i], &unused));
+	paramleaf = paramleaf->next;
+    }
+
+    PASS(errctx, akbasic_runtime_pump_generator(obj, loopenv));
+    SUCCEED_RETURN(errctx);
+}
+
+akerr_ErrorContext *akbasic_cmd_gen(akbasic_Runtime *obj, akbasic_ASTLeaf *expr, akbasic_Value *lval, akbasic_Value *rval, akbasic_Value **dest)
+{
+    PREPARE_ERROR(errctx);
+
+    (void)expr; (void)lval; (void)rval;
+    /* The parse handler already installed the generator, exactly as DEF's does. */
+    SUCCEED_TRUE(obj, dest);
+    SUCCEED_RETURN(errctx);
+}
+
+akerr_ErrorContext *akbasic_cmd_emit(akbasic_Runtime *obj, akbasic_ASTLeaf *expr, akbasic_Value *lval, akbasic_Value *rval, akbasic_Value **dest)
+{
+    PREPARE_ERROR(errctx);
+    akbasic_Environment *genenv = NULL;
+    akbasic_Environment *loopenv = NULL;
+    akbasic_Value *value = NULL;
+    int64_t zerosubscript[1] = { 0 };
+
+    (void)lval; (void)rval;
+    FAIL_ZERO_RETURN(errctx, (expr != NULL && expr->right != NULL), AKBASIC_ERR_SYNTAX,
+		     "Expected EMIT (expression)");
+    /*
+     * EMIT is not necessarily standing directly in the environment
+     * akbasic_runtime_generator_invoke() pushed: a GEN body is ordinary BASIC
+     * and may nest its own FOR, DO or GOSUB around an EMIT, each of which
+     * pushes an environment of its own -- exactly what the issue's own
+     * ROOMOBJECTS example does. Walk up to the nearest one that really is a
+     * GEN's own call frame.
+     */
+    for ( genenv = obj->environment; genenv != NULL && !genenv->isGenerator; genenv = genenv->parent ) {
+    }
+    FAIL_ZERO_RETURN(errctx, (genenv != NULL), AKBASIC_ERR_STATE,
+		     "EMIT outside the context of a GEN body");
+    loopenv = genenv->parent;
+    FAIL_ZERO_RETURN(errctx, (loopenv != NULL), AKBASIC_ERR_ENVIRONMENT,
+		     "EMIT from an orphaned environment");
+
+    PASS(errctx, akbasic_runtime_evaluate(obj, expr->right, &value));
+    /*
+     * Straight into the loop variable's storage, bypassing the arithmetic
+     * akbasic_environment_assign() and evaluate_for_condition() carry for a
+     * plain FOR: an EACH variable takes whatever type the GEN emits, string or
+     * structure element included, and there is no TO/STEP to compare it
+     * against.
+     */
+    PASS(errctx, akbasic_variable_set_subscript(loopenv->forNextVariable, value, zerosubscript, 1));
+    loopenv->nextline = loopenv->loopFirstLine;
+    /*
+     * The resume point, which may be several levels below `genenv` -- whatever
+     * nested FOR/DO/GOSUB environment this EMIT actually ran in. NEXT/LOOP
+     * reactivates exactly this one, so the nested structure picks up exactly
+     * where it left off rather than restarting at the top of the GEN body.
+     */
+    loopenv->forGeneratorEnv = obj->environment;
+    /*
+     * Not a pop, and not a single-level detach either: everything between here
+     * and `loopenv` -- `genenv` and any of its own descendants -- has to
+     * survive untouched to be resumed, so control moves to `loopenv` directly
+     * rather than walking the chain one release at a time.
+     */
+    obj->environment = loopenv;
+    SUCCEED_TRUE(obj, dest);
+    SUCCEED_RETURN(errctx);
+}
+
+akerr_ErrorContext *akbasic_cmd_end_gen(akbasic_Runtime *obj, akbasic_ASTLeaf *expr, akbasic_Value *lval, akbasic_Value *rval, akbasic_Value **dest)
+{
+    PREPARE_ERROR(errctx);
+    bool waiting = false;
+
+    (void)expr; (void)lval; (void)rval;
+    /*
+     * A END GEN reached while skipping forward to one is the end of a GEN
+     * body's *definition*, not the end of a call -- the same distinction
+     * RETURN draws for DEF.
+     */
+    PASS(errctx, akbasic_environment_is_waiting_for(obj->environment, "END GEN", &waiting));
+    if ( waiting ) {
+	PASS(errctx, akbasic_environment_stop_waiting(obj->environment, "END GEN"));
+	SUCCEED_TRUE(obj, dest);
+	SUCCEED_RETURN(errctx);
+    }
+    FAIL_ZERO_RETURN(errctx, (obj->environment->isGenerator), AKBASIC_ERR_STATE,
+		     "END GEN outside the context of a generator invocation");
+    /*
+     * Real exhaustion: release this environment and detach in the same
+     * motion prev_environment() always does, then clear the parent's
+     * reference to it so a caller pumping this loop can tell "still alive"
+     * apart from "nothing left to resume".
+     */
+    PASS(errctx, akbasic_runtime_prev_environment(obj));
+    obj->environment->forGeneratorEnv = NULL;
+    SUCCEED_TRUE(obj, dest);
+    SUCCEED_RETURN(errctx);
+}