Implement generators: GEN, EMIT, END GEN, FOR EACH and DO EACH (issue 57)
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m32s
akbasic CI Build / coverage (push) Successful in 4m7s
akbasic CI Build / sanitizers (push) Successful in 4m42s
akbasic CI Build / akgl_build (push) Successful in 8m12s
akbasic CI Build / mutation_test (push) Successful in 23m3s

Adds generator support per the plan in issue 57:

- environment.h: isGenerator/generatorFn on a GEN call's own environment,
  isEachLoop and forGeneratorEnv on a FOR EACH/DO EACH loop's own
  environment.
- runtime.c: splits akbasic_runtime_prev_environment() into
  akbasic_runtime_detach_environment() (return to parent without releasing)
  and akbasic_runtime_release_environment() (give variables and the pool
  slot back, on any environment); prev_environment() is now the two in
  sequence. akbasic_runtime_call_function() refuses to call a GEN like an
  ordinary function.
- verbs.c/verbs.h/scanner: new keywords GEN, EMIT, EACH, IN and the compound
  verb "END GEN" (built by a new akbasic_parse_end(), the same trick
  akbasic_parse_print() uses for PRINT #).
- parser_commands.c: akbasic_parse_gen() (modelled on multi-line DEF),
  akbasic_parse_end(), and EACH branches in akbasic_parse_for()/
  akbasic_parse_do().
- runtime_generator.c (new): akbasic_cmd_gen, akbasic_cmd_emit,
  akbasic_cmd_end_gen, and the invoke/pump/release machinery FOR EACH, DO
  EACH, NEXT and LOOP share. EMIT walks up to the nearest isGenerator
  ancestor rather than assuming it is standing directly in the GEN's own
  call frame, because a GEN body may nest its own FOR/DO/GOSUB around an
  EMIT -- the issue's own ROOMOBJECTS example does exactly that.
- runtime_commands.c/runtime_structure.c: EACH branches in cmd_for/cmd_do,
  matching EACH branches in cmd_next/cmd_loop, and forGeneratorEnv release
  on every path that can abandon a live generator (EXIT, a NEXT that pops
  for a mismatched loop variable).

Deviates from the plan in one place: FunctionDef gained an isGenerator flag
(not in the plan's field list) because refusing a GEN called like a
function has to happen before anything is pushed. Relying on EMIT's own
isGenerator check for that case doesn't work: akbasic_runtime_call_function()
drives its own step loop the same way akbasic_runtime_pump_generator() does,
and a BASIC-level error inside that loop is swallowed by process_line_run()
as reported-but-not-propagated, so the call would silently "succeed" with a
meaningless return value instead of failing.

Also: a zero-argument parameter list is not supported by the DEF/GEN
parameter parser this reuses (a pre-existing limitation, not
generator-specific); every generator in the tests takes at least one
parameter as a result.

Tests: tests/generators.c (pool exhaustion under repeated EXIT, calling a
GEN like a function, EMIT outside a GEN, self-recursion, sibling/nested
invocations) and tests/language/flowcontrol/generators_*.bas -- the
issue's own ROOMOBJECTS example in both loop shapes, an empty generator,
non-numeric EMIT, nested/interleaved invocations, and three error-path
golden cases. Docs: control-flow chapter 4 gets a GEN/EMIT/FOR EACH/DO EACH
section, the verb reference gets GEN/EMIT/END GEN entries and updated
FOR/DO/NEXT/LOOP/EXIT rows, and architecture chapter 14 documents the
detach/release split and the two-environment generator invocation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-06 10:02:43 -04:00
parent 11e4e96daf
commit f7e8d4b82b
32 changed files with 1322 additions and 23 deletions

View File

@@ -146,6 +146,89 @@ condition at all loops forever until an `EXIT` or a `GOTO` leaves it.
`EXIT` works here too.
## GEN ... EMIT and FOR EACH / DO EACH
A `GEN` is a subroutine that hands back more than one value, one at a time, instead of
returning once. It looks like a multi-line `DEF`, except its body runs `EMIT` where a
function would `RETURN`, and it ends in `END GEN` rather than `RETURN`:
```basic
10 GEN COUNTUP(N#)
20 FOR I# = 1 TO N#
30 EMIT I#
40 NEXT I#
50 END GEN
60 FOR EACH V# IN COUNTUP(3)
70 PRINT V#
80 NEXT V#
```
```output
1
2
3
```
`FOR EACH` is what runs a `GEN`: it calls `COUNTUP(3)` the way `FOR EACH V# IN` says,
and each `EMIT` inside the generator's body becomes one trip through the loop, with `V#`
holding whatever was emitted. When the generator's body reaches `END GEN` with nothing
left to emit, the loop ends -- there is no separate "no more values" check to write.
`DO EACH ... LOOP` does the same thing:
```basic
10 GEN COUNTUP(N#)
20 FOR I# = 1 TO N#
30 EMIT I#
40 NEXT I#
50 END GEN
60 DO EACH V# IN COUNTUP(3)
70 PRINT V#
80 LOOP
```
```output
1
2
3
```
A `GEN`'s body is ordinary BASIC: it may hold its own `FOR`, `DO`, `IF` or `GOSUB`
around the `EMIT`s, and even invoke another `GEN` with its own `FOR EACH`/`DO EACH` --
the same generator invoked with different arguments, nested or side by side, is not
recursion. A `GEN` invoking *itself* from within its own currently-running body is
refused, the same way it would be an error to call a function that has not returned yet
by name from inside its own body without meaning to recurse.
`EXIT` leaves a `FOR EACH`/`DO EACH` loop early, exactly as it does a plain `FOR` or
`DO`, and the generator it was consuming stops there -- nothing forces the rest of it to
run just because the loop started it:
```basic
10 GEN COUNTUP(N#)
20 FOR I# = 1 TO N#
30 EMIT I#
40 NEXT I#
50 END GEN
60 FOR EACH V# IN COUNTUP(100)
70 PRINT V#
80 IF V# = 3 THEN EXIT
90 NEXT V#
100 PRINT "STOPPED"
```
```output
1
2
3
STOPPED
```
A `GEN` shares its namespace with `DEF` -- the same name cannot be both -- but it is not
a function and cannot be called like one: `X# = COUNTUP(3)` is refused, because nothing
about an ordinary call means "resume where the last `EMIT` left off." `FOR EACH`/`DO
EACH` is the only thing that consumes a `GEN`.
## GOTO and GOSUB
```basic

View File

@@ -37,17 +37,20 @@ for the reasoning in each case.
| `DIM``AS` | `DIM S@ AS T`, `DIM P@ AS PTR TO T` | Make a structure, or a strict pointer to one. See Chapter 16. |
| `DIRECTORY` | `DIRECTORY` | **Refused.** Not written yet; the standard-library wrapper it waited on has landed. |
| `DLOAD` | `DLOAD "name"` | Load a program from a file. |
| `DO` | `DO [WHILE c | UNTIL c]` | Start a loop. The condition may be here, on the `LOOP`, or neither. |
| `DO` | `DO [WHILE c | UNTIL c]`, `DO EACH V IN gen(args)` | Start a loop. The condition may be here, on the `LOOP`, or neither. `EACH` consumes a `GEN` instead; see Chapter 4. |
| `DOPEN` | `DOPEN n, "name" [,W]` | Open a file on channel `n`. `W` opens it for writing. |
| `DRAW` | `DRAW src, x, y [TO x, y ...]` | Plot a point or draw a polyline. |
| `DSAVE` | `DSAVE "name"` | Save the program to a file. |
| `DVERIFY` | `DVERIFY "name"` | The other name for `VERIFY`. |
| `EMIT` | `EMIT expr` | Yield one value from a `GEN` body. Only valid inside one; see Chapter 4. |
| `END` | `END` | Stop the program. Does not arm `CONT`. |
| `END GEN` | `END GEN` | Close a `GEN` body, the way `RETURN` closes a multi-line `DEF`. |
| `ENVELOPE` | `ENVELOPE n, a, d, s, r` | Define one of `PLAY`'s ten envelope presets. |
| `EXIT` | `EXIT` | Leave the innermost `FOR` or `DO` loop. |
| `EXIT` | `EXIT` | Leave the innermost `FOR`, `FOR EACH`, `DO` or `DO EACH` loop. |
| `FETCH` | `FETCH count, from, to` | Copy bytes. The same as `STASH`; there is no expansion RAM. |
| `FILTER` | `FILTER ...` | **Refused.** There is no filter stage in the audio backend. |
| `FOR` | `FOR V = a TO b [STEP c]` | Start a counted loop, ended by `NEXT`. |
| `FOR` | `FOR V = a TO b [STEP c]`, `FOR EACH V IN gen(args)` | Start a counted loop, ended by `NEXT`. `EACH` consumes a `GEN` instead; see Chapter 4. |
| `GEN` | `GEN NAME(args) ... END GEN` | Define a generator: a subroutine that yields more than once via `EMIT`, consumed by `FOR EACH`/`DO EACH`. See Chapter 4. |
| `GET` | `GET V` | Take a keystroke if one is waiting, without stopping. |
| `GETKEY` | `GETKEY V` | Wait for a keystroke, holding the program but not the host. |
| `GETMENU` | `GETMENU n, V%` | Wait for a menu choice, holding the program but not the host. Assigns the entry number. See Chapter 19. |
@@ -67,11 +70,11 @@ for the reasoning in each case.
| `LIST` | `LIST [n][-n]` | List the program, or part of it. |
| `LOAD` | `LOAD "name"` | The other name for `DLOAD`. |
| `LOCATE` | `LOCATE x, y` | Move the pixel cursor. |
| `LOOP` | `LOOP [WHILE c | UNTIL c]` | End a `DO` loop. |
| `LOOP` | `LOOP [WHILE c | UNTIL c]` | End a `DO` loop, including a `DO EACH`. |
| `MENU` | `MENU [n [,"item", ...]]` | Show a menu the player picks from. No entries retires it; no arguments retire them all. See Chapter 19. |
| `MOVSPR` | `MOVSPR n, ...` | Move a sprite. Four forms; see Chapter 8. |
| `NEW` | `NEW` | Erase the program and every variable. |
| `NEXT` | `NEXT V` | End a `FOR` loop and advance its counter. |
| `NEXT` | `NEXT V` | End a `FOR` loop and advance its counter, or resume a `FOR EACH` for its next value. |
| `ON` | `ON e GOTO|GOSUB t [,...]` | Branch to the `e`th target, counting from one. |
| `PAINT` | `PAINT src, x, y` | Flood-fill the region containing a point. |
| `PLAY` | `PLAY "notes"` | Queue notes. Does not block. |

View File

@@ -407,6 +407,8 @@ block structure is executing: the `FOR` bounds and step, the `DO`/`LOOP` conditi
| `GOSUB` | `RETURN` |
| A call to a multi-line user function | that function's `RETURN` |
| An interrupt firing | the handler's `RETURN` |
| `FOR EACH`/`DO EACH` — the loop's own scope, during parsing | the `NEXT`/`LOOP` that finds the generator exhausted, or an `EXIT` |
| A `FOR EACH`/`DO EACH` invoking a `GEN` | `END GEN` reached for real, or the loop's own abandonment |
That `FOR` entry is not a typo. `akbasic_parse_for()` pushes the new environment while
parsing the line, parks `TO` and `STEP` in it as unevaluated leaves, and makes it active
@@ -454,6 +456,59 @@ Three consequences follow, and all three are things people report as bugs:
the value correctly inside the loop and gets `0` immediately after it, with nothing
raised anywhere.
### Generators: a scope that outlives the verb that pushed it
Everything above pops a scope by *releasing* it — `akbasic_runtime_prev_environment()`
gives its variables and its own pool slot back in the same motion that hands control to
its parent. A `GEN` needed a third option, because `EMIT` has to survive being
"returned" from: the next value comes from resuming exactly where the last `EMIT` left
off, not from starting over.
`akbasic_runtime_prev_environment()` is now built from two smaller pieces:
- `akbasic_runtime_detach_environment()` — moves `obj->environment` to the parent,
*without* releasing anything.
- `akbasic_runtime_release_environment()` — gives a scope's variables and pool slot
back, callable on a scope that is not necessarily the active one.
A `FOR EACH`/`DO EACH` invocation therefore holds **two** environments at once, for as
long as the loop is running:
```text
loop environment (isEachLoop) <- pushed like a plain FOR/DO's, at parse time
forGeneratorEnv -----------> generator environment (isGenerator)
<- pushed once, by akbasic_runtime_generator_invoke(),
and never released until the generator is
exhausted or abandoned
```
`EMIT` finds its generator environment by walking *up* from wherever it is actually
standing — a `GEN` body is ordinary BASIC and may nest its own `FOR`, `DO` or `GOSUB`
around an `EMIT`, each pushing scopes of its own — to the nearest ancestor with
`isGenerator` set. It assigns the emitted value into the loop's `forNextVariable`,
records *exactly* where it is standing (which may be several environments below the
generator's own call frame) as `forGeneratorEnv`, and moves `obj->environment` straight
to the loop environment — detaching, not popping, so every environment between the two
survives untouched.
`NEXT`/`LOOP` reactivate a suspended generator by setting `obj->environment` back to
`forGeneratorEnv` and driving the step loop (`akbasic_runtime_pump_generator()`) until
either another `EMIT` detaches it again or `END GEN` is reached for real — meaning
`isGenerator` is set and nothing is skipping forward to it, exactly the same test
`RETURN` makes for a multi-line `DEF`. Real exhaustion releases the generator
environment (`akbasic_runtime_prev_environment()`, same as anything else that pops) and
clears `forGeneratorEnv`, which is what tells the loop apart from one still waiting to
resume.
**Abandoning a live generator has to release it explicitly.** `EXIT` out of a `FOR
EACH`/`DO EACH` pops the loop environment the same way it always has, but a generator
paused mid-run is not on that direct parent chain from the loop back to the root — it
hangs off `forGeneratorEnv` instead, possibly several environments deep if `EMIT` last
ran inside a nested `FOR`/`DO` in the `GEN`'s own body. `akbasic_runtime_release_generator()`
is what walks that chain and releases all of it; every place that pops a `FOR EACH`/`DO
EACH` loop out from under a live generator calls it first, or the pool leaks one
generator at a time.
## Values
`akbasic_Value` carries its string **inline**, not behind a pointer, so a copy is a struct