Andrew Kesterson
342e4c07da
Some checks failed
akbasic CI Build / cmake_build (push) Successful in 3m2s
akbasic CI Build / sanitizers (push) Successful in 3m52s
akbasic CI Build / coverage (push) Failing after 3m24s
akbasic CI Build / akgl_build (push) Failing after 20s
akbasic CI Build / mutation_test (push) Has been cancelled
docs/ and README.md carry 85 fenced blocks. Every one was checked by hand exactly once, when it was written, which is not a standard that survives a changing interpreter -- and four were already wrong: two transcripts showing a leading space PRINT does not emit, akbasic_TextSink in README.md missing the two members it had grown hours earlier, and FILTER's refusal quoted with wording the code does not use. tests/docs_examples.sh reads a fence-tag vocabulary and runs what it finds. BASIC programs and transcripts run and are byte-compared against an `output` block; C snippets compile with -fsyntax-only against the real include path, which CMake writes out because it is transitive through akerror, akstdlib and akgl; shell blocks run in a sandbox. Anything that would reconfigure the build tree, hit the network or re-enter the suite is tagged norun with the reason in MAINTENANCE.md, and the two cmake blocks stay hand-maintained by decision. An untagged block is a failure rather than a default, and the pass line reports what it executed by kind. Both exist because the way a harness like this dies is by quietly matching nothing and passing -- which it duly did on the first CTest run, where a generator expression evaluating to nothing still contributed an empty argument that the script read as a filename. The count is what caught it. The excerpt check earns its own mention: a block tagged `c excerpt=include/akbasic/sink.h` must still appear in that header, comments and whitespace ignored. Compiling it would only redefine the type, so a compile check could not have found the stale struct, and did not. Registered as the CTest case docs_examples in both configurations. Fixing the four wrong examples turned up two interpreter defects, fixed in the previous commit and recorded in TODO.md section 8. MAINTENANCE.md is new: the fence-tag reference, what to do when the case fails, and the conventions that until now only existed inside source comments -- the three test lists and how two of them invert "passed", the sorted verb table, that a golden file is never edited to suit this interpreter, and that a fix gets mutation-checked with a file copy rather than git checkout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
226 lines
11 KiB
Markdown
226 lines
11 KiB
Markdown
# Maintaining akbasic
|
||
|
||
`README.md` is for people using the interpreter. `docs/` is for people writing BASIC in
|
||
it. `CLAUDE.md` is for agents. This file is for whoever has to change the thing: the
|
||
conventions that are enforced by a test rather than by a comment, and the ones that are
|
||
not enforced at all and therefore have to be written down.
|
||
|
||
---
|
||
|
||
## Editing the documentation
|
||
|
||
**Every fenced block in `README.md` and `docs/*.md` is executed by `ctest`.** The test is
|
||
`docs_examples`; it runs in both build configurations and it fails the build. It caught
|
||
four wrong examples the day it was added — two transcripts showing a leading space
|
||
`PRINT` does not emit, a `struct` in `README.md` that had grown two members hours
|
||
earlier, and a `FILTER` refusal quoted with the wrong wording.
|
||
|
||
An **untagged block is a failure**, not a default. That is deliberate: the way a harness
|
||
like this dies is by quietly matching nothing and passing, so leaving a block untagged is
|
||
a missing decision rather than a free pass.
|
||
|
||
### The tags
|
||
|
||
| Info string | What happens |
|
||
|---|---|
|
||
| `basic` | Written to a file and run. Must exit 0 and print no `? line : CLASS` error |
|
||
| `basic repl` | Fed to a fresh interpreter on **stdin**. The leading `READY` banner is dropped before comparison |
|
||
| `basic norun` | Shown, not run. For fragments and for anything that loops forever |
|
||
| `basic requires=akgl` | Run only in the `-DAKBASIC_WITH_AKGL=ON` build |
|
||
| `basic requires=noakgl` | Run only in the default build. For verbs whose *refusal* is the example |
|
||
| `basic setup=NAME` | Runs `tests/docs_setups/NAME.sh` in the sandbox first |
|
||
| `output` | The **exact stdout** of the block above it, compared byte for byte |
|
||
| `c` | Compiled with `-fsyntax-only -std=gnu99 -Wall -Wextra -Werror` against the real include path |
|
||
| `c wrap=NAME` | The same, with `tests/docs_preludes/NAME.pre` before it and `NAME.post` after |
|
||
| `c excerpt=PATH` | Must appear in `PATH`, ignoring comments and whitespace. Not compiled |
|
||
| `c norun` | Shown, not compiled |
|
||
| `sh` | Run in a sandbox with the built interpreter at `./build/basic`. Must exit 0. A leading `$ ` is stripped |
|
||
| `sh setup=NAME` | The same, after `tests/docs_setups/NAME.sh` |
|
||
| `sh norun` | Shown, not run |
|
||
| `cmake` | **Never executed.** Hand-maintained, by decision |
|
||
|
||
Attributes combine: `basic requires=akgl setup=ship` is a real tag in `docs/08-sprites.md`.
|
||
|
||
### Which one to reach for
|
||
|
||
- **A program with visible output** — `basic` plus an `output` block. Preferred: it is the
|
||
only shape that pins what the reader will actually see.
|
||
- **A program with no output** — `basic` alone. The harness still asserts it parses, runs,
|
||
and raises nothing.
|
||
- **An interactive transcript** — `basic repl`, with the input in one block and the output
|
||
in the next. A block that interleaves the two cannot be checked mechanically, and
|
||
splitting it is not a loss: the piped interpreter prints `READY` once at startup rather
|
||
than after each line, so an interleaved transcript is not literally true anyway.
|
||
- **A verb that draws, plays or moves** — `basic requires=akgl`. These produce no stdout,
|
||
so they get no `output` block; the assertion is that they do not refuse.
|
||
- **A refusal message** — write the program that provokes it and put the message in an
|
||
`output` block. Refusals carry a trailing blank line; the block has to have it too.
|
||
- **`norun`** — for a fragment (`60 DATA 255, 129, ...`), an infinite loop, or a command
|
||
that would reconfigure the tree the suite is running inside. Say why in the prose.
|
||
|
||
### `sh` blocks: what is not run, and why
|
||
|
||
Five of the seven shell blocks are `norun`, and the reason is the same for all of them:
|
||
they operate on the build tree the test is running inside, or they need the network.
|
||
|
||
| Block | Why |
|
||
|---|---|
|
||
| `git submodule update --init --recursive` | Network, and the sandbox is not a repository |
|
||
| `cmake -S . -B build` | Would reconfigure the live build tree mid-test |
|
||
| `ctest --test-dir build` | Re-enters this suite |
|
||
| `doxygen Doxyfile` | Slow, and writes into the build tree |
|
||
| the `mutation` target | Hours |
|
||
|
||
That leaves a real gap: a renamed CMake option in `README.md` would go unnoticed. It could
|
||
be closed later with an `excerpt`-style check against `CMakeLists.txt`. It has not been.
|
||
|
||
### Preludes
|
||
|
||
`tests/docs_preludes/NAME.pre` and `NAME.post` bracket a `c wrap=NAME` block. They exist so
|
||
an example can be written the way a reader wants to read it — `PASS(e, akbasic_runtime_global(...))`
|
||
and nothing else — while still compiling. What belongs in a prelude:
|
||
|
||
- Symbols the example **invents** to stay readable: `your_clock_ms()`, `my_renderer`.
|
||
- Scaffolding the macro protocol requires: a function body, a `PREPARE_ERROR`, an `ATTEMPT`
|
||
for a block of `CATCH` calls.
|
||
- Includes the surrounding prose already listed and the block does not repeat.
|
||
|
||
What does **not** belong in a prelude is anything that would let a wrong example compile.
|
||
A prelude declaring `akbasic_runtime_init` itself, for instance, would defeat the check.
|
||
|
||
Compiler diagnostics point at the markdown: the harness emits a `#line` directive, so a
|
||
broken example reports as `README.md:322: error: too few arguments`.
|
||
|
||
### Excerpts
|
||
|
||
`c excerpt=include/akbasic/sink.h` says the block is a copy of something in that header and
|
||
must still match it, comments and whitespace ignored. Use it where compiling the block
|
||
would be wrong — echoing a `typedef` compiles only by redefining the type. This is the
|
||
check that caught the stale `akbasic_TextSink`, and a compile check could not have.
|
||
|
||
### When `docs_examples` fails
|
||
|
||
The message names the file and the line of the block. For an output mismatch it prints both
|
||
sides through `cat -A`, because the errors it catches are trailing spaces and missing
|
||
newlines, which a plain diff renders invisibly.
|
||
|
||
**Fix the documentation, not the expectation** — unless the interpreter is what changed, in
|
||
which case fix the interpreter first and the documentation second. Never edit an `output`
|
||
block to match output you have not looked at.
|
||
|
||
Run one document at a time while you work:
|
||
|
||
```sh norun
|
||
./tests/docs_examples.sh --root . --basic ./build/basic \
|
||
--cflags-file build/docs_cflags.txt docs/04-control-flow.md
|
||
```
|
||
|
||
The pass line reports **what it executed, by kind**. Read it. A harness that passes because
|
||
it stopped matching anything looks exactly like a harness that passes because the
|
||
documentation is correct, and the count is the only thing that tells them apart — it has
|
||
already caught one such case, where a CMake generator expression evaluated to an empty
|
||
argument that the script read as a filename.
|
||
|
||
---
|
||
|
||
## Tests
|
||
|
||
### Three lists, and two of them invert "passed"
|
||
|
||
`CMakeLists.txt` declares `AKBASIC_TESTS`, `AKBASIC_WILL_FAIL_TESTS` and
|
||
`AKBASIC_KNOWN_FAILING_TESTS`. The first must exit 0. The second aborts by design. The
|
||
third **asserts the correct contract for a defect that is documented in `TODO.md`** and is
|
||
expected to fail.
|
||
|
||
A green `ctest` therefore does not mean defect-free. When a known-failing test starts
|
||
passing, CTest reports "unexpectedly passed" — that is the cue to move it into
|
||
`AKBASIC_TESTS` along with the fix, not to delete it.
|
||
|
||
`AKBASIC_WILL_FAIL_TESTS` is currently empty and is kept declared anyway, so the shape is
|
||
there when it is next needed.
|
||
|
||
### Test target names
|
||
|
||
Every test program builds as `akbasic_test_<name>` while registering under the bare CTest
|
||
name. That is not cosmetic: `add_executable` creates a dependency's targets even under
|
||
`EXCLUDE_FROM_ALL`, and when `libakstdlib` added a `test_version` it collided with
|
||
`libakgl`'s and stopped the configure dead.
|
||
|
||
### The golden corpora
|
||
|
||
`tests/reference/` is the Go implementation's own acceptance suite, byte-compared.
|
||
**Nothing in it is ever edited to suit this interpreter.** If a case fails, either this
|
||
interpreter is wrong or the divergence is deliberate — and a deliberate one goes in
|
||
`TODO.md` and `docs/13-differences.md`, not into the expectation file. `tests/reference/README.md`
|
||
says the same thing at more length.
|
||
|
||
`tests/language/` is ours and may be changed freely.
|
||
|
||
### Mutation-check a fix before you believe it
|
||
|
||
Coverage says a line ran; it does not say anything would have noticed if it were wrong.
|
||
That matters more here than usual, because the `akerror` macros expand at their call sites
|
||
and `gcov` attributes them to the caller.
|
||
|
||
The discipline for any fix with a test: **revert the fix, confirm the test fails, restore
|
||
it.** Use a file copy, not `git checkout` — `git checkout -- src` in a loop like this has
|
||
already wiped a session's worth of unrelated edits.
|
||
|
||
```sh norun
|
||
cmake --build build --target mutation # whole tree; hours
|
||
python3 scripts/mutation_test.py --target src/value.c --threshold 70
|
||
```
|
||
|
||
### Build trees stay out of the source directory
|
||
|
||
`gcovr` searches for `.gcda`/`.gcno` under its `--root`, and the dependencies set that to
|
||
their *source* directory — so a leftover instrumented build tree in the source dir is
|
||
folded into the report and a coverage run fails before any test executes. `build*/` is
|
||
`.gitignore`d, which makes the state easy to reach and hard to see. Always
|
||
`cmake -S . -B build`.
|
||
|
||
---
|
||
|
||
## Code
|
||
|
||
### The verb table is sorted, and a test says so
|
||
|
||
`src/verbs.c` is searched with `bsearch`. A mis-sorted table does not fail to compile — it
|
||
silently fails to find a verb, and the symptom is `Unknown command PRINT` a long way from
|
||
the cause. `tests/verbs_table.c` asserts the ordering; adding a verb in the wrong place
|
||
fails there.
|
||
|
||
### Error codes
|
||
|
||
`akbasic` owns **512–767** under the owner string `"akbasic"`, reserved in
|
||
`akbasic_runtime_init()`. Codes are an `enum` so they stay compile-time integer constants —
|
||
`HANDLE`'s `case` labels require that. Never define one as an offset from another library's
|
||
symbol; the coordinated map across the whole dependency stack is in `CLAUDE.md`.
|
||
|
||
### Nothing in the library terminates the process
|
||
|
||
Goal 3. `FINISH_NORETURN` appears only in a `main()` — today the driver's and the
|
||
examples'. A *script's* error is reported through the sink and swallowed; an *interpreter*
|
||
error propagates out as `akerr_ErrorContext *` for the host.
|
||
|
||
Both halves of that need saying, because the second one is easy to get right and the first
|
||
one is easy to get wrong: `process_line_run()` swallowed its context correctly from the
|
||
start while the direct-mode branch of `process_line_repl()` used a bare `PASS`, so a
|
||
`VERIFY` against a file that did not match — an ordinary user answer — tore down the driver
|
||
with a stack trace.
|
||
|
||
### Generated files
|
||
|
||
Never hand-edit `build/` trees, the generated `akerror.h`, `akgl.pc`, or
|
||
`include/akgl/SDL_GameControllerDB.h`. Change the template or the generator.
|
||
|
||
### Style
|
||
|
||
C99, four-space indent, braces on their own line for function bodies, spaces inside
|
||
control-flow parentheses. **Match the surrounding file** — several mix tabs and spaces and
|
||
there is no repo-wide formatter. Do not reformat code you are not otherwise changing;
|
||
style conversions get their own commit.
|
||
|
||
Public symbols take the `akbasic_` prefix, `akbasic_TypeName` for types, `AKBASIC_` for
|
||
macros. `static` helpers drop it.
|