docs_examples proves every listing in docs/ compiles and still matches the file it was quoted from. It cannot prove a chapter teaches anything -- a document can be made entirely of verified excerpts and still be unfollowable, because what a reader needs is the glue between them. So AGENTS.md gains "Testing a Tutorial": hand the chapter to a subagent on a weaker model with the library, its headers and the art but not the finished program, and make them build and run the game. The weaker model is the point; it will not paper over a gap with knowledge the document did not give it. The section carries the rules that make the result mean something -- copy the tree with examples/ and docs/ removed rather than asking them not to look, let them read the public headers because a real reader has them, do not make them hand-author a .tmj, give them a screenshot helper, and verify their binary yourself -- plus the sandbox recipe, and the warning to prove the build path works before handing it over. It also records how to read the report, because a weaker model states its own mistakes as documentation defects with total confidence. Reproduce before believing; and a rejected finding usually still leaves something behind. The evidence for doing any of this is in the table at the end: four read-only reviews of chapters 20 and 21 found real gaps and missed all three of the defects the first build-and-run found, including a published example output the chapter could not produce. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KzBDV2fqgnUAcqCKqKvc71
272 lines
14 KiB
Markdown
272 lines
14 KiB
Markdown
# Maintaining the manual
|
|
|
|
This file documents the harness that keeps `docs/` honest. It is not a chapter —
|
|
a reader learning libakgl never needs it — and it is checked by the same harness
|
|
it describes, so the file that publishes the convention is held to it.
|
|
|
|
**The harness checks that every listing is true. It cannot check that a chapter
|
|
teaches.** For that, the tutorials get a second test: the chapter is handed to a
|
|
subagent on a weaker model with the library, the headers and the art but *not*
|
|
the finished program, and that reader has to build and run the game. See
|
|
`AGENTS.md`, "Testing a Tutorial". Four read-only reviews of chapters 20 and 21
|
|
missed three defects the first build-and-run found in minutes.
|
|
|
|
## Documentation examples
|
|
|
|
**Every fenced block in `docs/*.md` is compiled, linked, run, or matched against
|
|
a source file by `ctest`.** The test is `docs_examples`; it runs in an ordinary
|
|
build and it fails the build.
|
|
|
|
It exists because the documentation this manual replaced had not been executed
|
|
since the day it was typed. In four hundred lines of `README.md`: two `PASS()`
|
|
calls with unbalanced parentheses, a `sprite->frameids = [0, 1, 2, 3];` that is
|
|
not C in any dialect, a stray `9` inside a bitmask expression, an
|
|
`int screenwidth = NULL`, and — in the first snippet a reader ever saw — the
|
|
exact `strncpy` call `AGENTS.md` forbids under **Copying Into Fixed-Width
|
|
Fields**. Two prose claims in the headers were false against `src/` as well.
|
|
That is not a reader routing around typos; it is what happens to samples nothing
|
|
runs.
|
|
|
|
Documentation goes stale because the **code** moved, not because somebody edited
|
|
a chapter. So this is an ordinary test case rather than something a docs-only CI
|
|
job runs behind a path filter — it fails when the library changes under the
|
|
chapter, which is precisely the case a path filter would miss.
|
|
|
|
**An untagged block is a failure**, not a default, and so is a tag this harness
|
|
does not know. That is deliberate: the way a harness like this dies is by
|
|
quietly matching nothing and passing, so an untagged block is a missing decision
|
|
rather than a free pass.
|
|
|
|
### The tags
|
|
|
|
| Info string | What happens |
|
|
|---|---|
|
|
| `c` | A whole translation unit. Compiled with `-fsyntax-only -std=gnu99 -Wall -Werror` against the real include path. It shows its own `#include` lines |
|
|
| `c wrap=NAME` | The same, with `tests/docs_preludes/NAME.pre` before it and `NAME.post` after |
|
|
| `c run=NAME` | Wrapped in `NAME`, **linked against `akgl` and run headless**. Must exit 0 and must not report an unhandled or ignored error |
|
|
| `c excerpt=PATH` | Must still appear in `PATH`, ignoring comments and whitespace. **Not compiled** |
|
|
| `json excerpt=PATH` | The same check against an asset file, for a fragment too large to show whole |
|
|
| `c screenshot=NAME` | Compiled as an `akglframe` body, and the source of `docs/images/NAME.png`. Optionally `size=WxH` |
|
|
| `json kind=KIND` | Written to a file and loaded through libakgl's own loader. `KIND` is `sprite`, `character`, `tilemap` or `properties` |
|
|
| `output` | The **exact stdout** of the runnable block above it, compared byte for byte |
|
|
| `sh` | Run in a sandbox. Must exit 0. A leading `$ ` is stripped |
|
|
| `sh norun` | Shown, not run. Destructive, networked, or it re-enters this suite |
|
|
| `cmake` | **Never executed.** Hand-maintained, by decision |
|
|
| `text` | **Never executed.** A block diagram, or a stack trace captured from a real run |
|
|
| `norun` | Shown, never executed, whatever it is |
|
|
|
|
These attributes combine with several of the above:
|
|
|
|
| Attribute | What it does |
|
|
|---|---|
|
|
| `setup=NAME` | Runs `tests/docs_setups/NAME.sh` in the sandbox first, with the repository root as its only argument. Valid on `c run=`, `json`, `sh` and `c screenshot=` |
|
|
| `size=WxH` | A figure's surface size. Only on `c screenshot=`; the default is 320x240 |
|
|
| `name=NAME` | Gives a `json` block a name a later block in the same chapter can preload |
|
|
| `preload=A,B` | Loads those named `json` blocks, in that order, before this one |
|
|
|
|
### Which one to reach for
|
|
|
|
In this order, and a step down the list is a decision that belongs in the prose:
|
|
|
|
- **`c run=akglapp` with an `output` block below it.** The strongest shape there
|
|
is. `-fsyntax-only` proves a call typechecks; it does not prove the startup
|
|
order works, that the sprite loaded, or that the `ATTEMPT` block gave back
|
|
what it took. A `run` block proves all three, and the `output` block pins what
|
|
the reader will actually see.
|
|
- **`c run=akglapp` alone**, where the example has no output. The harness still
|
|
asserts a clean exit and that nothing raised.
|
|
- **`c wrap=akglbody`**, for a call sequence whose effect is not visible on
|
|
stdout.
|
|
- **`c`**, for a translation unit the chapter wants shown whole — a header, a
|
|
backend definition, a `main`.
|
|
- **`c excerpt=PATH`**, for a constant table, a `typedef` or a declaration.
|
|
Compiling one of those would only redefine the type; the question worth asking
|
|
is whether it still says what the header says.
|
|
- **`json kind=`**, for any asset file.
|
|
- **`norun`**, for a fragment or something that cannot be run here. Say why in
|
|
the prose. **A rising `norun` count is a review finding** — it is the harness
|
|
being talked out of its job.
|
|
|
|
### Preludes
|
|
|
|
`tests/docs_preludes/NAME.pre` and `NAME.post` bracket the block. They exist so
|
|
an example can be written the way a reader wants to read it while still
|
|
compiling — four lines of `PREPARE_ERROR` scaffolding repeated in twenty
|
|
chapters teaches the protocol twenty times and the call once.
|
|
|
|
| Prelude | What the block supplies |
|
|
|---|---|
|
|
| `akglfile` | File scope: declarations, a `movementlogicfunc`, a backend vtable. Includes only; there is no `.post` |
|
|
| `akglbody` | The inside of a function returning an error context, with `errctx` already prepared |
|
|
| `akglapp` | The same body, plus a `main` that brings libakgl up headless and runs it. This is what `run=` uses |
|
|
| `akglframe` | One frame of drawing, for a figure. Linked against `tools/docs_screenshot.c` |
|
|
|
|
All four begin with `tests/docs_preludes/docs_prelude.h`, which is the include
|
|
set — every public header, since the `headers` suite already proves each one is
|
|
self-contained. A block with no `wrap=` gets none of it and shows its own
|
|
includes, which is usually what a chapter wants a reader to see.
|
|
|
|
What does **not** belong in a prelude is anything that would let a wrong example
|
|
compile. A prelude declaring `akgl_game_init` itself would defeat the entire
|
|
check.
|
|
|
|
Compiler diagnostics point at the markdown: the harness emits a `#line`
|
|
directive before each body, so a broken example reports as
|
|
`docs/14-physics.md:112: error: too few arguments`, not as a line in a scratch
|
|
file nobody can open.
|
|
|
|
### Why `-Werror` here and not on the library
|
|
|
|
`AGENTS.md` keeps `AKGL_WERROR` **off** by default, and for a good reason:
|
|
libakgl is consumed with `add_subdirectory()` — `akbasic` does exactly that — so
|
|
a target-level `-Werror` turns every diagnostic a newer compiler invents into a
|
|
broken build for somebody else's project.
|
|
|
|
A documentation snippet is not a consumer. A sample that warns is a sample that
|
|
teaches the warning, and it is the maintainer of this repository who sees the
|
|
failure. So the snippets get `-Werror` and the library does not.
|
|
|
|
`gnu99` rather than `c99`, because `akerror.h` uses `PATH_MAX`, which
|
|
`<limits.h>` hides under `__STRICT_ANSI__`. A stricter dialect would fail on the
|
|
dependency rather than on the example.
|
|
|
|
### Asset formats
|
|
|
|
A `json kind=` block is written to a file and handed to `akgl_docs_checkjson`,
|
|
which brings libakgl up headless and calls the loader that really reads that
|
|
format — `akgl_sprite_load_json`, `akgl_character_load_json`,
|
|
`akgl_tilemap_load` or `akgl_registry_load_properties`.
|
|
|
|
This exists because the formats are described in prose here and parsed in
|
|
`src/`, with nothing tying the two together. That gap is not hypothetical:
|
|
`util/assets/littleguy.json` ships beside the loader it is invalid against.
|
|
|
|
The loaders really open the images and tilesets a block names, so a block that
|
|
references one needs a `setup=`. Adding a format is a row in the `KINDS` table
|
|
in `tools/docs_checkjson.c`, not a branch.
|
|
|
|
**Three of the four formats are not self-contained.** A character names sprites
|
|
that have to be in `AKGL_REGISTRY_SPRITE` already; a tilemap names characters,
|
|
and really opens its tileset image. There are two ways to satisfy that, and
|
|
which one to use depends on whether the dependency is something the reader
|
|
should see.
|
|
|
|
**Show it, with `name=` and `preload=`.** A `json kind=sprite name=herostand`
|
|
block earlier in a chapter and a `json kind=character preload=herostand` block
|
|
later in the same one are loaded into a single process, in that order. The
|
|
dependency ends up in front of the reader — which is right when the chapter is
|
|
teaching how the two files relate. Names are per-document, so one chapter cannot
|
|
reach into another and make the order documents are checked in load-bearing.
|
|
|
|
**Hide it, with `setup=`.** When the chapter is about one file and the others
|
|
would be noise, the fixtures go in a setup script and the `prepare` column of
|
|
the `KINDS` table in `tools/docs_checkjson.c` loads them:
|
|
|
|
| Setup | What it stages | What loads it |
|
|
|---|---|---|
|
|
| `setup=spritesheet` | `./spritesheet.png`, a 12x8 grid of 48x48 frames | the `sprite` row's `prepare` |
|
|
| `setup=character` | `./sprites/*.json` plus a sheet, named `hero standing left` and `hero walking left` | the `character` row's `prepare` |
|
|
| `setup=tilemap` | `./sprites/*.json`, `./characters/testcharacter.json`, and `assets/tileset.png` | the `tilemap` row's `prepare` |
|
|
| `setup=sidescroller` | every PNG and sprite file from `docs/tutorials/assets/sidescroller`, in `./` and `./sprites` | both rows' `prepare` |
|
|
| `setup=jrpg` | the same, from `docs/tutorials/assets/jrpg` | both rows' `prepare` |
|
|
|
|
The last two are how the tutorials show a reader a *real* asset file rather than a
|
|
generic one. A beginner following chapter 20 or 21 types what is on the page, so
|
|
what is on the page has to be a file that loads — the setup stages the tutorial's
|
|
own art so the block on the page is the block in the repository.
|
|
|
|
Those names are a contract between the setup script and the chapter using it.
|
|
Change one and change the other, in the same commit. `assets/tileset.png` is
|
|
deliberately not the RPG-Maker-named image `tests/assets/testmap.tmj` uses — see
|
|
the comment in `tests/docs_setups/tilemap.sh`.
|
|
|
|
Two things about map fixtures, both confirmed against `src/tilemap.c` rather
|
|
than against the prose:
|
|
|
|
- **Tilesets must be embedded.** `"source"` appears nowhere in the loader;
|
|
`akgl_tilemap_load_tilesets_each` reads `columns`, `firstgid`, `tilecount` and
|
|
`image` straight off each element of the map's own `tilesets` array. The
|
|
external reference Tiled writes by default does not load.
|
|
- **Every object in an object group needs a `type` string**, plain rectangles
|
|
included. A missing one fails the whole load.
|
|
|
|
### Figures are output, not assets
|
|
|
|
A chapter about drawing wants a picture of what the drawing calls draw, and the
|
|
way a picture goes wrong is that it stops being of the code beside it —
|
|
silently, because a hand-taken screenshot still looks like a screenshot long
|
|
after the call changed. So a figure is generated from the listing it
|
|
illustrates:
|
|
|
|
```sh norun
|
|
cmake --build build --target docs_screenshots
|
|
```
|
|
|
|
`tools/docs_screenshot.c` is a small second host: the dummy video driver, a
|
|
software renderer, one call to the block's `docs_frame()`, read the target back,
|
|
write a PNG. `tools/docs_screenshots.sh` compiles the listing against it.
|
|
|
|
Three rules, and `docs/images/README.md` carries the long version:
|
|
|
|
- **The PNGs are tracked.** A reader on the forge has no build tree. They are
|
|
generated files tracked on purpose, so **do not hand-edit one** — change the
|
|
listing and regenerate.
|
|
- **The build never regenerates them.** The target is only ever run
|
|
deliberately.
|
|
- **`docs_examples` fails a tagged block with no image**, so a figure cannot be
|
|
added to a chapter and then forgotten. Whether the image is *of that listing*
|
|
is the separate `docs_screenshots` test's question: it re-renders every figure
|
|
into a scratch directory and compares byte for byte, and it never repairs what
|
|
it finds.
|
|
|
|
### When `docs_examples` fails
|
|
|
|
The message names the file and the line of the block's **opening fence**. For an
|
|
output mismatch it prints both sides through `cat -A`, because the errors it
|
|
catches are trailing spaces and missing newlines, which a plain diff renders
|
|
invisibly.
|
|
|
|
**Fix the documentation, not the expectation** — unless the library is what
|
|
changed, in which case fix the library first and the documentation second. Never
|
|
edit an `output` block to match output you have not looked at.
|
|
|
|
Run one document at a time while you work:
|
|
|
|
```sh norun
|
|
./tests/docs_examples.sh --root . \
|
|
--cflags-file build/docs_cflags.txt \
|
|
--ldflags-file build/docs_ldflags.txt \
|
|
--checkjson build/akgl_docs_checkjson \
|
|
docs/14-physics.md
|
|
```
|
|
|
|
Every path is made absolute by the script before it starts, because it `cd`s
|
|
into sandbox directories. A relative one used to fail every example and every
|
|
`setup=` — invisible under CTest, which passes absolute paths, and immediate for
|
|
anyone running one document by hand.
|
|
|
|
Omit a flag and the blocks that need it are **skipped and counted**, not
|
|
silently passed, so a partial invocation cannot look like a clean run.
|
|
|
|
**Exit status is the number of failed examples**, the house convention; `2` is a
|
|
usage or setup error, which is a different thing and has to be distinguishable.
|
|
A document named on the command line that cannot be read is `2` — never a
|
|
silent fall back to the default list, which is how a suite ends up passing
|
|
having checked nothing.
|
|
|
|
### Read the census line
|
|
|
|
The last two lines of a run report **what it executed, by kind**:
|
|
|
|
```text
|
|
ran: 41 c snippets, 6 linked programs, 6 output comparisons, 18 excerpts, 4 json blocks, 2 shell blocks, 3 figures
|
|
skipped: 5 norun, 2 cmake, 7 text, 0 c (no compiler flags given), 0 run (no linker flags given), 0 json (no validator given)
|
|
```
|
|
|
|
That count is part of the result, not decoration. A harness that passes because
|
|
a chapter stopped matching the extractor looks exactly like a harness that
|
|
passes because the documentation is correct, and the census is the only thing
|
|
that tells them apart. In `akbasic` it caught exactly that: a CMake generator
|
|
expression evaluated to an empty argument, the script read it as a filename, and
|
|
the suite passed having checked no documents at all.
|