docs_examples proves every listing in docs/ compiles and still matches the file it was quoted from. It cannot prove a chapter teaches anything -- a document can be made entirely of verified excerpts and still be unfollowable, because what a reader needs is the glue between them. So AGENTS.md gains "Testing a Tutorial": hand the chapter to a subagent on a weaker model with the library, its headers and the art but not the finished program, and make them build and run the game. The weaker model is the point; it will not paper over a gap with knowledge the document did not give it. The section carries the rules that make the result mean something -- copy the tree with examples/ and docs/ removed rather than asking them not to look, let them read the public headers because a real reader has them, do not make them hand-author a .tmj, give them a screenshot helper, and verify their binary yourself -- plus the sandbox recipe, and the warning to prove the build path works before handing it over. It also records how to read the report, because a weaker model states its own mistakes as documentation defects with total confidence. Reproduce before believing; and a rejected finding usually still leaves something behind. The evidence for doing any of this is in the table at the end: four read-only reviews of chapters 20 and 21 found real gaps and missed all three of the defects the first build-and-run found, including a published example output the chapter could not produce. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KzBDV2fqgnUAcqCKqKvc71
14 KiB
Maintaining the manual
This file documents the harness that keeps docs/ honest. It is not a chapter —
a reader learning libakgl never needs it — and it is checked by the same harness
it describes, so the file that publishes the convention is held to it.
The harness checks that every listing is true. It cannot check that a chapter
teaches. For that, the tutorials get a second test: the chapter is handed to a
subagent on a weaker model with the library, the headers and the art but not
the finished program, and that reader has to build and run the game. See
AGENTS.md, "Testing a Tutorial". Four read-only reviews of chapters 20 and 21
missed three defects the first build-and-run found in minutes.
Documentation examples
Every fenced block in docs/*.md is compiled, linked, run, or matched against
a source file by ctest. The test is docs_examples; it runs in an ordinary
build and it fails the build.
It exists because the documentation this manual replaced had not been executed
since the day it was typed. In four hundred lines of README.md: two PASS()
calls with unbalanced parentheses, a sprite->frameids = [0, 1, 2, 3]; that is
not C in any dialect, a stray 9 inside a bitmask expression, an
int screenwidth = NULL, and — in the first snippet a reader ever saw — the
exact strncpy call AGENTS.md forbids under Copying Into Fixed-Width
Fields. Two prose claims in the headers were false against src/ as well.
That is not a reader routing around typos; it is what happens to samples nothing
runs.
Documentation goes stale because the code moved, not because somebody edited a chapter. So this is an ordinary test case rather than something a docs-only CI job runs behind a path filter — it fails when the library changes under the chapter, which is precisely the case a path filter would miss.
An untagged block is a failure, not a default, and so is a tag this harness does not know. That is deliberate: the way a harness like this dies is by quietly matching nothing and passing, so an untagged block is a missing decision rather than a free pass.
The tags
| Info string | What happens |
|---|---|
c |
A whole translation unit. Compiled with -fsyntax-only -std=gnu99 -Wall -Werror against the real include path. It shows its own #include lines |
c wrap=NAME |
The same, with tests/docs_preludes/NAME.pre before it and NAME.post after |
c run=NAME |
Wrapped in NAME, linked against akgl and run headless. Must exit 0 and must not report an unhandled or ignored error |
c excerpt=PATH |
Must still appear in PATH, ignoring comments and whitespace. Not compiled |
json excerpt=PATH |
The same check against an asset file, for a fragment too large to show whole |
c screenshot=NAME |
Compiled as an akglframe body, and the source of docs/images/NAME.png. Optionally size=WxH |
json kind=KIND |
Written to a file and loaded through libakgl's own loader. KIND is sprite, character, tilemap or properties |
output |
The exact stdout of the runnable block above it, compared byte for byte |
sh |
Run in a sandbox. Must exit 0. A leading $ is stripped |
sh norun |
Shown, not run. Destructive, networked, or it re-enters this suite |
cmake |
Never executed. Hand-maintained, by decision |
text |
Never executed. A block diagram, or a stack trace captured from a real run |
norun |
Shown, never executed, whatever it is |
These attributes combine with several of the above:
| Attribute | What it does |
|---|---|
setup=NAME |
Runs tests/docs_setups/NAME.sh in the sandbox first, with the repository root as its only argument. Valid on c run=, json, sh and c screenshot= |
size=WxH |
A figure's surface size. Only on c screenshot=; the default is 320x240 |
name=NAME |
Gives a json block a name a later block in the same chapter can preload |
preload=A,B |
Loads those named json blocks, in that order, before this one |
Which one to reach for
In this order, and a step down the list is a decision that belongs in the prose:
c run=akglappwith anoutputblock below it. The strongest shape there is.-fsyntax-onlyproves a call typechecks; it does not prove the startup order works, that the sprite loaded, or that theATTEMPTblock gave back what it took. Arunblock proves all three, and theoutputblock pins what the reader will actually see.c run=akglappalone, where the example has no output. The harness still asserts a clean exit and that nothing raised.c wrap=akglbody, for a call sequence whose effect is not visible on stdout.c, for a translation unit the chapter wants shown whole — a header, a backend definition, amain.c excerpt=PATH, for a constant table, atypedefor a declaration. Compiling one of those would only redefine the type; the question worth asking is whether it still says what the header says.json kind=, for any asset file.norun, for a fragment or something that cannot be run here. Say why in the prose. A risingnoruncount is a review finding — it is the harness being talked out of its job.
Preludes
tests/docs_preludes/NAME.pre and NAME.post bracket the block. They exist so
an example can be written the way a reader wants to read it while still
compiling — four lines of PREPARE_ERROR scaffolding repeated in twenty
chapters teaches the protocol twenty times and the call once.
| Prelude | What the block supplies |
|---|---|
akglfile |
File scope: declarations, a movementlogicfunc, a backend vtable. Includes only; there is no .post |
akglbody |
The inside of a function returning an error context, with errctx already prepared |
akglapp |
The same body, plus a main that brings libakgl up headless and runs it. This is what run= uses |
akglframe |
One frame of drawing, for a figure. Linked against tools/docs_screenshot.c |
All four begin with tests/docs_preludes/docs_prelude.h, which is the include
set — every public header, since the headers suite already proves each one is
self-contained. A block with no wrap= gets none of it and shows its own
includes, which is usually what a chapter wants a reader to see.
What does not belong in a prelude is anything that would let a wrong example
compile. A prelude declaring akgl_game_init itself would defeat the entire
check.
Compiler diagnostics point at the markdown: the harness emits a #line
directive before each body, so a broken example reports as
docs/14-physics.md:112: error: too few arguments, not as a line in a scratch
file nobody can open.
Why -Werror here and not on the library
AGENTS.md keeps AKGL_WERROR off by default, and for a good reason:
libakgl is consumed with add_subdirectory() — akbasic does exactly that — so
a target-level -Werror turns every diagnostic a newer compiler invents into a
broken build for somebody else's project.
A documentation snippet is not a consumer. A sample that warns is a sample that
teaches the warning, and it is the maintainer of this repository who sees the
failure. So the snippets get -Werror and the library does not.
gnu99 rather than c99, because akerror.h uses PATH_MAX, which
<limits.h> hides under __STRICT_ANSI__. A stricter dialect would fail on the
dependency rather than on the example.
Asset formats
A json kind= block is written to a file and handed to akgl_docs_checkjson,
which brings libakgl up headless and calls the loader that really reads that
format — akgl_sprite_load_json, akgl_character_load_json,
akgl_tilemap_load or akgl_registry_load_properties.
This exists because the formats are described in prose here and parsed in
src/, with nothing tying the two together. That gap is not hypothetical:
util/assets/littleguy.json ships beside the loader it is invalid against.
The loaders really open the images and tilesets a block names, so a block that
references one needs a setup=. Adding a format is a row in the KINDS table
in tools/docs_checkjson.c, not a branch.
Three of the four formats are not self-contained. A character names sprites
that have to be in AKGL_REGISTRY_SPRITE already; a tilemap names characters,
and really opens its tileset image. There are two ways to satisfy that, and
which one to use depends on whether the dependency is something the reader
should see.
Show it, with name= and preload=. A json kind=sprite name=herostand
block earlier in a chapter and a json kind=character preload=herostand block
later in the same one are loaded into a single process, in that order. The
dependency ends up in front of the reader — which is right when the chapter is
teaching how the two files relate. Names are per-document, so one chapter cannot
reach into another and make the order documents are checked in load-bearing.
Hide it, with setup=. When the chapter is about one file and the others
would be noise, the fixtures go in a setup script and the prepare column of
the KINDS table in tools/docs_checkjson.c loads them:
| Setup | What it stages | What loads it |
|---|---|---|
setup=spritesheet |
./spritesheet.png, a 12x8 grid of 48x48 frames |
the sprite row's prepare |
setup=character |
./sprites/*.json plus a sheet, named hero standing left and hero walking left |
the character row's prepare |
setup=tilemap |
./sprites/*.json, ./characters/testcharacter.json, and assets/tileset.png |
the tilemap row's prepare |
setup=sidescroller |
every PNG and sprite file from docs/tutorials/assets/sidescroller, in ./ and ./sprites |
both rows' prepare |
setup=jrpg |
the same, from docs/tutorials/assets/jrpg |
both rows' prepare |
The last two are how the tutorials show a reader a real asset file rather than a generic one. A beginner following chapter 20 or 21 types what is on the page, so what is on the page has to be a file that loads — the setup stages the tutorial's own art so the block on the page is the block in the repository.
Those names are a contract between the setup script and the chapter using it.
Change one and change the other, in the same commit. assets/tileset.png is
deliberately not the RPG-Maker-named image tests/assets/testmap.tmj uses — see
the comment in tests/docs_setups/tilemap.sh.
Two things about map fixtures, both confirmed against src/tilemap.c rather
than against the prose:
- Tilesets must be embedded.
"source"appears nowhere in the loader;akgl_tilemap_load_tilesets_eachreadscolumns,firstgid,tilecountandimagestraight off each element of the map's owntilesetsarray. The external reference Tiled writes by default does not load. - Every object in an object group needs a
typestring, plain rectangles included. A missing one fails the whole load.
Figures are output, not assets
A chapter about drawing wants a picture of what the drawing calls draw, and the way a picture goes wrong is that it stops being of the code beside it — silently, because a hand-taken screenshot still looks like a screenshot long after the call changed. So a figure is generated from the listing it illustrates:
cmake --build build --target docs_screenshots
tools/docs_screenshot.c is a small second host: the dummy video driver, a
software renderer, one call to the block's docs_frame(), read the target back,
write a PNG. tools/docs_screenshots.sh compiles the listing against it.
Three rules, and docs/images/README.md carries the long version:
- The PNGs are tracked. A reader on the forge has no build tree. They are generated files tracked on purpose, so do not hand-edit one — change the listing and regenerate.
- The build never regenerates them. The target is only ever run deliberately.
docs_examplesfails a tagged block with no image, so a figure cannot be added to a chapter and then forgotten. Whether the image is of that listing is the separatedocs_screenshotstest's question: it re-renders every figure into a scratch directory and compares byte for byte, and it never repairs what it finds.
When docs_examples fails
The message names the file and the line of the block's opening fence. For an
output mismatch it prints both sides through cat -A, because the errors it
catches are trailing spaces and missing newlines, which a plain diff renders
invisibly.
Fix the documentation, not the expectation — unless the library is what
changed, in which case fix the library first and the documentation second. Never
edit an output block to match output you have not looked at.
Run one document at a time while you work:
./tests/docs_examples.sh --root . \
--cflags-file build/docs_cflags.txt \
--ldflags-file build/docs_ldflags.txt \
--checkjson build/akgl_docs_checkjson \
docs/14-physics.md
Every path is made absolute by the script before it starts, because it cds
into sandbox directories. A relative one used to fail every example and every
setup= — invisible under CTest, which passes absolute paths, and immediate for
anyone running one document by hand.
Omit a flag and the blocks that need it are skipped and counted, not silently passed, so a partial invocation cannot look like a clean run.
Exit status is the number of failed examples, the house convention; 2 is a
usage or setup error, which is a different thing and has to be distinguishable.
A document named on the command line that cannot be read is 2 — never a
silent fall back to the default list, which is how a suite ends up passing
having checked nothing.
Read the census line
The last two lines of a run report what it executed, by kind:
ran: 41 c snippets, 6 linked programs, 6 output comparisons, 18 excerpts, 4 json blocks, 2 shell blocks, 3 figures
skipped: 5 norun, 2 cmake, 7 text, 0 c (no compiler flags given), 0 run (no linker flags given), 0 json (no validator given)
That count is part of the result, not decoration. A harness that passes because
a chapter stopped matching the extractor looks exactly like a harness that
passes because the documentation is correct, and the census is the only thing
that tells them apart. In akbasic it caught exactly that: a CMake generator
expression evaluated to an empty argument, the script read it as a filename, and
the suite passed having checked no documents at all.