Files
libakgl/docs/MAINTENANCE.md
Andrew Kesterson 75da766724
Some checks failed
libakgl CI Build / cmake_build (push) Failing after 21s
libakgl CI Build / performance (push) Failing after 20s
libakgl CI Build / memory_check (push) Failing after 17s
libakgl CI Build / mutation_test (push) Failing after 19s
Record how to find out whether a tutorial teaches
docs_examples proves every listing in docs/ compiles and still matches the file
it was quoted from. It cannot prove a chapter teaches anything -- a document can
be made entirely of verified excerpts and still be unfollowable, because what a
reader needs is the glue between them.

So AGENTS.md gains "Testing a Tutorial": hand the chapter to a subagent on a
weaker model with the library, its headers and the art but not the finished
program, and make them build and run the game. The weaker model is the point; it
will not paper over a gap with knowledge the document did not give it.

The section carries the rules that make the result mean something -- copy the
tree with examples/ and docs/ removed rather than asking them not to look, let
them read the public headers because a real reader has them, do not make them
hand-author a .tmj, give them a screenshot helper, and verify their binary
yourself -- plus the sandbox recipe, and the warning to prove the build path
works before handing it over.

It also records how to read the report, because a weaker model states its own
mistakes as documentation defects with total confidence. Reproduce before
believing; and a rejected finding usually still leaves something behind.

The evidence for doing any of this is in the table at the end: four read-only
reviews of chapters 20 and 21 found real gaps and missed all three of the
defects the first build-and-run found, including a published example output the
chapter could not produce.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KzBDV2fqgnUAcqCKqKvc71
2026-08-02 09:45:28 -04:00

14 KiB

Maintaining the manual

This file documents the harness that keeps docs/ honest. It is not a chapter — a reader learning libakgl never needs it — and it is checked by the same harness it describes, so the file that publishes the convention is held to it.

The harness checks that every listing is true. It cannot check that a chapter teaches. For that, the tutorials get a second test: the chapter is handed to a subagent on a weaker model with the library, the headers and the art but not the finished program, and that reader has to build and run the game. See AGENTS.md, "Testing a Tutorial". Four read-only reviews of chapters 20 and 21 missed three defects the first build-and-run found in minutes.

Documentation examples

Every fenced block in docs/*.md is compiled, linked, run, or matched against a source file by ctest. The test is docs_examples; it runs in an ordinary build and it fails the build.

It exists because the documentation this manual replaced had not been executed since the day it was typed. In four hundred lines of README.md: two PASS() calls with unbalanced parentheses, a sprite->frameids = [0, 1, 2, 3]; that is not C in any dialect, a stray 9 inside a bitmask expression, an int screenwidth = NULL, and — in the first snippet a reader ever saw — the exact strncpy call AGENTS.md forbids under Copying Into Fixed-Width Fields. Two prose claims in the headers were false against src/ as well. That is not a reader routing around typos; it is what happens to samples nothing runs.

Documentation goes stale because the code moved, not because somebody edited a chapter. So this is an ordinary test case rather than something a docs-only CI job runs behind a path filter — it fails when the library changes under the chapter, which is precisely the case a path filter would miss.

An untagged block is a failure, not a default, and so is a tag this harness does not know. That is deliberate: the way a harness like this dies is by quietly matching nothing and passing, so an untagged block is a missing decision rather than a free pass.

The tags

Info string What happens
c A whole translation unit. Compiled with -fsyntax-only -std=gnu99 -Wall -Werror against the real include path. It shows its own #include lines
c wrap=NAME The same, with tests/docs_preludes/NAME.pre before it and NAME.post after
c run=NAME Wrapped in NAME, linked against akgl and run headless. Must exit 0 and must not report an unhandled or ignored error
c excerpt=PATH Must still appear in PATH, ignoring comments and whitespace. Not compiled
json excerpt=PATH The same check against an asset file, for a fragment too large to show whole
c screenshot=NAME Compiled as an akglframe body, and the source of docs/images/NAME.png. Optionally size=WxH
json kind=KIND Written to a file and loaded through libakgl's own loader. KIND is sprite, character, tilemap or properties
output The exact stdout of the runnable block above it, compared byte for byte
sh Run in a sandbox. Must exit 0. A leading $ is stripped
sh norun Shown, not run. Destructive, networked, or it re-enters this suite
cmake Never executed. Hand-maintained, by decision
text Never executed. A block diagram, or a stack trace captured from a real run
norun Shown, never executed, whatever it is

These attributes combine with several of the above:

Attribute What it does
setup=NAME Runs tests/docs_setups/NAME.sh in the sandbox first, with the repository root as its only argument. Valid on c run=, json, sh and c screenshot=
size=WxH A figure's surface size. Only on c screenshot=; the default is 320x240
name=NAME Gives a json block a name a later block in the same chapter can preload
preload=A,B Loads those named json blocks, in that order, before this one

Which one to reach for

In this order, and a step down the list is a decision that belongs in the prose:

  • c run=akglapp with an output block below it. The strongest shape there is. -fsyntax-only proves a call typechecks; it does not prove the startup order works, that the sprite loaded, or that the ATTEMPT block gave back what it took. A run block proves all three, and the output block pins what the reader will actually see.
  • c run=akglapp alone, where the example has no output. The harness still asserts a clean exit and that nothing raised.
  • c wrap=akglbody, for a call sequence whose effect is not visible on stdout.
  • c, for a translation unit the chapter wants shown whole — a header, a backend definition, a main.
  • c excerpt=PATH, for a constant table, a typedef or a declaration. Compiling one of those would only redefine the type; the question worth asking is whether it still says what the header says.
  • json kind=, for any asset file.
  • norun, for a fragment or something that cannot be run here. Say why in the prose. A rising norun count is a review finding — it is the harness being talked out of its job.

Preludes

tests/docs_preludes/NAME.pre and NAME.post bracket the block. They exist so an example can be written the way a reader wants to read it while still compiling — four lines of PREPARE_ERROR scaffolding repeated in twenty chapters teaches the protocol twenty times and the call once.

Prelude What the block supplies
akglfile File scope: declarations, a movementlogicfunc, a backend vtable. Includes only; there is no .post
akglbody The inside of a function returning an error context, with errctx already prepared
akglapp The same body, plus a main that brings libakgl up headless and runs it. This is what run= uses
akglframe One frame of drawing, for a figure. Linked against tools/docs_screenshot.c

All four begin with tests/docs_preludes/docs_prelude.h, which is the include set — every public header, since the headers suite already proves each one is self-contained. A block with no wrap= gets none of it and shows its own includes, which is usually what a chapter wants a reader to see.

What does not belong in a prelude is anything that would let a wrong example compile. A prelude declaring akgl_game_init itself would defeat the entire check.

Compiler diagnostics point at the markdown: the harness emits a #line directive before each body, so a broken example reports as docs/14-physics.md:112: error: too few arguments, not as a line in a scratch file nobody can open.

Why -Werror here and not on the library

AGENTS.md keeps AKGL_WERROR off by default, and for a good reason: libakgl is consumed with add_subdirectory()akbasic does exactly that — so a target-level -Werror turns every diagnostic a newer compiler invents into a broken build for somebody else's project.

A documentation snippet is not a consumer. A sample that warns is a sample that teaches the warning, and it is the maintainer of this repository who sees the failure. So the snippets get -Werror and the library does not.

gnu99 rather than c99, because akerror.h uses PATH_MAX, which <limits.h> hides under __STRICT_ANSI__. A stricter dialect would fail on the dependency rather than on the example.

Asset formats

A json kind= block is written to a file and handed to akgl_docs_checkjson, which brings libakgl up headless and calls the loader that really reads that format — akgl_sprite_load_json, akgl_character_load_json, akgl_tilemap_load or akgl_registry_load_properties.

This exists because the formats are described in prose here and parsed in src/, with nothing tying the two together. That gap is not hypothetical: util/assets/littleguy.json ships beside the loader it is invalid against.

The loaders really open the images and tilesets a block names, so a block that references one needs a setup=. Adding a format is a row in the KINDS table in tools/docs_checkjson.c, not a branch.

Three of the four formats are not self-contained. A character names sprites that have to be in AKGL_REGISTRY_SPRITE already; a tilemap names characters, and really opens its tileset image. There are two ways to satisfy that, and which one to use depends on whether the dependency is something the reader should see.

Show it, with name= and preload=. A json kind=sprite name=herostand block earlier in a chapter and a json kind=character preload=herostand block later in the same one are loaded into a single process, in that order. The dependency ends up in front of the reader — which is right when the chapter is teaching how the two files relate. Names are per-document, so one chapter cannot reach into another and make the order documents are checked in load-bearing.

Hide it, with setup=. When the chapter is about one file and the others would be noise, the fixtures go in a setup script and the prepare column of the KINDS table in tools/docs_checkjson.c loads them:

Setup What it stages What loads it
setup=spritesheet ./spritesheet.png, a 12x8 grid of 48x48 frames the sprite row's prepare
setup=character ./sprites/*.json plus a sheet, named hero standing left and hero walking left the character row's prepare
setup=tilemap ./sprites/*.json, ./characters/testcharacter.json, and assets/tileset.png the tilemap row's prepare
setup=sidescroller every PNG and sprite file from docs/tutorials/assets/sidescroller, in ./ and ./sprites both rows' prepare
setup=jrpg the same, from docs/tutorials/assets/jrpg both rows' prepare

The last two are how the tutorials show a reader a real asset file rather than a generic one. A beginner following chapter 20 or 21 types what is on the page, so what is on the page has to be a file that loads — the setup stages the tutorial's own art so the block on the page is the block in the repository.

Those names are a contract between the setup script and the chapter using it. Change one and change the other, in the same commit. assets/tileset.png is deliberately not the RPG-Maker-named image tests/assets/testmap.tmj uses — see the comment in tests/docs_setups/tilemap.sh.

Two things about map fixtures, both confirmed against src/tilemap.c rather than against the prose:

  • Tilesets must be embedded. "source" appears nowhere in the loader; akgl_tilemap_load_tilesets_each reads columns, firstgid, tilecount and image straight off each element of the map's own tilesets array. The external reference Tiled writes by default does not load.
  • Every object in an object group needs a type string, plain rectangles included. A missing one fails the whole load.

Figures are output, not assets

A chapter about drawing wants a picture of what the drawing calls draw, and the way a picture goes wrong is that it stops being of the code beside it — silently, because a hand-taken screenshot still looks like a screenshot long after the call changed. So a figure is generated from the listing it illustrates:

cmake --build build --target docs_screenshots

tools/docs_screenshot.c is a small second host: the dummy video driver, a software renderer, one call to the block's docs_frame(), read the target back, write a PNG. tools/docs_screenshots.sh compiles the listing against it.

Three rules, and docs/images/README.md carries the long version:

  • The PNGs are tracked. A reader on the forge has no build tree. They are generated files tracked on purpose, so do not hand-edit one — change the listing and regenerate.
  • The build never regenerates them. The target is only ever run deliberately.
  • docs_examples fails a tagged block with no image, so a figure cannot be added to a chapter and then forgotten. Whether the image is of that listing is the separate docs_screenshots test's question: it re-renders every figure into a scratch directory and compares byte for byte, and it never repairs what it finds.

When docs_examples fails

The message names the file and the line of the block's opening fence. For an output mismatch it prints both sides through cat -A, because the errors it catches are trailing spaces and missing newlines, which a plain diff renders invisibly.

Fix the documentation, not the expectation — unless the library is what changed, in which case fix the library first and the documentation second. Never edit an output block to match output you have not looked at.

Run one document at a time while you work:

./tests/docs_examples.sh --root . \
    --cflags-file  build/docs_cflags.txt \
    --ldflags-file build/docs_ldflags.txt \
    --checkjson    build/akgl_docs_checkjson \
    docs/14-physics.md

Every path is made absolute by the script before it starts, because it cds into sandbox directories. A relative one used to fail every example and every setup= — invisible under CTest, which passes absolute paths, and immediate for anyone running one document by hand.

Omit a flag and the blocks that need it are skipped and counted, not silently passed, so a partial invocation cannot look like a clean run.

Exit status is the number of failed examples, the house convention; 2 is a usage or setup error, which is a different thing and has to be distinguishable. A document named on the command line that cannot be read is 2 — never a silent fall back to the default list, which is how a suite ends up passing having checked nothing.

Read the census line

The last two lines of a run report what it executed, by kind:

ran: 41 c snippets, 6 linked programs, 6 output comparisons, 18 excerpts, 4 json blocks, 2 shell blocks, 3 figures
skipped: 5 norun, 2 cmake, 7 text, 0 c (no compiler flags given), 0 run (no linker flags given), 0 json (no validator given)

That count is part of the result, not decoration. A harness that passes because a chapter stopped matching the extractor looks exactly like a harness that passes because the documentation is correct, and the census is the only thing that tells them apart. In akbasic it caught exactly that: a CMake generator expression evaluated to an empty argument, the script read it as a filename, and the suite passed having checked no documents at all.