diff --git a/AGENTS.md b/AGENTS.md index 705a047..0b51199 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -682,6 +682,101 @@ of a hundred thousand times, and the timings a checked run prints are labelled as meaningless. **Do not add memory-check suites**: a new path worth checking belongs in a benchmark, where it gets both. +## Testing a Tutorial + +`ctest -R docs_examples` proves every listing in `docs/` still compiles and +still matches the file it was quoted from. It cannot prove the chapter *teaches* +anything: a document can be composed entirely of verified excerpts and still be +unfollowable, because what a reader needs is the glue between them. + +**So test a tutorial by making somebody follow it.** Hand it to a subagent on a +weaker model with no context beyond the chapter, and have them build the thing it +describes. The weaker model is the point -- it will not paper over a gap with +knowledge the document did not give it, which is exactly the failure mode a +capable reviewer has. + +### The rules that make the result mean something + +- **Give them what a real reader has, and nothing more.** The chapter, the + library source, its public headers, and the art. **Not the finished program.** + Copy the tree with `examples/` and `docs/` removed rather than telling them not + to look -- a rule they can break is not a control. +- **Headers are fair game.** The manual defers signatures to Doxygen and a real + user has `include/akgl/*.h` in front of them. Forbidding those tests a reader + who does not exist. +- **They must compile it and run it.** This is the whole thing. A design nobody + built proves nothing, and the defects that matter most do not fail to compile. +- **Verify their work yourself.** Run the binary they produced. Look at the + screenshot they took. Do not take a report's word for what it built -- + see "Reading the report" below. +- **Do not make them author assets by hand.** A real reader draws a map in Tiled; + hand-writing a `.tmj` tests nothing about the chapter and eats the whole + session. Give them `docs/tutorials/assets/`. The asset *formats* are already + covered by `docs_examples`, which runs the chapter's own JSON blocks through + `akgl_sprite_load_json` and `akgl_character_load_json`. +- **Give them a screenshot helper**, marked as scaffolding and not part of the + tutorial, so there is a picture to check. Screen output is evidence; an exit + status of 0 is not. + +### Setting the sandbox up + +One directory per reader, because two of them building at once in the same tree +collide: + +```sh +rsync -a --exclude='.git' --exclude='build' --exclude='examples' --exclude='docs' \ + . "$SB/reader/libakgl/" +cp docs/20-tutorial-sidescroller.md "$SB/reader/TUTORIAL.md" +cp -r docs/tutorials/assets/sidescroller "$SB/reader/art" +mkdir -p "$SB/reader/game" +``` + +A consumer using the CMake the chapter itself teaches -- +`add_subdirectory(../libakgl libakgl)` plus the documented +`target_link_libraries` line -- configures and builds in about a minute from +cold, and seconds after that. Tell them to use +`cmake --build build --target --parallel`, or they will also build +every test suite in the tree. + +**Prove the path works before you hand it over.** Write a throwaway consumer +that opens a window and takes a screenshot, build it, run it, and delete it. A +reader who cannot build is a reader who finds nothing, and the fault will be +yours -- the screenshot helper's first draft included `akgl/renderer.h` for +`akgl_renderer`, which is declared in `akgl/game.h`, and would have cost them +the session. + +### Reading the report + +**Every finding is a hypothesis until you check it.** A weaker model reports its +own mistakes as documentation defects with total confidence, and both kinds are +worth having -- but only one is worth acting on. + +- Reproduce the failure before believing it. One reader reported the dialogue + freeze broken; it had drawn its own map with the NPC out of reach and never + used the one it was given. Running the reference showed the player moving zero + pixels through the whole freeze window. +- A rejected finding usually still leaves something. That same map failure was + not a code defect, but it did show the chapter stated `TALK_RANGE` without + saying what it means when you are placing NPCs -- and that a too-distant NPC + produces *success*, not an error. +- Fix the document, not the reader. If they guessed a header name, the chapter + never named it. + +### What this catches that review does not + +Four read-only passes over the two tutorials in `docs/` found real gaps and +missed all three of these, which the first build-and-run found immediately: + +| Defect | Why reading missed it | +|---|---| +| libakstdlib's header was never named -- `aksl_*` functions, `` file | Every reviewer already knew, or did not have to write the `#include` | +| The asset directory defaulted to `"."` instead of the compiled-in macro | Prose said "falls back", and nobody had to run it from another directory | +| A published example output the chapter could not produce | The capture happened after teardown released the pool. Nothing fails to compile | + +The third is the shape to remember: **a tutorial that prints an expected result +is making a claim, and a claim nobody executed is a claim that is probably +wrong.** + ## Commit & Pull Request Guidelines Recent commits use concise, imperative summaries such as `Fix a scale bug...` and often explain related changes in one sentence. Keep each commit scoped and describe user-visible behavior. Pull requests should summarize the change, identify affected modules, list the CMake/CTest commands run, and link relevant issues. Include screenshots only for rendering, map, or `charviewer` changes. diff --git a/docs/MAINTENANCE.md b/docs/MAINTENANCE.md index f11e6b4..70a5491 100644 --- a/docs/MAINTENANCE.md +++ b/docs/MAINTENANCE.md @@ -4,6 +4,13 @@ This file documents the harness that keeps `docs/` honest. It is not a chapter a reader learning libakgl never needs it — and it is checked by the same harness it describes, so the file that publishes the convention is held to it. +**The harness checks that every listing is true. It cannot check that a chapter +teaches.** For that, the tutorials get a second test: the chapter is handed to a +subagent on a weaker model with the library, the headers and the art but *not* +the finished program, and that reader has to build and run the game. See +`AGENTS.md`, "Testing a Tutorial". Four read-only reviews of chapters 20 and 21 +missed three defects the first build-and-run found in minutes. + ## Documentation examples **Every fenced block in `docs/*.md` is compiled, linked, run, or matched against