Compile, link and run every example in the documentation
libakgl exports 157 functions and the only user-facing documentation was a FAQ in README.md whose examples did not compile: two unbalanced `PASS()` calls, a `sprite->frameids = [0, 1, 2, 3];` that is not C in any dialect, a stray `9` inside a bitmask expression, an `int screenwidth = NULL`, and -- in the first snippet a reader ever saw -- the exact `strncpy` call AGENTS.md forbids. That is not a reader routing around typos. It is what happens to samples that nothing executes. This is akbasic's documentation harness with the interpreter taken out and the ability to link and run put in. The contract is a set of fence info strings, so an example is ordinary markdown that still highlights on the forge: ```c compiled with -fsyntax-only -Wall -Werror ```c wrap=NAME the same, wrapped in tests/docs_preludes/NAME.pre/.post ```c run=NAME linked against akgl and run headless ```c excerpt=PATH must still appear verbatim in PATH ```c screenshot=NAME also the source of docs/images/NAME.png ```json kind=KIND loaded through the real akgl_*_load_json ```output the exact stdout of the runnable block above it `run=` is why this links at all: -fsyntax-only proves a call typechecks, not that the startup order works or that an ATTEMPT block gives back what it took. `json kind=` exists because the asset formats are documented in prose and read by four loaders with nothing tying the two together -- util/assets/littleguy.json is already invalid against the loader it ships with. A fence with no info string is a hard error, and so is an unknown one. The failure mode this exists to prevent is passing because it quietly ran nothing, so an unannotated block is a missing decision rather than a default. Exit status is the number of failed examples; 2 for a usage error, which is a different thing and has to be distinguishable. Proven to fail on all ten of its failure modes -- untagged fence, non-compiling snippet, stale excerpt, orphan output block, unknown info string, run= output mismatch, invalid JSON, missing figure, a -Werror warning, and a dangling preload= -- because a check that has never failed has not been tested. `-Werror` here although AKGL_WERROR stays off for the library: that option is off so a new compiler's diagnostic cannot break a consumer's build, and a doc snippet is not a consumer. A sample that warns is a sample that teaches the warning. Registered as `docs_examples` and `docs_screenshots`. No docs-path filter in CI, deliberately: documentation goes stale because the code moved, not because somebody edited a chapter. Co-Authored-By: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
680
tests/docs_examples.sh
Executable file
680
tests/docs_examples.sh
Executable file
@@ -0,0 +1,680 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# Compile, link, run and cross-check every example in docs/.
|
||||
#
|
||||
# libakgl exports 156 functions and, before this existed, the only user-facing
|
||||
# documentation was a FAQ in README.md whose examples did not compile: two
|
||||
# unbalanced PASS() calls, a `sprite->frameids = [0, 1, 2, 3];` that is not C in
|
||||
# any dialect, a stray `9` inside a bitmask expression, an `int screenwidth =
|
||||
# NULL`, and -- in the first snippet a reader ever saw -- the exact strncpy call
|
||||
# AGENTS.md forbids. The prose had drifted the same way: two claims in the
|
||||
# headers were false against src/.
|
||||
#
|
||||
# That is not a reader routing around typos. It is what happens to samples and
|
||||
# prose that nothing executes. AGENTS.md already makes the argument in another
|
||||
# context -- "A test that has not failed has not been tested" -- and the sibling
|
||||
# akbasic repository solved this exact problem, so this is its harness with the
|
||||
# interpreter taken out and the ability to link and run put in.
|
||||
#
|
||||
# Documentation goes stale because the *code* moved, not because somebody edited
|
||||
# a chapter, so this runs on every ctest rather than behind a docs path filter.
|
||||
#
|
||||
# **Exit status is the number of failed examples**, the house convention. 2 for a
|
||||
# usage or setup error, which is a different thing and has to be distinguishable.
|
||||
#
|
||||
# The contract with the documentation is a set of fence info strings. They are
|
||||
# ordinary markdown, they render as syntax highlighting on the forge, and they
|
||||
# sit next to the thing they describe. docs/MAINTENANCE.md is the reference; the
|
||||
# short version:
|
||||
#
|
||||
# ```c a translation unit: compile it with -fsyntax-only
|
||||
# ```c wrap=NAME the same, wrapped in tests/docs_preludes/NAME.pre/.post
|
||||
# ```c run=NAME wrapped, linked against akgl, and run headless
|
||||
# ```c excerpt=PATH must appear verbatim in PATH; not compiled
|
||||
# ```c screenshot=NAME also the source of docs/images/NAME.png
|
||||
# ```json kind=KIND loaded through the real akgl_*_load_json
|
||||
# ```output the exact stdout of the runnable block above it
|
||||
# ```sh run in a sandbox; must exit 0
|
||||
# ```sh setup=NAME the same, after tests/docs_setups/NAME.sh
|
||||
# ```sh norun destructive, networked, or re-enters this suite
|
||||
# ```cmake never executed; hand-maintained, by decision
|
||||
# ```text a diagram or a captured dump; nothing to execute
|
||||
# ```norun shown, never executed, whatever it is
|
||||
#
|
||||
# A block with **no** info string is a hard error, and so is one whose info
|
||||
# string this script does not know. That is deliberate: the failure mode the
|
||||
# whole harness exists to avoid is passing because it quietly ran nothing, so an
|
||||
# unannotated block is a missing decision rather than a default.
|
||||
|
||||
set -u
|
||||
|
||||
ROOT=""
|
||||
CFLAGS_FILE=""
|
||||
LDFLAGS_FILE=""
|
||||
CHECKJSON=""
|
||||
|
||||
usage()
|
||||
{
|
||||
cat >&2 <<'EOF'
|
||||
usage: docs_examples.sh --root DIR [--cflags-file FILE] [--ldflags-file FILE]
|
||||
[--checkjson PATH] [FILE...]
|
||||
|
||||
--root DIR repository root; documentation paths are relative to it
|
||||
--cflags-file FILE one compiler flag per line, for the ```c blocks
|
||||
--ldflags-file FILE one linker argument per line, for the ```c run= blocks
|
||||
--checkjson PATH the built akgl_docs_checkjson, for the ```json blocks
|
||||
FILE... which documents to check (default: docs/*.md)
|
||||
|
||||
Exit status is the number of failed examples, or 2 for a usage or setup error.
|
||||
EOF
|
||||
exit 2
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--root) ROOT="$2"; shift 2 ;;
|
||||
--cflags-file) CFLAGS_FILE="$2"; shift 2 ;;
|
||||
--ldflags-file) LDFLAGS_FILE="$2"; shift 2 ;;
|
||||
--checkjson) CHECKJSON="$2"; shift 2 ;;
|
||||
--help|-h) usage ;;
|
||||
--*) echo "unknown option $1" >&2; usage ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "${ROOT}" ] || usage
|
||||
|
||||
# Every path is forced absolute before anything else runs, because this script
|
||||
# cd's into sandbox directories. akbasic learned that the hard way: a relative
|
||||
# --root could not find tests/docs_setups, which is invisible under CTest -- it
|
||||
# passes absolute paths -- and immediate for anybody running one document by
|
||||
# hand.
|
||||
for _var in CHECKJSON CFLAGS_FILE LDFLAGS_FILE; do
|
||||
_val="${!_var}"
|
||||
case "${_val}" in
|
||||
""|/*) ;;
|
||||
*) printf -v "${_var}" '%s' "${PWD}/${_val}" ;;
|
||||
esac
|
||||
done
|
||||
unset _var _val
|
||||
|
||||
cd "${ROOT}" || exit 2
|
||||
ROOT="${PWD}"
|
||||
|
||||
DOCS=("$@")
|
||||
if [ ${#DOCS[@]} -eq 0 ]; then
|
||||
# The default list is a glob and is allowed to match nothing: the chapters
|
||||
# are written in parallel with this, and a harness that refuses to configure
|
||||
# until docs/ exists is a harness nobody can land first. A document named on
|
||||
# the command line is a different case entirely -- see below.
|
||||
DOCS=()
|
||||
for _doc in docs/*.md; do
|
||||
[ -r "${_doc}" ] && DOCS+=("${_doc}")
|
||||
done
|
||||
unset _doc
|
||||
if [ ${#DOCS[@]} -eq 0 ]; then
|
||||
echo "ran: nothing -- no documents matched docs/*.md"
|
||||
echo "PASS: there is no documentation to check yet"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# Refuse a named document that is not there rather than checking nothing and
|
||||
# passing. An empty argument is the specific case that got here in akbasic: a
|
||||
# CMake generator expression evaluating to nothing still contributes an empty
|
||||
# argument, which looked like a filename and silently replaced the whole list.
|
||||
for _doc in "${DOCS[@]}"; do
|
||||
if [ ! -r "${_doc}" ]; then
|
||||
echo "FAIL: no document at \"${_doc}\"" >&2
|
||||
exit 2
|
||||
fi
|
||||
done
|
||||
unset _doc
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "${WORK}"' EXIT
|
||||
|
||||
# The compiler flags the ```c blocks need. CMake writes them, because the
|
||||
# include path is transitive through akerror, akstdlib, SDL3 and jansson, and
|
||||
# hardcoding it here would rot exactly the way the documentation does.
|
||||
CFLAGS=()
|
||||
if [ -n "${CFLAGS_FILE}" ] && [ -r "${CFLAGS_FILE}" ]; then
|
||||
while IFS= read -r _flag; do
|
||||
[ -n "${_flag}" ] && CFLAGS+=("${_flag}")
|
||||
done < "${CFLAGS_FILE}"
|
||||
# The preludes reach their shared include set as <docs_prelude.h>, and the
|
||||
# unit is compiled from a scratch directory, so the search path has to say
|
||||
# where that is.
|
||||
CFLAGS+=("-I${ROOT}/tests/docs_preludes")
|
||||
fi
|
||||
|
||||
LDFLAGS=()
|
||||
if [ -n "${LDFLAGS_FILE}" ] && [ -r "${LDFLAGS_FILE}" ]; then
|
||||
while IFS= read -r _flag; do
|
||||
[ -n "${_flag}" ] && LDFLAGS+=("${_flag}")
|
||||
done < "${LDFLAGS_FILE}"
|
||||
fi
|
||||
unset _flag
|
||||
|
||||
CC="${CC:-cc}"
|
||||
|
||||
FAILURES=0
|
||||
declare -A RAN=([c]=0 [run]=0 [output]=0 [excerpt]=0 [json]=0 [sh]=0 [screenshot]=0)
|
||||
declare -A SKIPPED=([norun]=0 [cmake]=0 [text]=0 [nocc]=0 [nold]=0 [nojson]=0)
|
||||
|
||||
# Named `json` blocks, so a later one can name them in `preload=`. Reset per
|
||||
# document: a name is a fact about one chapter, and letting chapter 13 reach
|
||||
# into chapter 11 would make the order documents are checked in load-bearing.
|
||||
declare -A JSON_KIND_OF=()
|
||||
declare -A JSON_BODY_OF=()
|
||||
|
||||
# Every failure names the file and the line of the block's *opening* fence, so
|
||||
# the message points at the thing to edit rather than at this script.
|
||||
function fail()
|
||||
{
|
||||
local where="$1"; shift
|
||||
echo "FAIL ${where}: $*" >&2
|
||||
FAILURES=$((FAILURES + 1))
|
||||
}
|
||||
|
||||
# Show a byte-exact comparison. `cat -A` because the errors this catches are
|
||||
# trailing spaces and missing newlines, which a plain diff renders invisibly.
|
||||
function show_diff()
|
||||
{
|
||||
local want="$1" got="$2"
|
||||
echo "--- expected ---" >&2
|
||||
cat -A "${want}" >&2
|
||||
echo "--- actual ---" >&2
|
||||
cat -A "${got}" >&2
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------- extraction
|
||||
#
|
||||
# One pass per document, writing each block's body to ${WORK}/blocks/NNNN and a
|
||||
# line to ${WORK}/blocks/index. Keeping the bodies in files rather than in shell
|
||||
# variables means a block containing a NUL, a backslash or an unbalanced quote
|
||||
# is carried through untouched.
|
||||
function extract()
|
||||
{
|
||||
local doc="$1" out="$2"
|
||||
|
||||
awk -v OUT="${out}" -v INDEX="${out}/index" '
|
||||
function pad(n) { return sprintf("%04d", n) }
|
||||
/^```/ {
|
||||
if ( !inblock ) {
|
||||
inblock = 1
|
||||
tag = substr($0, 4)
|
||||
sub(/[ \t]+$/, "", tag)
|
||||
start = NR
|
||||
count++
|
||||
body = OUT "/" pad(count)
|
||||
printf "" > body
|
||||
next
|
||||
}
|
||||
if ( $0 == "```" ) {
|
||||
close(body)
|
||||
printf "%s\t%d\t%s\n", pad(count), start, tag >> INDEX
|
||||
inblock = 0
|
||||
next
|
||||
}
|
||||
}
|
||||
inblock { print >> body }
|
||||
' "${doc}"
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------- helpers
|
||||
|
||||
# The value of attribute $1 in an info string $2, or empty. `wrap=akglbody` and
|
||||
# `kind=sprite` are both read this way.
|
||||
function attr()
|
||||
{
|
||||
local name="$1" info="$2" word
|
||||
for word in ${info}; do
|
||||
case "${word}" in
|
||||
"${name}"=*) echo "${word#*=}"; return ;;
|
||||
esac
|
||||
done
|
||||
}
|
||||
|
||||
# True when a bare word appears in an info string. Used for `norun`.
|
||||
function has_word()
|
||||
{
|
||||
local want="$1" info="$2" word
|
||||
for word in ${info}; do
|
||||
[ "${word}" = "${want}" ] && return 0
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
# Strip C comments and collapse whitespace, so an excerpt can be compared to the
|
||||
# header it came from without either side having to carry the other's doc
|
||||
# comments or indentation. The bit-flag tables in actor.h and iterator.h are
|
||||
# hand-aligned on purpose -- scripts/reindent.el goes out of its way not to
|
||||
# disturb them -- so an excerpt of one has to survive being re-aligned.
|
||||
function normalise_c()
|
||||
{
|
||||
awk '
|
||||
{
|
||||
line = $0
|
||||
out = ""
|
||||
while ( length(line) > 0 ) {
|
||||
if ( incomment ) {
|
||||
i = index(line, "*/")
|
||||
if ( i == 0 ) { line = ""; break }
|
||||
incomment = 0
|
||||
line = substr(line, i + 2)
|
||||
continue
|
||||
}
|
||||
i = index(line, "/*")
|
||||
j = index(line, "//")
|
||||
if ( j > 0 && (i == 0 || j < i) ) {
|
||||
out = out substr(line, 1, j - 1)
|
||||
line = ""
|
||||
break
|
||||
}
|
||||
if ( i == 0 ) { out = out line; line = ""; break }
|
||||
out = out substr(line, 1, i - 1)
|
||||
incomment = 1
|
||||
line = substr(line, i + 2)
|
||||
}
|
||||
printf "%s ", out
|
||||
}
|
||||
END { printf "\n" }
|
||||
' "$1" | tr -s ' \t' ' ' | sed -e 's/^ //' -e 's/ $//'
|
||||
}
|
||||
|
||||
# Run tests/docs_setups/NAME.sh in ${1}, with the repository root as its only
|
||||
# argument. An example that loads an asset needs that asset; putting the fixture
|
||||
# in a setup script rather than in the chapter keeps the example the shape a
|
||||
# reader wants to see.
|
||||
function run_setup()
|
||||
{
|
||||
local sandbox="$1" setup="$2" where="$3"
|
||||
|
||||
[ -n "${setup}" ] || return 0
|
||||
if [ ! -r "${ROOT}/tests/docs_setups/${setup}.sh" ]; then
|
||||
fail "${where}" "setup=${setup} names no tests/docs_setups/${setup}.sh"
|
||||
return 1
|
||||
fi
|
||||
if ! ( cd "${sandbox}" && bash "${ROOT}/tests/docs_setups/${setup}.sh" "${ROOT}" ) \
|
||||
>/dev/null 2>&1; then
|
||||
fail "${where}" "setup=${setup} failed"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# Assemble the translation unit: prelude, a #line directive naming the markdown,
|
||||
# the body, and the closing half of the prelude. Writes ${WORK}/unit.c.
|
||||
#
|
||||
# The #line is what makes a diagnostic read `docs/14-physics.md:112: error: ...`
|
||||
# rather than pointing into a scratch file nobody can open.
|
||||
function build_unit()
|
||||
{
|
||||
local where="$1" body="$2" wrap="$3" doc="$4" line="$5"
|
||||
local unit="${WORK}/unit.c"
|
||||
|
||||
: > "${unit}"
|
||||
if [ -n "${wrap}" ]; then
|
||||
if [ ! -r "${ROOT}/tests/docs_preludes/${wrap}.pre" ]; then
|
||||
fail "${where}" "names no tests/docs_preludes/${wrap}.pre"
|
||||
return 1
|
||||
fi
|
||||
cat "${ROOT}/tests/docs_preludes/${wrap}.pre" >> "${unit}"
|
||||
fi
|
||||
printf '#line %d "%s"\n' "$((line + 1))" "${doc}" >> "${unit}"
|
||||
cat "${body}" >> "${unit}"
|
||||
if [ -n "${wrap}" ] && [ -r "${ROOT}/tests/docs_preludes/${wrap}.post" ]; then
|
||||
printf '#line 1 "tests/docs_preludes/%s.post"\n' "${wrap}" >> "${unit}"
|
||||
cat "${ROOT}/tests/docs_preludes/${wrap}.post" >> "${unit}"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# --------------------------------------------------------------- block kinds
|
||||
|
||||
# A C snippet, compiled and not linked: what most of these examples are for is
|
||||
# showing the API, and the API is exactly what -fsyntax-only checks.
|
||||
#
|
||||
# -Werror here, unlike the library. AGENTS.md keeps AKGL_WERROR off by default
|
||||
# because libakgl is consumed with add_subdirectory() and a new compiler's
|
||||
# diagnostic should not break somebody else's build. A documentation snippet is
|
||||
# not a consumer, and a sample that warns is a sample that teaches the warning.
|
||||
#
|
||||
# gnu99 rather than c99: akerror.h uses PATH_MAX, which <limits.h> hides under
|
||||
# __STRICT_ANSI__, so a stricter dialect would fail on the dependency rather
|
||||
# than on the example.
|
||||
function run_c()
|
||||
{
|
||||
local where="$1" body="$2" info="$3" doc="$4" line="$5"
|
||||
local wrap log="${WORK}/cc.log"
|
||||
|
||||
if [ ${#CFLAGS[@]} -eq 0 ]; then
|
||||
SKIPPED[nocc]=$((SKIPPED[nocc] + 1))
|
||||
return
|
||||
fi
|
||||
|
||||
wrap="$(attr wrap "${info}")"
|
||||
build_unit "${where}" "${body}" "${wrap}" "${doc}" "${line}" || return
|
||||
|
||||
if "${CC}" -fsyntax-only -std=gnu99 -Wall -Werror \
|
||||
"${CFLAGS[@]}" "${WORK}/unit.c" > "${log}" 2>&1; then
|
||||
RAN[c]=$((RAN[c] + 1))
|
||||
else
|
||||
fail "${where}" "does not compile"
|
||||
sed -n '1,25p' "${log}" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
# A C snippet linked into a real program and run headless.
|
||||
#
|
||||
# This is the reason to port the harness at all for a game library. -fsyntax-only
|
||||
# proves a call typechecks; it does not prove the startup order works, that a
|
||||
# sprite loads, or that an ATTEMPT block releases what it acquired. A run= block
|
||||
# asserts a clean exit *and* that nothing reported a raised context on stderr,
|
||||
# because a libakgl program can fail and still exit 0.
|
||||
function run_c_run()
|
||||
{
|
||||
local where="$1" body="$2" info="$3" doc="$4" line="$5" expected="$6"
|
||||
local wrap log="${WORK}/cc.log" prog="${WORK}/prog"
|
||||
local sandbox="${WORK}/run" got="${WORK}/got" errout="${WORK}/stderr" status=0
|
||||
|
||||
if [ ${#CFLAGS[@]} -eq 0 ] || [ ${#LDFLAGS[@]} -eq 0 ]; then
|
||||
SKIPPED[nold]=$((SKIPPED[nold] + 1))
|
||||
return
|
||||
fi
|
||||
|
||||
wrap="$(attr run "${info}")"
|
||||
build_unit "${where}" "${body}" "${wrap}" "${doc}" "${line}" || return
|
||||
|
||||
rm -f "${prog}"
|
||||
if ! "${CC}" -std=gnu99 -Wall -Werror "${CFLAGS[@]}" \
|
||||
-o "${prog}" "${WORK}/unit.c" "${LDFLAGS[@]}" > "${log}" 2>&1; then
|
||||
fail "${where}" "does not build"
|
||||
sed -n '1,25p' "${log}" >&2
|
||||
return
|
||||
fi
|
||||
|
||||
rm -rf "${sandbox}"
|
||||
mkdir -p "${sandbox}"
|
||||
run_setup "${sandbox}" "$(attr setup "${info}")" "${where}" || return
|
||||
|
||||
# The same headless forcing scripts/memcheck.sh uses, so the answer does not
|
||||
# depend on whose desktop it ran on.
|
||||
( cd "${sandbox}" && \
|
||||
SDL_VIDEODRIVER=dummy SDL_RENDER_DRIVER=software SDL_AUDIODRIVER=dummy \
|
||||
"${prog}" ) > "${got}" 2> "${errout}"
|
||||
status=$?
|
||||
RAN[run]=$((RAN[run] + 1))
|
||||
|
||||
if [ "${status}" -ne 0 ]; then
|
||||
fail "${where}" "the example exited ${status}"
|
||||
sed -n '1,20p' "${got}" "${errout}" >&2
|
||||
return
|
||||
fi
|
||||
|
||||
# An error that was ignored, or one that reached libakerror's unhandled
|
||||
# handler, is a failure even where the process still exited 0. Both print a
|
||||
# recognisable line, and both mean the example did not do what it says.
|
||||
if grep -qE '\*\* IGNORED ERROR \*\*|Unhandled Error' "${errout}"; then
|
||||
fail "${where}" "the example raised an error it did not handle"
|
||||
grep -E '\*\* IGNORED ERROR \*\*|Unhandled Error' "${errout}" >&2
|
||||
return
|
||||
fi
|
||||
|
||||
# stdout only. libakgl logs registry activity to stderr -- three lines every
|
||||
# time a sprite is created -- and folding that into the comparison would put
|
||||
# library chatter into a chapter's expected output.
|
||||
if [ -n "${expected}" ]; then
|
||||
RAN[output]=$((RAN[output] + 1))
|
||||
if ! cmp -s "${expected}" "${got}"; then
|
||||
fail "${where}" "output does not match the block below it"
|
||||
show_diff "${expected}" "${got}"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# A declaration echoed from a header. Compiling it would redefine the type, so
|
||||
# the check is that it still says what the header says -- which is the drift
|
||||
# that actually happens, and the one a compile could not catch.
|
||||
function run_excerpt()
|
||||
{
|
||||
local where="$1" body="$2" path="$3"
|
||||
|
||||
if [ ! -r "${path}" ]; then
|
||||
fail "${where}" "excerpt=${path} names no such file"
|
||||
return
|
||||
fi
|
||||
RAN[excerpt]=$((RAN[excerpt] + 1))
|
||||
if ! normalise_c "${path}" | grep -qF -- "$(normalise_c "${body}")"; then
|
||||
fail "${where}" "no longer appears in ${path}"
|
||||
echo "--- the documentation says ---" >&2
|
||||
cat "${body}" >&2
|
||||
echo "--- ${path} has drifted from it; comments and whitespace are ignored ---" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
# An asset file, run through the loader that really reads it.
|
||||
#
|
||||
# The formats are documented in prose and read by src/sprite.c, src/character.c,
|
||||
# src/tilemap.c and src/registry.c, with nothing tying the two together. That gap
|
||||
# is not hypothetical: util/assets/littleguy.json is already invalid against the
|
||||
# loader it ships beside. tools/docs_checkjson.c closes it.
|
||||
function run_json()
|
||||
{
|
||||
local where="$1" body="$2" info="$3"
|
||||
local kind name preload preloads target
|
||||
local sandbox="${WORK}/json" log="${WORK}/json.log"
|
||||
local args=()
|
||||
|
||||
kind="$(attr kind "${info}")"
|
||||
if [ -z "${kind}" ]; then
|
||||
fail "${where}" "a json block with no kind= (sprite, character, tilemap or properties)"
|
||||
return
|
||||
fi
|
||||
|
||||
# A named block can be preloaded by a later one. Recorded whether or not the
|
||||
# validator is available, so omitting --checkjson does not turn a bad
|
||||
# preload= into a different failure.
|
||||
name="$(attr name "${info}")"
|
||||
if [ -n "${name}" ]; then
|
||||
JSON_KIND_OF["${name}"]="${kind}"
|
||||
JSON_BODY_OF["${name}"]="${body}"
|
||||
fi
|
||||
|
||||
if [ -z "${CHECKJSON}" ] || [ ! -x "${CHECKJSON}" ]; then
|
||||
SKIPPED[nojson]=$((SKIPPED[nojson] + 1))
|
||||
return
|
||||
fi
|
||||
|
||||
rm -rf "${sandbox}"
|
||||
mkdir -p "${sandbox}"
|
||||
run_setup "${sandbox}" "$(attr setup "${info}")" "${where}" || return
|
||||
|
||||
# Three of the four formats are not self-contained: a character names sprites
|
||||
# that have to be in the registry already, and a tilemap names characters.
|
||||
# `preload=` lists the earlier named blocks that have to be loaded first, in
|
||||
# order -- which puts the dependency in front of the reader rather than in a
|
||||
# setup script they cannot see.
|
||||
preloads="$(attr preload "${info}")"
|
||||
for preload in ${preloads//,/ }; do
|
||||
if [ -z "${JSON_KIND_OF[${preload}]:-}" ]; then
|
||||
fail "${where}" "preload=${preload} names no earlier \`json name=${preload}\` block in this document"
|
||||
return
|
||||
fi
|
||||
cp "${JSON_BODY_OF[${preload}]}" "${sandbox}/${preload}.json"
|
||||
args+=("${JSON_KIND_OF[${preload}]}" "${preload}.json")
|
||||
done
|
||||
|
||||
target="${name:-docs_example}"
|
||||
cp "${body}" "${sandbox}/${target}.json"
|
||||
args+=("${kind}" "${target}.json")
|
||||
|
||||
if ( cd "${sandbox}" && \
|
||||
SDL_VIDEODRIVER=dummy SDL_RENDER_DRIVER=software SDL_AUDIODRIVER=dummy \
|
||||
"${CHECKJSON}" "${args[@]}" ) > "${log}" 2>&1; then
|
||||
RAN[json]=$((RAN[json] + 1))
|
||||
else
|
||||
fail "${where}" "is not valid ${kind} JSON as far as the loader is concerned"
|
||||
sed -n '1,25p' "${log}" >&2
|
||||
fi
|
||||
}
|
||||
|
||||
# A shell command, in a sandbox.
|
||||
function run_sh()
|
||||
{
|
||||
local where="$1" body="$2" info="$3" expected="$4"
|
||||
local sandbox="${WORK}/sh" script="${WORK}/block.sh" got="${WORK}/got"
|
||||
|
||||
rm -rf "${sandbox}"
|
||||
mkdir -p "${sandbox}"
|
||||
run_setup "${sandbox}" "$(attr setup "${info}")" "${where}" || return
|
||||
|
||||
# A leading "$ " is a prompt in a transcript, not part of the command.
|
||||
sed -e 's/^\$ //' "${body}" > "${script}"
|
||||
if ! ( cd "${sandbox}" && bash -e "${script}" ) > "${got}" 2>&1; then
|
||||
fail "${where}" "the command failed"
|
||||
sed -n '1,20p' "${got}" >&2
|
||||
return
|
||||
fi
|
||||
RAN[sh]=$((RAN[sh] + 1))
|
||||
|
||||
if [ -n "${expected}" ]; then
|
||||
RAN[output]=$((RAN[output] + 1))
|
||||
if ! cmp -s "${expected}" "${got}"; then
|
||||
fail "${where}" "output does not match the block below it"
|
||||
show_diff "${expected}" "${got}"
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# ------------------------------------------------------------------ the pass
|
||||
for DOC in "${DOCS[@]}"; do
|
||||
BLOCKS="${WORK}/blocks"
|
||||
rm -rf "${BLOCKS}"
|
||||
mkdir -p "${BLOCKS}"
|
||||
: > "${BLOCKS}/index"
|
||||
extract "${DOC}" "${BLOCKS}"
|
||||
JSON_KIND_OF=()
|
||||
JSON_BODY_OF=()
|
||||
|
||||
# Read the index into arrays first. An `output` block belongs to the block
|
||||
# before it, so each iteration has to be able to look one ahead.
|
||||
CLAIMED=0
|
||||
IDS=(); LINES=(); INFOS=()
|
||||
while IFS=$'\t' read -r _id _line _info; do
|
||||
IDS+=("${_id}"); LINES+=("${_line}"); INFOS+=("${_info}")
|
||||
done < "${BLOCKS}/index"
|
||||
|
||||
for ((n = 0; n < ${#IDS[@]}; n++)); do
|
||||
INFO="${INFOS[n]}"
|
||||
BODY="${BLOCKS}/${IDS[n]}"
|
||||
WHERE="${DOC}:${LINES[n]}"
|
||||
KIND="${INFO%% *}"
|
||||
|
||||
# An `output` block belongs to the block above it, which claimed it on
|
||||
# the way past. One that nothing claimed is compared against nothing at
|
||||
# all -- indistinguishable from a passing test, and the exact failure
|
||||
# this harness exists to rule out. A *skipped* block still claims its
|
||||
# output: `norun` is a decision, not an accident.
|
||||
if [ "${KIND}" = "output" ]; then
|
||||
if [ "${CLAIMED}" -eq 0 ]; then
|
||||
fail "${WHERE}" "an output block with nothing runnable above it"
|
||||
fi
|
||||
CLAIMED=0
|
||||
continue
|
||||
fi
|
||||
CLAIMED=0
|
||||
|
||||
EXPECTED=""
|
||||
if [ $((n + 1)) -lt ${#IDS[@]} ] && [ "${INFOS[n + 1]%% *}" = "output" ]; then
|
||||
case "${KIND}" in
|
||||
sh)
|
||||
EXPECTED="${BLOCKS}/${IDS[n + 1]}"
|
||||
CLAIMED=1
|
||||
;;
|
||||
c)
|
||||
if [ -n "$(attr run "${INFO}")" ]; then
|
||||
EXPECTED="${BLOCKS}/${IDS[n + 1]}"
|
||||
CLAIMED=1
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
if has_word norun "${INFO}"; then
|
||||
SKIPPED[norun]=$((SKIPPED[norun] + 1))
|
||||
continue
|
||||
fi
|
||||
|
||||
# A `screenshot=NAME` block is the source of docs/images/NAME.png,
|
||||
# generated by tools/docs_screenshots.sh. The image is tracked, because a
|
||||
# reader on the forge has no build tree -- so nothing would otherwise
|
||||
# notice a figure that was tagged and never generated, and the chapter
|
||||
# would render a broken image. Whether the file is *of that listing* is
|
||||
# the docs_screenshots test's question, not this one's.
|
||||
SHOT="$(attr screenshot "${INFO}")"
|
||||
if [ -n "${SHOT}" ]; then
|
||||
if [ -r "docs/images/${SHOT}.png" ]; then
|
||||
RAN[screenshot]=$((RAN[screenshot] + 1))
|
||||
else
|
||||
fail "${WHERE}" "screenshot=${SHOT} has no docs/images/${SHOT}.png; run \`cmake --build build --target docs_screenshots\`"
|
||||
fi
|
||||
fi
|
||||
|
||||
case "${KIND}" in
|
||||
c)
|
||||
EXCERPT="$(attr excerpt "${INFO}")"
|
||||
if [ -n "${EXCERPT}" ]; then
|
||||
run_excerpt "${WHERE}" "${BODY}" "${EXCERPT}"
|
||||
elif [ -n "$(attr run "${INFO}")" ]; then
|
||||
run_c_run "${WHERE}" "${BODY}" "${INFO}" "${DOC}" "${LINES[n]}" "${EXPECTED}"
|
||||
elif [ -n "${SHOT}" ]; then
|
||||
# Compiled here as well as rendered there, so a figure's listing
|
||||
# is held to the same -Werror as every other snippet even in a
|
||||
# tree where the figures are not being regenerated. wrap= is
|
||||
# implied: a frame body is not a translation unit.
|
||||
run_c "${WHERE}" "${BODY}" "wrap=akglframe" "${DOC}" "${LINES[n]}"
|
||||
else
|
||||
run_c "${WHERE}" "${BODY}" "${INFO}" "${DOC}" "${LINES[n]}"
|
||||
fi
|
||||
;;
|
||||
json)
|
||||
run_json "${WHERE}" "${BODY}" "${INFO}"
|
||||
;;
|
||||
sh)
|
||||
run_sh "${WHERE}" "${BODY}" "${INFO}" "${EXPECTED}"
|
||||
;;
|
||||
cmake)
|
||||
SKIPPED[cmake]=$((SKIPPED[cmake] + 1))
|
||||
;;
|
||||
# A block diagram, or a stack trace captured from a real run. There is no
|
||||
# executable claim in either, so the tag exists to say that out loud
|
||||
# rather than to let one sit here untagged and unnoticed.
|
||||
text)
|
||||
SKIPPED[text]=$((SKIPPED[text] + 1))
|
||||
;;
|
||||
norun)
|
||||
SKIPPED[norun]=$((SKIPPED[norun] + 1))
|
||||
;;
|
||||
"")
|
||||
fail "${WHERE}" "fenced block with no info string; tag it (see docs/MAINTENANCE.md)"
|
||||
;;
|
||||
*)
|
||||
fail "${WHERE}" "unknown info string \"${INFO}\" (see docs/MAINTENANCE.md)"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
done
|
||||
|
||||
# The count is part of the result, not decoration. A harness that passes because
|
||||
# a chapter stopped matching the extractor looks exactly like a harness that
|
||||
# passes because the documentation is correct, and this is what tells them apart.
|
||||
# A rising `norun` is the harness being talked out of its job.
|
||||
echo "ran: ${RAN[c]} c snippets, ${RAN[run]} linked programs, ${RAN[output]} output comparisons, ${RAN[excerpt]} excerpts, ${RAN[json]} json blocks, ${RAN[sh]} shell blocks, ${RAN[screenshot]} figures"
|
||||
echo "skipped: ${SKIPPED[norun]} norun, ${SKIPPED[cmake]} cmake, ${SKIPPED[text]} text, ${SKIPPED[nocc]} c (no compiler flags given), ${SKIPPED[nold]} run (no linker flags given), ${SKIPPED[nojson]} json (no validator given)"
|
||||
|
||||
if [ "${FAILURES}" -eq 0 ]; then
|
||||
echo "PASS: every documented example does what the documentation says"
|
||||
fi
|
||||
exit "${FAILURES}"
|
||||
Reference in New Issue
Block a user