Execute every documented example as a test
Some checks failed
akbasic CI Build / cmake_build (push) Successful in 3m2s
akbasic CI Build / sanitizers (push) Successful in 3m52s
akbasic CI Build / coverage (push) Failing after 3m24s
akbasic CI Build / akgl_build (push) Failing after 20s
akbasic CI Build / mutation_test (push) Has been cancelled
Some checks failed
akbasic CI Build / cmake_build (push) Successful in 3m2s
akbasic CI Build / sanitizers (push) Successful in 3m52s
akbasic CI Build / coverage (push) Failing after 3m24s
akbasic CI Build / akgl_build (push) Failing after 20s
akbasic CI Build / mutation_test (push) Has been cancelled
docs/ and README.md carry 85 fenced blocks. Every one was checked by hand exactly once, when it was written, which is not a standard that survives a changing interpreter -- and four were already wrong: two transcripts showing a leading space PRINT does not emit, akbasic_TextSink in README.md missing the two members it had grown hours earlier, and FILTER's refusal quoted with wording the code does not use. tests/docs_examples.sh reads a fence-tag vocabulary and runs what it finds. BASIC programs and transcripts run and are byte-compared against an `output` block; C snippets compile with -fsyntax-only against the real include path, which CMake writes out because it is transitive through akerror, akstdlib and akgl; shell blocks run in a sandbox. Anything that would reconfigure the build tree, hit the network or re-enter the suite is tagged norun with the reason in MAINTENANCE.md, and the two cmake blocks stay hand-maintained by decision. An untagged block is a failure rather than a default, and the pass line reports what it executed by kind. Both exist because the way a harness like this dies is by quietly matching nothing and passing -- which it duly did on the first CTest run, where a generator expression evaluating to nothing still contributed an empty argument that the script read as a filename. The count is what caught it. The excerpt check earns its own mention: a block tagged `c excerpt=include/akbasic/sink.h` must still appear in that header, comments and whitespace ignored. Compiling it would only redefine the type, so a compile check could not have found the stale struct, and did not. Registered as the CTest case docs_examples in both configurations. Fixing the four wrong examples turned up two interpreter defects, fixed in the previous commit and recorded in TODO.md section 8. MAINTENANCE.md is new: the fence-tag reference, what to do when the case fails, and the conventions that until now only existed inside source comments -- the three test lists and how two of them invert "passed", the sorted verb table, that a golden file is never edited to suit this interpreter, and that a fix gets mutation-checked with a file copy rather than git checkout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -33,7 +33,7 @@ The default build has no dependency on SDL and no graphics, sound, sprites or wi
|
||||
text. Everything else works, and the whole test suite runs on a machine with no SDL
|
||||
installed at all.
|
||||
|
||||
```sh
|
||||
```sh norun
|
||||
cmake -S . -B build
|
||||
cmake --build build
|
||||
```
|
||||
@@ -42,7 +42,7 @@ The SDL build adds a window, the Commodore font, and the graphics, sound and spr
|
||||
devices. It needs [libakgl](https://source.starfort.tech/andrew/libakgl), which is
|
||||
vendored as a submodule.
|
||||
|
||||
```sh
|
||||
```sh norun
|
||||
git submodule update --init --recursive
|
||||
cmake -S . -B build-akgl -DAKBASIC_WITH_AKGL=ON
|
||||
cmake --build build-akgl
|
||||
@@ -51,8 +51,13 @@ cmake --build build-akgl
|
||||
A verb that needs a device the build does not have does not crash and does not lie. It
|
||||
reports itself by name:
|
||||
|
||||
```basic requires=noakgl
|
||||
10 DRAW 1, 0, 0 TO 100, 100
|
||||
```
|
||||
|
||||
```output
|
||||
? 10 : RUNTIME ERROR DRAW needs a graphics device and this runtime has none
|
||||
|
||||
```
|
||||
|
||||
That is the same message an embedded host sees when it deliberately withholds a
|
||||
@@ -60,7 +65,7 @@ device — a game may want a script that can print but not draw.
|
||||
|
||||
## Running the tests
|
||||
|
||||
```sh
|
||||
```sh norun
|
||||
ctest --test-dir build --output-on-failure
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user