3 Commits
34 ... 10

Author SHA1 Message Date
d715bc0625 Report prescan errors on their source lines
Some checks failed
akbasic CI Build / coverage (push) Failing after 27s
akbasic CI Build / cmake_build (push) Failing after 33s
akbasic CI Build / akgl_build (push) Failing after 18s
akbasic CI Build / sanitizers (push) Failing after 1m7s
akbasic CI Build / mutation_test (push) Failing after 39s
2026-08-05 06:40:14 -04:00
350deb3a45 Merge pull request 'Move BASIC fixtures into the editable language corpus' (#42) from 40 into main
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 4m2s
akbasic CI Build / sanitizers (push) Successful in 4m59s
akbasic CI Build / coverage (push) Successful in 4m35s
akbasic CI Build / akgl_build (push) Successful in 9m17s
akbasic CI Build / mutation_test (push) Successful in 29m15s
Reviewed-on: #42
2026-08-04 16:30:25 -04:00
8a02674af5 Move BASIC fixtures into the editable language corpus
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s
Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths.

Co-authored-by: andrew <andrew@aklabs.net>
2026-08-04 16:22:16 -04:00
103 changed files with 167 additions and 172 deletions

View File

@@ -17,7 +17,7 @@ jobs:
# Not recursive, deliberately. The top-level build needs
# deps/libakerror and deps/libakstdlib, via add_subdirectory, and
# nothing else: the golden corpus and the Commodore font now live in
# this repository (tests/reference/ and assets/fonts/), so
# this repository (tests/language/ and assets/fonts/), so
# deps/basicinterpret is no longer a build dependency at all.
# It does *not* need deps/libakgl, which is guarded behind
# AKBASIC_WITH_AKGL and defaults OFF -- and recursing into it would
@@ -62,11 +62,9 @@ jobs:
run: |
cmake -S . -B build
cmake --build build --parallel 2
# The suite is 78 cases: 41 golden files byte-compared against the Go
# reference's own corpus (checked in at tests/reference/, see its README),
# 9 local golden cases for verbs the reference never implemented, 25 unit
# tests, 2 embedding examples, and 1 known-failing test that asserts the
# *correct* contract for defects carried over from the reference (TODO.md
# The suite is 112 cases: 65 language files with sibling expectations,
# 43 unit tests, 3 embedding examples, and docs_examples.
# Some unit tests assert the *correct* contract for known defects (TODO.md
# section 6). A green run therefore does not mean defect-free -- see
# AKBASIC_KNOWN_FAILING_TESTS.
#

View File

@@ -57,9 +57,9 @@ repeating where you will see them:
`include/akgl/SDL_GameControllerDB.h`. Change the template or the generator script.
- **Do not reformat code you are not otherwise changing.** Several files mix tabs and spaces
and there is no repo-wide formatter; style conversions get their own commit.
- **Do not edit `tests/reference/`.** Those expectations came from the Go implementation and
are never edited to suit this interpreter. A deliberate divergence goes in
`tests/reference/README.md`'s divergence table and `docs/13-differences.md`.
- **Keep `tests/language/` editable.** Its `.bas` programs and sibling `.txt` expectations
are changed together when behavior changes. Record deliberate language decisions in
`TODO.md` or `docs/13-differences.md`.
- **Open an issue for outstanding work; do not add it to `TODO.md`.**
<https://source.starfort.tech/andrew/akbasic/issues>, or `tea issues create --repo
andrew/akbasic`. Name the file and line, the functional consequence, and what closing it would

View File

@@ -448,7 +448,7 @@ if(AKBASIC_WITH_AKGL)
# against.
#
# **A byte comparison of a rendered PNG is a deliberate bet**, the same bet
# tests/reference/ already makes about golden output: that the dummy video
# the language corpus makes about golden output: that the dummy video
# driver and the software renderer are reproducible. They are, run to run and
# build to build. What is untested is an SDL upgrade that shifts one pixel of
# a diagonal, and the answer to that is to regenerate the figures in the same
@@ -531,63 +531,20 @@ if(AKBASIC_WILL_FAIL_TESTS OR AKBASIC_KNOWN_FAILING_TESTS)
)
endif()
# The reference's own corpus, byte-compared against the sibling .txt. One CTest
# case per .bas so a failure names the file.
# The editable language corpus. One CTest case per .bas so a failure names the
# file. Each program is paired with a sibling .txt expectation; see
# tests/language/README.md for the editing rule.
#
# It used to be driven in place out of deps/basicinterpret, on the reasoning that
# copying a submodule's corpus guarantees drift. That reasoning was sound and it
# has been overruled deliberately: the Go dependency is being deprecated, and a
# build that cannot run its acceptance suite without cloning the implementation
# it replaced is not finished. The copy is byte-identical to
# basicinterpreter@d76162c and tests/reference/README.md records
# where it came from and what the drift now costs.
file(GLOB_RECURSE AKBASIC_GOLDEN_CASES
RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}/tests/reference"
"${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/*.bas"
)
foreach(_case IN LISTS AKBASIC_GOLDEN_CASES)
string(REGEX REPLACE "^tests/" "" _name "${_case}")
string(REGEX REPLACE "\\.bas$" "" _name "${_name}")
string(REPLACE "/" "_" _name "${_name}")
_add_test(
NAME golden_${_name}
COMMAND ${CMAKE_COMMAND}
-DBASIC=$<TARGET_FILE:basic>
-DCASE=${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/${_case}
-P ${CMAKE_CURRENT_SOURCE_DIR}/tests/golden.cmake
)
endforeach()
if(AKBASIC_GOLDEN_CASES)
set(AKBASIC_GOLDEN_NAMES)
foreach(_case IN LISTS AKBASIC_GOLDEN_CASES)
string(REGEX REPLACE "^tests/" "" _name "${_case}")
string(REGEX REPLACE "\\.bas$" "" _name "${_name}")
string(REPLACE "/" "_" _name "${_name}")
list(APPEND AKBASIC_GOLDEN_NAMES golden_${_name})
endforeach()
_set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES TIMEOUT 30)
# An AKGL build of `basic` opens a window, and forty-one of them is not what
# anybody running the suite wanted. The dummy driver produces the same stdout,
# which is the only thing a golden case compares.
if(AKBASIC_WITH_AKGL)
_set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES
ENVIRONMENT "SDL_VIDEODRIVER=dummy;SDL_AUDIODRIVER=dummy;SDL_RENDER_DRIVER=software")
endif()
endif()
# The local golden corpus, for verbs the reference never implemented.
# The corpus includes programs carried over from the deprecated Go implementation
# and cases written for this interpreter. Their provenance is useful when
# investigating a regression, but it does not make any case immutable.
#
# Still separate now that the reference's corpus lives in this repository too,
# and the reason changed rather than went away: tests/reference/ is a *record* of
# what the Go implementation did and nothing in it should ever be edited to suit
# this one, while tests/language/ is ours to change. Registered under local_ so a
# failure says at a glance which of the two it came from -- and so a diff that
# touches tests/reference/ stands out as the thing it is.
# Programs carried over from the deprecated Go implementation and cases written
# for this interpreter use the same editable `.bas`/`.txt` contract. Registered
# under local_ so every failure identifies the program that produced it.
#
# Note what this can and cannot cover. The graphics and sound verbs draw and play
# rather than print, so what a golden file sees of them is their *refusals* and
# rather than print, so what a golden file sees of them is their refusals and
# whatever a program can PRINT about the state they changed. The behaviour that
# reaches a device is asserted against tests/mockdevice.h instead.
file(GLOB_RECURSE AKBASIC_LOCAL_CASES

View File

@@ -88,7 +88,7 @@ source stays readable as documentation of what the original did. Neither is bind
**It is not a build or test dependency.** Both configurations have been configured, built and
run from scratch with it moved out of the tree. Its acceptance corpus is checked in at
`tests/reference/` and its Commodore font at `assets/fonts/`.
`tests/language/` and its Commodore font at `assets/fonts/`.
```sh norun
cd deps/basicinterpret
@@ -351,15 +351,10 @@ name. That is not cosmetic: `add_executable` creates a dependency's targets even
### The golden corpora
`tests/reference/` is the Go implementation's own acceptance suite, byte-compared.
**Nothing in it is ever edited to suit this interpreter.** If a case fails, either this
interpreter is wrong or the divergence is deliberate — and a deliberate one goes in
`tests/reference/README.md`'s divergence table and `docs/13-differences.md`, not into the