Move BASIC fixtures into the editable language corpus
All checks were successful
akbasic CI Build / cmake_build (push) Successful in 3m34s
akbasic CI Build / coverage (push) Successful in 4m4s
akbasic CI Build / sanitizers (push) Successful in 6m59s
akbasic CI Build / akgl_build (push) Successful in 7m57s
akbasic CI Build / mutation_test (push) Successful in 23m28s

Move every program and expectation out of tests/reference and register the unified tests/language corpus as local cases. Remove the old immutable-corpus protections from build, maintenance, and documentation paths.

Co-authored-by: andrew <andrew@aklabs.net>
This commit is contained in:
2026-08-04 16:21:59 -04:00
parent fac84acdaa
commit 8a02674af5
96 changed files with 69 additions and 168 deletions

View File

@@ -448,7 +448,7 @@ if(AKBASIC_WITH_AKGL)
# against.
#
# **A byte comparison of a rendered PNG is a deliberate bet**, the same bet
# tests/reference/ already makes about golden output: that the dummy video
# the language corpus makes about golden output: that the dummy video
# driver and the software renderer are reproducible. They are, run to run and
# build to build. What is untested is an SDL upgrade that shifts one pixel of
# a diagonal, and the answer to that is to regenerate the figures in the same
@@ -531,63 +531,20 @@ if(AKBASIC_WILL_FAIL_TESTS OR AKBASIC_KNOWN_FAILING_TESTS)
)
endif()
# The reference's own corpus, byte-compared against the sibling .txt. One CTest
# case per .bas so a failure names the file.
# The editable language corpus. One CTest case per .bas so a failure names the
# file. Each program is paired with a sibling .txt expectation; see
# tests/language/README.md for the editing rule.
#
# It used to be driven in place out of deps/basicinterpret, on the reasoning that
# copying a submodule's corpus guarantees drift. That reasoning was sound and it
# has been overruled deliberately: the Go dependency is being deprecated, and a
# build that cannot run its acceptance suite without cloning the implementation
# it replaced is not finished. The copy is byte-identical to
# basicinterpreter@d76162c and tests/reference/README.md records
# where it came from and what the drift now costs.
file(GLOB_RECURSE AKBASIC_GOLDEN_CASES
RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}/tests/reference"
"${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/*.bas"
)
foreach(_case IN LISTS AKBASIC_GOLDEN_CASES)
string(REGEX REPLACE "^tests/" "" _name "${_case}")
string(REGEX REPLACE "\\.bas$" "" _name "${_name}")
string(REPLACE "/" "_" _name "${_name}")
_add_test(
NAME golden_${_name}
COMMAND ${CMAKE_COMMAND}
-DBASIC=$<TARGET_FILE:basic>
-DCASE=${CMAKE_CURRENT_SOURCE_DIR}/tests/reference/${_case}
-P ${CMAKE_CURRENT_SOURCE_DIR}/tests/golden.cmake
)
endforeach()
if(AKBASIC_GOLDEN_CASES)
set(AKBASIC_GOLDEN_NAMES)
foreach(_case IN LISTS AKBASIC_GOLDEN_CASES)
string(REGEX REPLACE "^tests/" "" _name "${_case}")
string(REGEX REPLACE "\\.bas$" "" _name "${_name}")
string(REPLACE "/" "_" _name "${_name}")
list(APPEND AKBASIC_GOLDEN_NAMES golden_${_name})
endforeach()
_set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES TIMEOUT 30)
# An AKGL build of `basic` opens a window, and forty-one of them is not what
# anybody running the suite wanted. The dummy driver produces the same stdout,
# which is the only thing a golden case compares.
if(AKBASIC_WITH_AKGL)
_set_tests_properties(${AKBASIC_GOLDEN_NAMES} PROPERTIES
ENVIRONMENT "SDL_VIDEODRIVER=dummy;SDL_AUDIODRIVER=dummy;SDL_RENDER_DRIVER=software")
endif()
endif()
# The local golden corpus, for verbs the reference never implemented.
# The corpus includes programs carried over from the deprecated Go implementation
# and cases written for this interpreter. Their provenance is useful when
# investigating a regression, but it does not make any case immutable.
#
# Still separate now that the reference's corpus lives in this repository too,
# and the reason changed rather than went away: tests/reference/ is a *record* of
# what the Go implementation did and nothing in it should ever be edited to suit
# this one, while tests/language/ is ours to change. Registered under local_ so a
# failure says at a glance which of the two it came from -- and so a diff that
# touches tests/reference/ stands out as the thing it is.
# Programs carried over from the deprecated Go implementation and cases written
# for this interpreter use the same editable `.bas`/`.txt` contract. Registered
# under local_ so every failure identifies the program that produced it.
#
# Note what this can and cannot cover. The graphics and sound verbs draw and play
# rather than print, so what a golden file sees of them is their *refusals* and
# rather than print, so what a golden file sees of them is their refusals and
# whatever a program can PRINT about the state they changed. The behaviour that
# reaches a device is asserted against tests/mockdevice.h instead.
file(GLOB_RECURSE AKBASIC_LOCAL_CASES