Cover -DAKBASIC_WITH_AKGL=ON in CI
A fifth push job, akgl_build: the text sink and the graphics, audio and input backends, plus the akgl_backends suite that drives them against a real software renderer under the dummy video and audio drivers and reads the pixels back. It stays a separate job rather than a flag on cmake_build because AKBASIC_WITH_AKGL is off by default and that default is the claim being made: the interpreter and its whole 70-case suite build and pass on a machine with no SDL. Keeping the two jobs apart proves that on every push instead of asserting it. Deferring this assumed it needed submodules: recursive and a long build. Both guesses were wrong, and measurement is the only reason we know: - Recursive is not needed and costs 461 MB. It descends into SDL_image/external, SDL_mixer/external and SDL_ttf/external -- aom, dav1d, libjxl, libtiff, mpg123, opus, flac and a dozen more -- and not one of them is used. Every one configures as "Could NOT find". Six submodules initialised non-recursively take about 25 seconds instead. - The build is roughly a minute of CPU across 330 object files, because SDL3 compiles out almost every backend that is not wanted here. The one real system dependency is libfreetype-dev and libharfbuzz-dev. SDL_ttf reports "Using system freetype library" and links libfreetype.so.6; without them it would reach for deps/SDL_ttf/external/freetype, which the checkout deliberately does not clone. Two apt packages against two more submodules. Every step was run verbatim from a clean clone before being written down -- checkout, submodule init, configure, build and test -- rather than inferred from the working tree. 71/71. Also corrects three counts elsewhere in the file that had gone stale: the suite is 70 cases rather than 61, line coverage is 93.5% rather than 92.3%, and branch coverage reads 18% rather than 17%. Those numbers are assertions about the gate, so a wrong one is worse than none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -32,12 +32,17 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
cmake -S . -B build
|
cmake -S . -B build
|
||||||
cmake --build build --parallel
|
cmake --build build --parallel
|
||||||
# The suite is 61 cases: 41 golden files byte-compared against the Go
|
# The suite is 70 cases: 41 golden files byte-compared against the Go
|
||||||
# reference's own corpus, 17 unit tests, 2 embedding examples, and 1
|
# reference's own corpus, 5 local golden cases for verbs the reference
|
||||||
|
# never implemented, 21 unit tests, 2 embedding examples, and 1
|
||||||
# known-failing test that asserts the *correct* contract for defects
|
# known-failing test that asserts the *correct* contract for defects
|
||||||
# carried over from the reference (TODO.md section 6). A green run
|
# carried over from the reference (TODO.md section 6). A green run
|
||||||
# therefore does not mean defect-free -- see AKBASIC_KNOWN_FAILING_TESTS.
|
# therefore does not mean defect-free -- see AKBASIC_KNOWN_FAILING_TESTS.
|
||||||
#
|
#
|
||||||
|
# AKBASIC_WITH_AKGL is off here, which is the point rather than an
|
||||||
|
# omission: this job is what proves the interpreter builds and passes on a
|
||||||
|
# machine with no SDL. The akgl_build job below covers the other half.
|
||||||
|
#
|
||||||
# --output-junit resolves relative to the test dir, so give an absolute
|
# --output-junit resolves relative to the test dir, so give an absolute
|
||||||
# path to land the report in the workspace root for the reporter below.
|
# path to land the report in the workspace root for the reporter below.
|
||||||
- name: test (JUnit)
|
- name: test (JUnit)
|
||||||
@@ -91,11 +96,15 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
sudo apt-get update -y
|
sudo apt-get update -y
|
||||||
sudo apt-get install -y cmake gcc gcovr
|
sudo apt-get install -y cmake gcc gcovr
|
||||||
# The gate is a ratchet, not a target: src/ sits at 92.3% of lines and
|
# The gate is a ratchet, not a target: src/ sits at 93.5% of lines and
|
||||||
# 96.9% of functions, so 90 fails on a real regression (a test deleted, or
|
# 97.8% of functions, so 90 fails on a real regression (a test deleted, or
|
||||||
# new untested code added) without tripping over rounding.
|
# new untested code added) without tripping over rounding.
|
||||||
#
|
#
|
||||||
# There is deliberately no branch gate. Branch coverage reads about 17%
|
# The akbasic_akgl target is not in this figure. It is not built in a
|
||||||
|
# default configuration, and instrumenting it would drag SDL into the
|
||||||
|
# coverage job for four thin adaptors.
|
||||||
|
#
|
||||||
|
# There is deliberately no branch gate. Branch coverage reads about 18%
|
||||||
# and is not a meaningful number here, for the reason libakgl's and
|
# and is not a meaningful number here, for the reason libakgl's and
|
||||||
# libakstdlib's TODO.md both record: the akerror control-flow macros expand
|
# libakstdlib's TODO.md both record: the akerror control-flow macros expand
|
||||||
# into large branch trees at every call site, most of them unreachable in
|
# into large branch trees at every call site, most of them unreachable in
|
||||||
@@ -130,6 +139,84 @@ jobs:
|
|||||||
if-no-files-found: warn
|
if-no-files-found: warn
|
||||||
- run: echo "🍏 This job's status is ${{ job.status }}."
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|
||||||
|
|
||||||
|
akgl_build:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Check out repository code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
# submodules: true, *not* recursive, and then the libakgl dependencies
|
||||||
|
# by hand in the next step. Recursive would work and costs an extra
|
||||||
|
# 461 MB: it descends into SDL_image/external, SDL_mixer/external and
|
||||||
|
# SDL_ttf/external, which are aom, dav1d, libjxl, libtiff, mpg123,
|
||||||
|
# opus, flac, freetype, harfbuzz and a dozen more. None of them is
|
||||||
|
# used -- every one configures as "Could NOT find" or falls back to
|
||||||
|
# the system copy -- so cloning them is pure checkout time.
|
||||||
|
submodules: true
|
||||||
|
# libakgl builds its vendored SDL only when it is the top-level project;
|
||||||
|
# embedded it takes a find_package path instead, so our CMakeLists declares
|
||||||
|
# those targets first and needs the submodules present. Six of them, none
|
||||||
|
# recursive. Filed upstream as libakgl API-gap item 5.
|
||||||
|
- name: libakgl dependencies
|
||||||
|
run: |
|
||||||
|
git -C deps/libakgl submodule update --init \
|
||||||
|
deps/SDL deps/SDL_image deps/SDL_mixer deps/SDL_ttf \
|
||||||
|
deps/jansson deps/semver
|
||||||
|
# libfreetype-dev and libharfbuzz-dev are load-bearing, not incidental.
|
||||||
|
# SDL_ttf prefers the system copies -- it reports "Using system freetype
|
||||||
|
# library" and links libfreetype.so.6 -- and without them it would reach
|
||||||
|
# for deps/SDL_ttf/external/freetype, which the checkout above
|
||||||
|
# deliberately does not clone. Installing two dev packages is much cheaper
|
||||||
|
# than cloning freetype and harfbuzz.
|
||||||
|
- name: dependencies
|
||||||
|
run: |
|
||||||
|
sudo apt-get update -y
|
||||||
|
sudo apt-get install -y cmake gcc g++ pkg-config \
|
||||||
|
libfreetype-dev libharfbuzz-dev
|
||||||
|
# The akgl-backed half: the text sink and the graphics, audio and input
|
||||||
|
# backends, plus the akgl_backends suite that drives them against a real
|
||||||
|
# software renderer and reads the pixels back.
|
||||||
|
#
|
||||||
|
# It is a separate job rather than a flag on cmake_build because
|
||||||
|
# AKBASIC_WITH_AKGL is off by default and that default is the point: the
|
||||||
|
# interpreter and its whole 70-case suite build and pass on a machine with
|
||||||
|
# no SDL. Keeping the two apart is what proves that claim on every push
|
||||||
|
# rather than asserting it.
|
||||||
|
#
|
||||||
|
# Cost, measured rather than guessed: about 25 s to clone the six
|
||||||
|
# submodules, 25 s to configure and 330 object files to compile -- roughly
|
||||||
|
# a minute of CPU. Cheaper than it looks, because SDL3 compiles out almost
|
||||||
|
# every backend it does not need here.
|
||||||
|
- name: build with libakgl
|
||||||
|
run: |
|
||||||
|
cmake -S . -B build-akgl -DAKBASIC_WITH_AKGL=ON
|
||||||
|
cmake --build build-akgl --parallel
|
||||||
|
# Dummy video and audio drivers: the suite creates a 128x128 software
|
||||||
|
# renderer and reads it back with SDL_RenderReadPixels, so it needs no
|
||||||
|
# display, no sound card and no offscreen harness. Same approach
|
||||||
|
# deps/libakgl/tests/draw.c takes.
|
||||||
|
#
|
||||||
|
# 71 cases: the 70 the default build runs, plus akgl_backends. The other
|
||||||
|
# 70 are run here too on purpose -- they are the ones that must keep
|
||||||
|
# passing when SDL *is* present, and a linker that picked up the wrong
|
||||||
|
# akstdlib or akerror would show up here first.
|
||||||
|
- name: test with libakgl
|
||||||
|
env:
|
||||||
|
SDL_VIDEODRIVER: dummy
|
||||||
|
SDL_AUDIODRIVER: dummy
|
||||||
|
run: ctest --test-dir build-akgl --output-on-failure --output-junit "$(pwd)/ctest-akgl-junit.xml"
|
||||||
|
- name: publish test results
|
||||||
|
if: always()
|
||||||
|
uses: mikepenz/action-junit-report@v4
|
||||||
|
with:
|
||||||
|
report_paths: 'ctest-akgl-junit.xml'
|
||||||
|
annotate_only: true
|
||||||
|
detailed_summary: true
|
||||||
|
include_passed: true
|
||||||
|
fail_on_failure: 'true'
|
||||||
|
check_name: 'akgl test results'
|
||||||
|
- run: echo "🍏 This job's status is ${{ job.status }}."
|
||||||
|
|
||||||
mutation_test:
|
mutation_test:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
|
|||||||
24
TODO.md
24
TODO.md
@@ -798,7 +798,7 @@ entire test corpus and passes clean under ASan and UBSan.
|
|||||||
| Gate | Result |
|
| Gate | Result |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `ctest` | 70/70 — 41 upstream golden cases, 5 local ones, 21 unit tests, 2 embedding examples, 1 known-failing |
|
| `ctest` | 70/70 — 41 upstream golden cases, 5 local ones, 21 unit tests, 2 embedding examples, 1 known-failing |
|
||||||
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 71/71 — the above plus `akgl_backends` |
|
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 71/71 — the above plus `akgl_backends`; the `akgl_build` CI job |
|
||||||
| Golden corpus | 41/41 byte-exact, driven in place from the submodule |
|
| Golden corpus | 41/41 byte-exact, driven in place from the submodule |
|
||||||
| ASan + UBSan | 70/70 |
|
| ASan + UBSan | 70/70 |
|
||||||
| Line coverage | 93.5% (3631/3884) — above the 90% gate |
|
| Line coverage | 93.5% (3631/3884) — above the 90% gate |
|
||||||
@@ -880,11 +880,23 @@ What remains, in priority order:
|
|||||||
2. **A line editor for the akgl sink.** `readline` reports EOF, so `INPUT` does not work
|
2. **A line editor for the akgl sink.** `readline` reports EOF, so `INPUT` does not work
|
||||||
through a drawn text layer. It wants the keystroke ring plus a cursor, and it is also what
|
through a drawn text layer. It wants the keystroke ring plus a cursor, and it is also what
|
||||||
`WINDOW` and `KEY` in group E need. See §3.
|
`WINDOW` and `KEY` in group E need. See §3.
|
||||||
3. **CI does not cover `-DAKBASIC_WITH_AKGL=ON`.** Doing so needs `submodules: recursive`,
|
3. ~~**CI does not cover `-DAKBASIC_WITH_AKGL=ON`.**~~ **Done** — the `akgl_build` job in
|
||||||
which pulls SDL, SDL_image, SDL_mixer, SDL_ttf and jansson — a long build on a job the other
|
`.gitea/workflows/ci.yaml`. It turned out much cheaper than the deferral assumed, and the
|
||||||
four do not need. Deliberately deferred, not forgotten: the akgl target is built and tested
|
two reasons are worth keeping because both were guesses that measurement corrected:
|
||||||
locally with the two commands in §3, and it will rot silently until somebody decides that
|
|
||||||
cost is worth paying.
|
- **`submodules: recursive` is not needed and costs 461 MB.** It descends into
|
||||||
|
`SDL_image/external`, `SDL_mixer/external` and `SDL_ttf/external` — aom, dav1d, libjxl,
|
||||||
|
libtiff, mpg123, opus, flac and a dozen more — and *not one of them is used*. Every one
|
||||||
|
configures as "Could NOT find". The job initialises exactly six submodules
|
||||||
|
non-recursively instead, which takes about 25 seconds.
|
||||||
|
- **The build is about a minute of CPU, not "a long build".** 330 object files, because
|
||||||
|
SDL3 compiles out almost every backend that is not wanted here.
|
||||||
|
|
||||||
|
The one real system dependency is `libfreetype-dev` and `libharfbuzz-dev`: SDL_ttf prefers
|
||||||
|
the system copies and would otherwise reach for `external/freetype`, which the checkout
|
||||||
|
deliberately does not clone. Two apt packages against two more submodules is an easy trade.
|
||||||
|
|
||||||
|
Every step was verified from a clean clone before being written down, not inferred.
|
||||||
4. **§6 items 12–17** — the defects the port uncovered. Item 12 is the one that produces a
|
4. **§6 items 12–17** — the defects the port uncovered. Item 12 is the one that produces a
|
||||||
*wrong answer* rather than a refused one, and should be fixed first.
|
*wrong answer* rather than a refused one, and should be fixed first.
|
||||||
5. **Mutation survivors in `src/value.c`.** The harness is in place (`scripts/mutation_test.py`,
|
5. **Mutation survivors in `src/value.c`.** The harness is in place (`scripts/mutation_test.py`,
|
||||||
|
|||||||
Reference in New Issue
Block a user