Type at the window with a real keyboard: the akgl_typing test
Some checks failed
akbasic CI Build / cmake_build (push) Failing after 18s
akbasic CI Build / sanitizers (push) Failing after 13s
akbasic CI Build / coverage (push) Failing after 16s
akbasic CI Build / akgl_build (push) Failing after 17s
akbasic CI Build / mutation_test (push) Failing after 17s

xdotool closes the gap that shipped the missing SDL_StartTextInput(). Every
other keyboard test synthesises SDL events, which covers the code downstream
of SDL and cannot cover the code upstream of it -- so the suite stayed green
while the real keyboard was dead.

akgl_typing starts the driver under a pty, waits for the window with xdotool
search --sync, gives it focus, and types a program containing a string literal
and a lower-case one, polling the mirrored stdout for what they print. Verified
by reverting both halves of the text-input fix and watching it fail with the
reported symptom: READY, and nothing after it.

Skips rather than fails without a display, xdotool, script(1) or a window
manager -- none of those means the answer is no. It steals keyboard focus for
about fifteen seconds; AKBASIC_SKIP_INTERACTIVE=1 skips it deliberately.

xdotool and script(1) are documented as optional test dependencies, alongside
what gcovr and python3 already bought.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 14:35:07 -04:00
parent 024a33dbba
commit cfdde907df
6 changed files with 195 additions and 6 deletions

View File

@@ -214,6 +214,11 @@ jobs:
# they are the ones that must keep passing when SDL is present, *and* they # they are the ones that must keep passing when SDL is present, *and* they
# are what proves the SDL frontend changes no output anywhere in the # are what proves the SDL frontend changes no output anywhere in the
# corpus. # corpus.
# akgl_typing reports Skipped here and that is correct: it needs a real X
# server, a window manager and xdotool to type at a focused window, and a
# CI runner has none of the three. It is a developer-machine gate, and the
# one that covers the path upstream of SDL -- see its script for why that
# distinction cost a bug once.
- name: test with libakgl - name: test with libakgl
env: env:
SDL_VIDEODRIVER: dummy SDL_VIDEODRIVER: dummy

View File

@@ -265,6 +265,12 @@ ctest --test-dir build --output-on-failure
ctest --test-dir build -R <name> --output-on-failure # single test ctest --test-dir build -R <name> --output-on-failure # single test
``` ```
**Optional test tooling for this repository.** `xdotool` and `script`(1) drive the `akgl_typing`
test, which types at a real focused SDL window -- the only test that covers the path upstream of
SDL, and the one that would have caught the missing `SDL_StartTextInput()`. It needs a real X
server, steals keyboard focus for about fifteen seconds, and skips itself (CTest `Skipped`, not
a failure) when it cannot run. `AKBASIC_SKIP_INTERACTIVE=1` skips it deliberately.
Extra harnesses, by library (option prefixes differ per repo): Extra harnesses, by library (option prefixes differ per repo):
```sh ```sh

View File

@@ -317,6 +317,34 @@ if(AKBASIC_WITH_AKGL)
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/tests" WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/tests"
TIMEOUT 60 TIMEOUT 60
ENVIRONMENT "SDL_VIDEODRIVER=dummy;SDL_AUDIODRIVER=dummy;SDL_RENDER_DRIVER=software") ENVIRONMENT "SDL_VIDEODRIVER=dummy;SDL_AUDIODRIVER=dummy;SDL_RENDER_DRIVER=software")
# The keyboard test, and the only one in the repository that uses a real X
# server rather than the dummy driver. Note the deliberate absence of an
# ENVIRONMENT line: forcing the dummy driver here would defeat the whole point.
#
# Everything else synthesises SDL events, which tests the code *downstream* of
# SDL and cannot test the code upstream of it. That gap shipped a bug -- the
# frontend never called SDL_StartTextInput(), so SDL emitted no text-input
# events, the line editor dropped every keystroke, and the suite stayed green
# because it was pushing those events itself. This one drives xdotool at a
# focused window, so X11, SDL's composition and the window manager are all in
# the path.
#
# Needs xdotool and script(1), and it **steals keyboard focus** for a few
# seconds. Skipped rather than failed when it cannot run -- headless, no
# xdotool, no window manager -- because none of those means the answer is no.
# AKBASIC_SKIP_INTERACTIVE=1 skips it while you are using the machine.
find_program(XDOTOOL_EXECUTABLE xdotool)
if(NOT XDOTOOL_EXECUTABLE)
message(STATUS "xdotool not found: the akgl_typing test will skip. "
"Install it to cover the real keyboard path.")
endif()
_add_test(NAME akgl_typing
COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/tests/akgl_typing.sh $<TARGET_FILE:basic>)
_set_tests_properties(akgl_typing PROPERTIES
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
TIMEOUT 180
SKIP_RETURN_CODE 77)
endif() endif()
if(AKBASIC_TESTS OR AKBASIC_WILL_FAIL_TESTS OR AKBASIC_KNOWN_FAILING_TESTS) if(AKBASIC_TESTS OR AKBASIC_WILL_FAIL_TESTS OR AKBASIC_KNOWN_FAILING_TESTS)

View File

@@ -28,6 +28,30 @@ ctest --test-dir build-akgl --output-on-failure
./build-akgl/basic tests/reference/language/functions.bas ./build-akgl/basic tests/reference/language/functions.bas
``` ```
### Optional tools, and what you lose without them
Everything above works with cmake and a C compiler. Three tools are optional, and each one buys
a specific test rather than general convenience:
| Tool | Buys | Without it |
|---|---|---|
| `xdotool` + `script`(1) | `akgl_typing` — types at a real focused SDL window with a real keyboard | The test reports **Skipped**. Nothing fails |
| `gcovr` | the `coverage` target and its 90%-of-lines gate | `-DAKBASIC_COVERAGE=ON` will not configure |
| `python3` | `scripts/mutation_test.py` and the `mutation` target | The target is not created |
**`akgl_typing` is worth installing `xdotool` for.** It is the only test that exercises the path
a person actually uses — X11 delivers a key press to a focused window, SDL composes it, libakgl
rings it, the editor echoes it, the interpreter runs it. Every other keyboard test synthesises
SDL events, which tests the code *downstream* of SDL and cannot test the code upstream of it.
That gap shipped a bug: the frontend never called `SDL_StartTextInput()`, so SDL emitted no
text-input events at all, the line editor dropped every keystroke, and the whole suite stayed
green because it was pushing those events itself.
It needs a real X server and **it steals keyboard focus for about fifteen seconds**. Set
`AKBASIC_SKIP_INTERACTIVE=1` to skip it while you are using the machine. It skips itself
automatically when there is no display, no `xdotool`, or no window manager answering — CTest
reports `Skipped` rather than a failure, because none of those means the answer is no.
There are two workflows. **`.gitea/workflows/ci.yaml`** runs on every push: the suite, There are two workflows. **`.gitea/workflows/ci.yaml`** runs on every push: the suite,
ASan+UBSan, coverage gated at 90% of lines, and mutation testing over two files. The coverage ASan+UBSan, coverage gated at 90% of lines, and mutation testing over two files. The coverage
report is uploaded as a `code-coverage` artifact. report is uploaded as a `code-coverage` artifact.

23
TODO.md
View File

@@ -529,11 +529,22 @@ box's left edge at x=100 is a solid white line (mean 255), the box interior is b
outlines rather than fills — and an empty corner is black. stdout carried `HELLO WORLD` at the outlines rather than fills — and an empty corner is black. stdout carried `HELLO WORLD` at the
same time. `QUIT` exits cleanly with status 0. same time. `QUIT` exits cleanly with status 0.
**What that check does not cover, and could not here:** typing at a real focused window. No key **The typing half is no longer manual, and it is no longer a gap.** It used to read "somebody
synthesis tool (`xdotool`, `xte`, python-xlib) is installed on this machine, so the keyboard should type at it once", because no key-synthesis tool was installed. `xdotool` now is, and
path is proved only through SDL's own event queue in `tests/akgl_frontend.c` — which is the `tests/akgl_typing.sh` is registered as the `akgl_typing` CTest case: it starts the driver under
same queue a real keyboard delivers into, but not the same as a window manager giving the a pty, waits for the window with `xdotool search --sync`, gives it keyboard focus, types a
window focus. Somebody should type at it once. program containing a **string literal** and a **lower-case** one, and polls the mirrored stdout
for what they print.
That case earns its keep twice over. It is the only test that covers the path *upstream* of SDL
— X11 → SDL composition → libakgl's ring → the editor — and everything else synthesises SDL
events, which is precisely how the missing `SDL_StartTextInput()` shipped with a green suite.
Confirmed by reverting both halves of that fix and watching it fail with the same symptom that
was reported: `READY`, and nothing after it.
It needs a real X server, a window manager and `xdotool`, and it steals keyboard focus for about
fifteen seconds. Absent any of those it reports CTest `Skipped` rather than failing, which is
what it does in CI. `AKBASIC_SKIP_INTERACTIVE=1` skips it deliberately.
### The five things that had to be worked around — **all gone** ### The five things that had to be worked around — **all gone**
@@ -1122,7 +1133,7 @@ requirement; exactly one case has diverged on purpose since, and
| Gate | Result | | Gate | Result |
|---|---| |---|---|
| `ctest` | 77/77 — 41 reference golden cases, 9 local ones, 25 unit tests, 2 embedding examples. No known-failing tests: the list is empty | | `ctest` | 77/77 — 41 reference golden cases, 9 local ones, 25 unit tests, 2 embedding examples. No known-failing tests: the list is empty |
| `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 76/76 — the same, minus the three `no_device` cases the SDL driver contradicts, plus `akgl_backends` and `akgl_frontend`; the `akgl_build` CI job | | `ctest` with `-DAKBASIC_WITH_AKGL=ON` | 77/77 on a machine with a display; 76 passed + 1 skipped headless. The same set minus the three `no_device` cases the SDL driver contradicts, plus `akgl_backends`, `akgl_frontend` and `akgl_typing` — the last of which is the skip, and the `akgl_build` CI job is where it skips |
| Golden corpus | 41/41 byte-exact from `tests/reference/`**and 41/41 again through the SDL binary**, which is most of what proves the frontend changes no output | | Golden corpus | 41/41 byte-exact from `tests/reference/`**and 41/41 again through the SDL binary**, which is most of what proves the frontend changes no output |
| ASan + UBSan | 77/77 | | ASan + UBSan | 77/77 |
| Line coverage | 94.6% (3857/4076) — above the 90% gate | | Line coverage | 94.6% (3857/4076) — above the 90% gate |

115
tests/akgl_typing.sh Executable file
View File

@@ -0,0 +1,115 @@
#!/bin/bash
#
# Type at the SDL window with a real keyboard, and check what comes out.
#
# This is the only test in the repository that exercises the path a person
# actually uses: X11 delivers a key press to a focused window, SDL composes it,
# libakgl rings it, the line editor echoes it, and the interpreter runs it. Every
# other keyboard test synthesises SDL events, which is a fine way to test the
# code downstream of SDL and no way at all to test the code upstream of it.
#
# **It exists because that gap shipped a bug.** The frontend never called
# SDL_StartTextInput(), so SDL emitted no text-input events, every keystroke
# arrived with an empty `text`, and the editor dropped the lot -- no echo, and
# nothing on stdout either, because RUN could never be typed. The suite was
# green throughout, because it pushed the text-input events itself.
#
# Skips rather than fails when it cannot run: no display, no xdotool, no
# script(1), no window, or no focus are all "this machine cannot answer the
# question", not "the answer is no". CTest is told exit 77 means skipped.
#
# **It steals keyboard focus for a few seconds.** That is inherent -- the point
# is a focused window. Set AKBASIC_SKIP_INTERACTIVE=1 to skip it while you work.
set -u
SKIP=77
BASIC="${1:?usage: akgl_typing.sh <path to basic>}"
WORK="$(mktemp -d)"
trap 'rm -rf "${WORK}"' EXIT
skip() { echo "SKIP: $*"; exit "${SKIP}"; }
fail() { echo "FAIL: $*"; exit 1; }
# ---------------------------------------------------------------- environment
[ "${AKBASIC_SKIP_INTERACTIVE:-0}" = "1" ] && skip "AKBASIC_SKIP_INTERACTIVE=1"
[ -n "${DISPLAY:-}" ] || skip "no DISPLAY; this needs a real X server, not the dummy driver"
command -v xdotool >/dev/null 2>&1 || skip "xdotool is not installed"
command -v script >/dev/null 2>&1 || skip "script(1) is not installed"
xdotool getactivewindow >/dev/null 2>&1 || skip "no window manager is answering on ${DISPLAY}"
# script(1) gives the driver a pty for stdin, which is what makes it choose the
# *window* as its console. Redirected stdin is a deliberate signal to read the
# pipe instead -- see run_akgl() in src/main.c -- and under CTest stdin is never
# a terminal, so without this the driver would ignore the keyboard entirely and
# the test would be measuring nothing.
OUT="${WORK}/typescript"
script -qefc "${BASIC}" "${OUT}" >/dev/null 2>&1 &
DRIVER=$!
# shellcheck disable=SC2317 # reached only through the EXIT trap below
cleanup_driver() {
kill "${DRIVER}" 2>/dev/null
wait "${DRIVER}" 2>/dev/null
}
trap 'cleanup_driver; rm -rf "${WORK}"' EXIT
# ------------------------------------------------------------------ the window
# --sync blocks until it appears rather than sleeping and hoping.
WID="$(timeout 20 xdotool search --sync --onlyvisible --name '^BASIC$' 2>/dev/null | head -1)"
[ -n "${WID}" ] || skip "the BASIC window never appeared (headless, or a WM that does not map it)"
xdotool windowactivate --sync "${WID}" >/dev/null 2>&1
xdotool windowfocus --sync "${WID}" >/dev/null 2>&1
FOCUS="$(xdotool getwindowfocus 2>/dev/null)"
[ "${FOCUS}" = "${WID}" ] || skip "could not give the window keyboard focus (got '${FOCUS}', wanted '${WID}')"
# ------------------------------------------------------------------- the typing
# Poll for expected output instead of sleeping a guessed interval: a fixed sleep
# is either flaky or slow, and usually both.
await() {
local want="$1" limit="${2:-15}" i=0
while [ "${i}" -lt $((limit * 10)) ]; do
if tr -d '\r' < "${OUT}" 2>/dev/null | grep -qF "${want}"; then
return 0
fi
sleep 0.1
i=$((i + 1))
done
return 1
}
type_line() {
xdotool type --clearmodifiers --delay 30 "$1"
xdotool key --clearmodifiers Return
}
await "READY" 20 || skip "the interpreter never reached its prompt"
# A string literal on purpose: the double quote is the character that was
# unreachable before the keystroke ring carried composed text, and a BASIC
# program that cannot contain a string is not much of a BASIC.
type_line '10 PRINT "TYPED OK"'
type_line 'RUN'
if ! await "TYPED OK" 15; then
echo "--- what the driver actually wrote ---"
tr -d '\r' < "${OUT}" 2>/dev/null | sed -n '1,20p'
fail "typing at the window produced nothing: the keyboard path is broken"
fi
# Lower case has to survive too. It could not before libakgl 0.3.0, because a
# keycode cannot express shift and the editor folded everything to upper case.
type_line '20 PRINT "lower"'
type_line 'RUN'
if ! await "lower" 15; then
echo "--- what the driver actually wrote ---"
tr -d '\r' < "${OUT}" 2>/dev/null | sed -n '1,20p'
fail "lower case did not survive the keyboard: composed text is not reaching the editor"
fi
type_line 'QUIT'
await "" 1 >/dev/null 2>&1 || true
echo "PASS: typed at a real focused window and the interpreter ran it"
exit 0