The perf suites were already running in CI, because they are ordinary CTest
tests -- but only in the coverage job, which is a Debug build with gcov
instrumentation. That is the one place their numbers cannot mean anything,
and it was spending full-scale iteration counts to prove it. They now run
there at AKGL_BENCH_SCALE=0.02, for path coverage alone, which takes the
whole coverage run to four seconds.
The `performance` job is where the timings are taken: RelWithDebInfo, full
scale, `ctest -L perf`, which is the only configuration in which
tests/benchutil.h enforces a budget. It keeps the tables as an artifact
whether it passes or fails -- a red run says which measurement moved, and a
green one is the next baseline for PERFORMANCE.md. The step uses --verbose
rather than --output-on-failure, because --output-on-failure prints nothing
at all for a passing benchmark, and sets pipefail, without which the status
reaching the runner is tee's and a blown budget reads as a pass.
The `memory_check` job runs scripts/memcheck.sh over every suite under
valgrind and keeps the logs. It carries continue-on-error, and that is a
deliberate, temporary lie: the six findings in TODO.md "Memory checking" are
real and all of them are libakgl's, so gating on it today would only teach
everyone to ignore a red build. The flag comes off with the commit that
closes the last item, and item 34 -- the missing json_decref in all four
asset loaders -- is four lines.
Both new jobs were run locally against a clean copy of the tree with the
exact commands the workflow uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Andrew Kesterson
80fdf2e098
