andrew/libakgl
1
0
Fork 0
Code Issues Pull Requests 1 Actions 1 Packages Projects Releases Wiki Activity

Run the benchmarks and the memory check in CI

Browse Source 
The perf suites were already running in CI, because they are ordinary CTest
tests -- but only in the coverage job, which is a Debug build with gcov
instrumentation. That is the one place their numbers cannot mean anything,
and it was spending full-scale iteration counts to prove it. They now run
there at AKGL_BENCH_SCALE=0.02, for path coverage alone, which takes the
whole coverage run to four seconds.

The `performance` job is where the timings are taken: RelWithDebInfo, full
scale, `ctest -L perf`, which is the only configuration in which
tests/benchutil.h enforces a budget. It keeps the tables as an artifact
whether it passes or fails -- a red run says which measurement moved, and a
green one is the next baseline for PERFORMANCE.md. The step uses --verbose
rather than --output-on-failure, because --output-on-failure prints nothing
at all for a passing benchmark, and sets pipefail, without which the status
reaching the runner is tee's and a blown budget reads as a pass.

The `memory_check` job runs scripts/memcheck.sh over every suite under
valgrind and keeps the logs. It carries continue-on-error, and that is a
deliberate, temporary lie: the six findings in TODO.md "Memory checking" are
real and all of them are libakgl's, so gating on it today would only teach
everyone to ignore a red build. The flag comes off with the commit that
closes the last item, and item 34 -- the missing json_decref in all four
asset loaders -- is four lines.

Both new jobs were run locally against a clean copy of the tree with the
exact commands the workflow uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Andrew Kesterson
2026-07-31 15:33:58 -04:00
parent af304dc2f9
commit 80fdf2e098
2 changed files with 137 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
AGENTS.md
View File

@@ -79,6 +79,21 @@ Run mutation testing with `cmake --build build --target mutation`. For a quick s
Generate HTML and Cobertura coverage reports with `cmake -S . -B build-coverage -DAKGL_COVERAGE=ON -DCMAKE_BUILD_TYPE=Debug`, then build and run CTest. Reports are written to `build-coverage/coverage/`; this mode requires `gcovr` and GCC or Clang.
## Continuous integration
`.gitea/workflows/ci.yaml` runs four jobs on every push, and each one exists
because the others cannot do its work:
| Job | Build | What it is for |
|---|---|---|
| `cmake_build` | Debug + `AKGL_COVERAGE=ON` | The unit suites and the coverage report. The perf suites run here too, at `AKGL_BENCH_SCALE=0.02`, for path coverage only — a timing taken under gcov is a measurement of gcov. |
| `performance` | RelWithDebInfo | `ctest -L perf`. The only job where budgets are enforced, because `tests/benchutil.h` enforces them only when compiled optimized at full scale. Keeps the tables as an artifact. |
| `memory_check` | RelWithDebInfo | `scripts/memcheck.sh`, every suite under valgrind. `continue-on-error` until TODO.md "Memory checking" items 34-39 are closed; keeps the valgrind logs as an artifact. |
| `mutation_test` | Debug | One focused source file, so CI stays bounded. |
The `character` suite is excluded wherever a whole run is selected: it fails
deliberately, and pinning current-but-wrong behavior is not what it is for.
## Coding Style
The canonical style is Emacs `cc-mode` **`stroustrup`**, with tabs enabled. This