Run the benchmarks and the memory check in CI

The perf suites were already running in CI, because they are ordinary CTest
tests -- but only in the coverage job, which is a Debug build with gcov
instrumentation. That is the one place their numbers cannot mean anything,
and it was spending full-scale iteration counts to prove it. They now run
there at AKGL_BENCH_SCALE=0.02, for path coverage alone, which takes the
whole coverage run to four seconds.

The `performance` job is where the timings are taken: RelWithDebInfo, full
scale, `ctest -L perf`, which is the only configuration in which
tests/benchutil.h enforces a budget. It keeps the tables as an artifact
whether it passes or fails -- a red run says which measurement moved, and a
green one is the next baseline for PERFORMANCE.md. The step uses --verbose
rather than --output-on-failure, because --output-on-failure prints nothing
at all for a passing benchmark, and sets pipefail, without which the status
reaching the runner is tee's and a blown budget reads as a pass.

The `memory_check` job runs scripts/memcheck.sh over every suite under
valgrind and keeps the logs. It carries continue-on-error, and that is a
deliberate, temporary lie: the six findings in TODO.md "Memory checking" are
real and all of them are libakgl's, so gating on it today would only teach
everyone to ignore a red build. The flag comes off with the commit that
closes the last item, and item 34 -- the missing json_decref in all four
asset loaders -- is four lines.

Both new jobs were run locally against a clean copy of the tree with the
exact commands the workflow uses.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.gitea/workflows/ci.yaml

@@ -28,7 +28,16 @@ jobs:
           cmake --build build --parallel
       - name: Build API documentation
         run: doxygen Doxyfile
+      # The perf suites run here too -- they are ordinary CTest tests -- but this
+      # is a Debug build with coverage instrumentation, where a timing is a
+      # measurement of gcov. AKGL_BENCH_SCALE cuts their iteration counts so
+      # they contribute path coverage without spending ten minutes proving
+      # nothing; benchutil.h already refuses to enforce budgets in an
+      # unoptimized build. The performance job below is where the timings are
+      # taken and the budgets are checked.
       - name: Test (JUnit)
+        env:
+          AKGL_BENCH_SCALE: '0.02'
         run: |
           export LD_LIBRARY_PATH="$PWD/build:$PWD/build/deps/SDL:$PWD/build/deps/SDL_image:$PWD/build/deps/SDL_mixer:$PWD/build/deps/SDL_ttf"
           ctest \
@@ -53,6 +62,119 @@ jobs:
           path: build/coverage/
           if-no-files-found: warn
 
+  # The benchmarks, in the only build where their numbers mean anything.
+  # tests/benchutil.h enforces a budget per measurement only when it is compiled
+  # optimized and running at full scale, which is exactly this job and no other:
+  # the coverage job above is Debug, and the memory job below is under valgrind.
+  # A budget is roughly ten times the baseline recorded in PERFORMANCE.md, which
+  # is what makes a shared, slower runner viable -- it catches an algorithmic
+  # regression and ignores a busy machine. If a budget still proves flaky here,
+  # raise that one budget with a measurement behind it and re-record the
+  # baseline; do not drop the gate.
+  performance:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository code
+        uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - name: Install build dependencies
+        run: |
+          sudo apt-get update -y
+          sudo apt-get install -y \
+            cmake gcc pkg-config \
+            libasound2-dev libfreetype-dev libharfbuzz-dev \
+            libpng-dev libtiff-dev libwebp-dev \
+            libudev-dev libx11-dev libxcursor-dev libxext-dev \
+            libxfixes-dev libxi-dev libxrandr-dev libxrender-dev \
+            libxss-dev libxtst-dev
+      - name: Configure and build
+        run: |
+          cmake -S . -B build \
+            -DCMAKE_BUILD_TYPE=RelWithDebInfo
+          cmake --build build --parallel
+      # --verbose rather than --output-on-failure: a passing benchmark's output
+      # is the entire point of running it, and --output-on-failure prints
+      # nothing at all for a green run. pipefail because the step ends in a
+      # pipe, whose status would otherwise be tee's -- always 0 -- and a blown
+      # budget would be reported as a passing step.
+      - name: Benchmarks (JUnit)
+        run: |
+          set -o pipefail
+          export LD_LIBRARY_PATH="$PWD/build:$PWD/build/deps/SDL:$PWD/build/deps/SDL_image:$PWD/build/deps/SDL_mixer:$PWD/build/deps/SDL_ttf"
+          ctest \
+            --test-dir build \
+            -L perf \
+            --verbose \
+            --output-junit "$(pwd)/perf-junit.xml" \
+            2>&1 | tee "$(pwd)/perf-baseline.txt"
+      - name: Publish benchmark results
+        if: always()
+        uses: mikepenz/action-junit-report@v4
+        with:
+          report_paths: 'perf-junit.xml'
+          annotate_only: true
+          detailed_summary: true
+          include_passed: true
+          fail_on_failure: 'true'
+      # Kept whether the job passed or failed: the tables are the point. A run
+      # that went red says which measurement moved, and a run that went green is
+      # the next baseline if somebody re-records PERFORMANCE.md.
+      - name: Upload benchmark tables
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: performance-baseline
+          path: perf-baseline.txt
+          if-no-files-found: warn
+
+  # Every suite under valgrind. No memory-check programs of their own: the perf
+  # suites scale themselves down when they detect valgrind, which turns them
+  # into the broadest path coverage in the tree at a cost this job can afford.
+  #
+  # continue-on-error until TODO.md "Memory checking" items 34-39 are closed.
+  # The findings are real and they are all in libakgl, so this job is red today
+  # by telling the truth; gating on it now would only teach everyone to ignore a
+  # red build. Remove the flag with the commit that closes the last item -- and
+  # the JSON one (item 34) is four lines, so that should not be a long wait.
+  memory_check:
+    runs-on: ubuntu-latest
+    continue-on-error: true
+    steps:
+      - name: Check out repository code
+        uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - name: Install memory-check dependencies
+        run: |
+          sudo apt-get update -y
+          sudo apt-get install -y \
+            cmake gcc pkg-config valgrind \
+            libasound2-dev libfreetype-dev libharfbuzz-dev \
+            libpng-dev libtiff-dev libwebp-dev \
+            libudev-dev libx11-dev libxcursor-dev libxext-dev \
+            libxfixes-dev libxi-dev libxrandr-dev libxrender-dev \
+            libxss-dev libxtst-dev
+      # RelWithDebInfo rather than Debug: valgrind needs the symbols, and -O0
+      # would leave every inlined frame in the stacks it prints.
+      - name: Configure and build
+        run: |
+          cmake -S . -B build \
+            -DCMAKE_BUILD_TYPE=RelWithDebInfo
+          cmake --build build --parallel
+      # The character suite is excluded here for the same reason as in the
+      # coverage job: it fails deliberately, and a failing suite would be
+      # reported as "valgrind found nothing but ctest exited non-zero".
+      - name: Memory check
+        run: scripts/memcheck.sh -E '^character$'
+      - name: Upload valgrind logs
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: valgrind-logs
+          path: build/Testing/Temporary/MemoryChecker.*.log
+          if-no-files-found: warn
+
   mutation_test:
     runs-on: ubuntu-latest
     steps:

AGENTS.md

@@ -79,6 +79,21 @@ Run mutation testing with `cmake --build build --target mutation`. For a quick s
 
 Generate HTML and Cobertura coverage reports with `cmake -S . -B build-coverage -DAKGL_COVERAGE=ON -DCMAKE_BUILD_TYPE=Debug`, then build and run CTest. Reports are written to `build-coverage/coverage/`; this mode requires `gcovr` and GCC or Clang.
 
+## Continuous integration
+
+`.gitea/workflows/ci.yaml` runs four jobs on every push, and each one exists
+because the others cannot do its work:
+
+| Job | Build | What it is for |
+|---|---|---|
+| `cmake_build` | Debug + `AKGL_COVERAGE=ON` | The unit suites and the coverage report. The perf suites run here too, at `AKGL_BENCH_SCALE=0.02`, for path coverage only — a timing taken under gcov is a measurement of gcov. |
+| `performance` | RelWithDebInfo | `ctest -L perf`. The only job where budgets are enforced, because `tests/benchutil.h` enforces them only when compiled optimized at full scale. Keeps the tables as an artifact. |
+| `memory_check` | RelWithDebInfo | `scripts/memcheck.sh`, every suite under valgrind. `continue-on-error` until TODO.md "Memory checking" items 34-39 are closed; keeps the valgrind logs as an artifact. |
+| `mutation_test` | Debug | One focused source file, so CI stays bounded. |
+
+The `character` suite is excluded wherever a whole run is selected: it fails
+deliberately, and pinning current-but-wrong behavior is not what it is for.
+
 ## Coding Style
 
 The canonical style is Emacs `cc-mode` **`stroustrup`**, with tabs enabled. This