Files
libakstdlib/tests/aksl_capture.h

276 lines
12 KiB
C
Raw Normal View History

Build a real test harness (TODO.md section 1.0) The suite could not fail and did not run. test_linkedlist.c asserted nothing at all -- it printed node names and returned 0 -- so every confirmed list defect passed it. test_tree.c was red on every run because parms.steps was never reset between its three searches, and four libakerror tests registered but never built, reporting Not Run. CI, meanwhile, was not fetching the submodule, so configure failed before reaching any of it. - tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to run an akerror-returning call, assert its status and release the context, a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver that fails any test leaking an error-pool slot. - test_linkedlist.c: rewritten as 15 assertion cases over the append, iterate and pop behaviour that is correct today. - test_tree.c: each search builds its own tree and params. - Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new tests asserting correct behaviour for the confirmed defects in TODO.md 2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed", which is the cue to promote it into AKSL_TESTS. - add_test/set_tests_properties are shadowed across the libakerror add_subdirectory call: CMake cannot un-register a test and set_tests_properties cannot cross directory scopes. The dependency has its own CI. - AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan. - scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to a manual `mutation` target, not a CI gate until 1.1-1.9 exist. - CI: checkout with submodules: recursive, and ctest --output-on-failure. ctest is now 5/5 green in both the default and sanitizer builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:54 -04:00
#ifndef AKSL_TEST_CAPTURE_H
#define AKSL_TEST_CAPTURE_H
/*
* Shared test helpers for libakstdlib.
*
* Modelled on libakerror's tests/err_capture.h, with additions for the shape of
* this library: almost every akstdlib entry point returns an
* akerr_ErrorContext * that the caller owns and must release, so the common
* assertion is "this call returned status X" rather than "this call logged Y".
*
* What is here:
*
* AKSL_CHECK() an NDEBUG-proof assertion. Fails the test by
* returning 1 from the enclosing function (unlike
* assert(), which is compiled out in release builds
* and would silently turn a test into a no-op).
* AKSL_CHECK_STATUS() run an akerror-returning expression, assert on the
* status it came back with, and release the context
* so the error pool does not leak.
* AKSL_CHECK_STATUS_MSG_CONTAINS()
* assert status first, then message content.
* AKSL_CHECK_OK() assert a call returned no error context at all.
Build a real test harness (TODO.md section 1.0) The suite could not fail and did not run. test_linkedlist.c asserted nothing at all -- it printed node names and returned 0 -- so every confirmed list defect passed it. test_tree.c was red on every run because parms.steps was never reset between its three searches, and four libakerror tests registered but never built, reporting Not Run. CI, meanwhile, was not fetching the submodule, so configure failed before reaching any of it. - tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to run an akerror-returning call, assert its status and release the context, a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver that fails any test leaking an error-pool slot. - test_linkedlist.c: rewritten as 15 assertion cases over the append, iterate and pop behaviour that is correct today. - test_tree.c: each search builds its own tree and params. - Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new tests asserting correct behaviour for the confirmed defects in TODO.md 2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed", which is the cue to promote it into AKSL_TESTS. - add_test/set_tests_properties are shadowed across the libakerror add_subdirectory call: CMake cannot un-register a test and set_tests_properties cannot cross directory scopes. The dependency has its own CI. - AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan. - scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to a manual `mutation` target, not a CI gate until 1.1-1.9 exist. - CI: checkout with submodules: recursive, and ctest --output-on-failure. ctest is now 5/5 green in both the default and sanitizer builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:54 -04:00
* aksl_last_status/... the status, message and function name of the most
* recent context taken by AKSL_CHECK_STATUS.
* aksl_capture_install() swap in a capturing akerr_log_method so a test can
* assert on the *content* of stack traces and
* unhandled-error output.
* aksl_slots_in_use() how many slots are currently checked out of
* AKERR_ARRAY_ERROR, for pool-leak assertions.
* AKSL_RUN() run one test function and tally the result.
*
* Tests are written as a set of `static int test_xxx(void)` functions that
* return 0 on success and non-zero on failure, driven from main() by AKSL_RUN.
*/
#include <akstdlib.h>
#include <stdarg.h>
#include <stdio.h>
#include <string.h>
/* ---------------------------------------------------------------------- */
/* Log capture */
/* ---------------------------------------------------------------------- */
#define AKSL_CAPTURE_BUFSZ 65536
static char aksl_capture_buf[AKSL_CAPTURE_BUFSZ];
static size_t aksl_capture_len = 0;
static void __attribute__((unused)) aksl_capture_logger(const char *fmt, ...)
{
va_list ap;
va_start(ap, fmt);
int n = vsnprintf(aksl_capture_buf + aksl_capture_len,
AKSL_CAPTURE_BUFSZ - aksl_capture_len, fmt, ap);
va_end(ap);
if ( n > 0 ) {
aksl_capture_len += (size_t)n;
if ( aksl_capture_len >= AKSL_CAPTURE_BUFSZ ) {
aksl_capture_len = AKSL_CAPTURE_BUFSZ - 1;
}
}
}
static void __attribute__((unused)) aksl_capture_reset(void)
{
aksl_capture_len = 0;
aksl_capture_buf[0] = '\0';
}
/*
* Install the capturing logger. akerr_init() only assigns a default logger when
* akerr_log_method is NULL, and it is idempotent, so calling this either before
* or after the first PREPARE_ERROR keeps our logger in place.
*/
static void __attribute__((unused)) aksl_capture_install(void)
{
aksl_capture_reset();
akerr_log_method = &aksl_capture_logger;
}
/* ---------------------------------------------------------------------- */
/* Error pool accounting */
/* ---------------------------------------------------------------------- */
/* Count array slots currently checked out of the pool (refcount != 0). */
static int __attribute__((unused)) aksl_slots_in_use(void)
{
int n = 0;
for ( int i = 0; i < AKERR_MAX_ARRAY_ERROR; i++ ) {
if ( AKERR_ARRAY_ERROR[i].refcount != 0 ) {
n++;
}
}
return n;
}
/* ---------------------------------------------------------------------- */
/* Taking ownership of a returned error context */
/* ---------------------------------------------------------------------- */
static int aksl_last_status = 0;
static char aksl_last_message[AKERR_MAX_ERROR_CONTEXT_STRING_LENGTH];
/* Sized to match akerr_ErrorContext.function, which akerror declares with
* AKERR_MAX_ERROR_FNAME_LENGTH rather than AKERR_MAX_ERROR_FUNCTION_LENGTH. */
static char aksl_last_function[AKERR_MAX_ERROR_FNAME_LENGTH];
/*
* Record the status/message/function of a returned context, release it back to
* the pool, and hand back the status. A NULL context means success, which is
* status 0. Failure-path assertions should go through this so that no test
* leaks a pool slot. Success-path assertions use AKSL_CHECK_OK, which also
* verifies that no bogus status-0 context was returned.
Build a real test harness (TODO.md section 1.0) The suite could not fail and did not run. test_linkedlist.c asserted nothing at all -- it printed node names and returned 0 -- so every confirmed list defect passed it. test_tree.c was red on every run because parms.steps was never reset between its three searches, and four libakerror tests registered but never built, reporting Not Run. CI, meanwhile, was not fetching the submodule, so configure failed before reaching any of it. - tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to run an akerror-returning call, assert its status and release the context, a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver that fails any test leaking an error-pool slot. - test_linkedlist.c: rewritten as 15 assertion cases over the append, iterate and pop behaviour that is correct today. - test_tree.c: each search builds its own tree and params. - Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new tests asserting correct behaviour for the confirmed defects in TODO.md 2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed", which is the cue to promote it into AKSL_TESTS. - add_test/set_tests_properties are shadowed across the libakerror add_subdirectory call: CMake cannot un-register a test and set_tests_properties cannot cross directory scopes. The dependency has its own CI. - AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan. - scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to a manual `mutation` target, not a CI gate until 1.1-1.9 exist. - CI: checkout with submodules: recursive, and ctest --output-on-failure. ctest is now 5/5 green in both the default and sanitizer builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:54 -04:00
*/
static int __attribute__((unused)) aksl_take(akerr_ErrorContext *e)
{
akerr_ErrorContext *released = NULL;
aksl_last_message[0] = '\0';
aksl_last_function[0] = '\0';
if ( e == NULL ) {
aksl_last_status = 0;
return 0;
}
aksl_last_status = e->status;
snprintf(aksl_last_message, sizeof(aksl_last_message), "%s", e->message);
snprintf(aksl_last_function, sizeof(aksl_last_function), "%s", e->function);
/*
* akerr_release_error is marked warn_unused_result, and a (void) cast does
* not silence that in GCC, so the result is assigned and discarded.
*/
released = akerr_release_error(e);
(void)released;
return aksl_last_status;
}
/* ---------------------------------------------------------------------- */
/* Assertions */
/* ---------------------------------------------------------------------- */
#define AKSL_CHECK(cond) \
do { \
if ( !(cond) ) { \
fprintf(stderr, " CHECK FAILED: %s at %s:%d\n", \
#cond, __FILE__, __LINE__); \
return 1; \
} \
} while ( 0 )
/*
* Run an akerr_ErrorContext *-returning expression and assert on its status.
* The context is always released, including on the failure path, so a failing
* assertion does not also corrupt the pool-leak checks that follow it.
*/
#define AKSL_CHECK_STATUS(__expr, __expected) \
do { \
int __st = aksl_take(__expr); \
if ( __st != (__expected) ) { \
fprintf(stderr, \
" CHECK FAILED: %s\n" \
" got %d (%s) \"%s\"\n" \
" expected %d (%s)\n" \
" at %s:%d\n", \
#__expr, \
__st, akerr_name_for_status(__st, NULL), \
aksl_last_message, \
(__expected), \
akerr_name_for_status((__expected), NULL), \
__FILE__, __LINE__); \
return 1; \
} \
} while ( 0 )
/*
* Success is represented by a NULL error context, not just status 0. This catches
* wrappers that incorrectly raise an error using a stale errno value of 0.
*/
#define AKSL_CHECK_OK(__expr) \
do { \
akerr_ErrorContext *__e = (__expr); \
if ( __e != NULL ) { \
int __st = aksl_take(__e); \
fprintf(stderr, \
" CHECK FAILED: %s\n" \
" returned error context with status %d (%s) \"%s\"\n" \
" expected no error context\n" \
" at %s:%d\n", \
#__expr, \
__st, akerr_name_for_status(__st, NULL), \
aksl_last_message, \
__FILE__, __LINE__); \
return 1; \
} \
aksl_last_status = 0; \
aksl_last_message[0] = '\0'; \
aksl_last_function[0] = '\0'; \
} while ( 0 )
Build a real test harness (TODO.md section 1.0) The suite could not fail and did not run. test_linkedlist.c asserted nothing at all -- it printed node names and returned 0 -- so every confirmed list defect passed it. test_tree.c was red on every run because parms.steps was never reset between its three searches, and four libakerror tests registered but never built, reporting Not Run. CI, meanwhile, was not fetching the submodule, so configure failed before reaching any of it. - tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to run an akerror-returning call, assert its status and release the context, a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver that fails any test leaking an error-pool slot. - test_linkedlist.c: rewritten as 15 assertion cases over the append, iterate and pop behaviour that is correct today. - test_tree.c: each search builds its own tree and params. - Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new tests asserting correct behaviour for the confirmed defects in TODO.md 2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed", which is the cue to promote it into AKSL_TESTS. - add_test/set_tests_properties are shadowed across the libakerror add_subdirectory call: CMake cannot un-register a test and set_tests_properties cannot cross directory scopes. The dependency has its own CI. - AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan. - scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to a manual `mutation` target, not a CI gate until 1.1-1.9 exist. - CI: checkout with submodules: recursive, and ctest --output-on-failure. ctest is now 5/5 green in both the default and sanitizer builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:54 -04:00
#define AKSL_CHECK_CONTAINS(needle) \
AKSL_CHECK(strstr(aksl_capture_buf, (needle)) != NULL)
#define AKSL_CHECK_NOT_CONTAINS(needle) \
AKSL_CHECK(strstr(aksl_capture_buf, (needle)) == NULL)
/*
* Message checks are secondary: the returned akerr status is the contract. Keep
* the status and message assertions coupled so tests cannot pass on text alone.
*/
#define AKSL_CHECK_STATUS_MSG_CONTAINS(__expr, __expected, __needle) \
do { \
int __st = aksl_take(__expr); \
if ( __st != (__expected) ) { \
fprintf(stderr, \
" CHECK FAILED: %s\n" \
" got %d (%s) \"%s\"\n" \
" expected %d (%s)\n" \
" at %s:%d\n", \
#__expr, \
__st, akerr_name_for_status(__st, NULL), \
aksl_last_message, \
(__expected), \
akerr_name_for_status((__expected), NULL), \
__FILE__, __LINE__); \
return 1; \
} \
if ( strstr(aksl_last_message, (__needle)) == NULL ) { \
fprintf(stderr, \
" CHECK FAILED: message from %s\n" \
" got \"%s\"\n" \
" expected substring \"%s\"\n" \
" status %d (%s)\n" \
" at %s:%d\n", \
#__expr, \
aksl_last_message, \
(__needle), \
__st, akerr_name_for_status(__st, NULL), \
__FILE__, __LINE__); \
return 1; \
} \
} while ( 0 )
Build a real test harness (TODO.md section 1.0) The suite could not fail and did not run. test_linkedlist.c asserted nothing at all -- it printed node names and returned 0 -- so every confirmed list defect passed it. test_tree.c was red on every run because parms.steps was never reset between its three searches, and four libakerror tests registered but never built, reporting Not Run. CI, meanwhile, was not fetching the submodule, so configure failed before reaching any of it. - tests/aksl_capture.h: AKSL_CHECK (NDEBUG-proof), AKSL_CHECK_STATUS/_OK to run an akerror-returning call, assert its status and release the context, a capturing akerr_log_method, aksl_slots_in_use(), and an AKSL_RUN driver that fails any test leaking an error-pool slot. - test_linkedlist.c: rewritten as 15 assertion cases over the append, iterate and pop behaviour that is correct today. - test_tree.c: each search builds its own tree and params. - Registration driven by AKSL_TESTS, plus AKSL_WILL_FAIL_TESTS (aborts by design) and AKSL_KNOWN_FAILING_TESTS, which marks WILL_FAIL the three new tests asserting correct behaviour for the confirmed defects in TODO.md 2.1.1-2.1.3. Fixing a defect flips its test to "unexpectedly passed", which is the cue to promote it into AKSL_TESTS. - add_test/set_tests_properties are shadowed across the libakerror add_subdirectory call: CMake cannot un-register a test and set_tests_properties cannot cross directory scopes. The dependency has its own CI. - AKSL_SANITIZE=ON builds library, tests and dependency with ASan+UBSan. - scripts/mutation_test.py ported and retargeted at src/stdlib.c; wired to a manual `mutation` target, not a CI gate until 1.1-1.9 exist. - CI: checkout with submodules: recursive, and ctest --output-on-failure. ctest is now 5/5 green in both the default and sanitizer builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 12:39:54 -04:00
/* ---------------------------------------------------------------------- */
/* Test driver */
/* ---------------------------------------------------------------------- */
/*
* Run one test function, report it, and tally failures. Also asserts that the
* test left the error pool as it found it -- a wrapper that fails to release a
* context is a bug in the library, not just in the test.
*/
#define AKSL_RUN(__failures, __fn) \
do { \
int __before = aksl_slots_in_use(); \
int __r = __fn(); \
int __after = aksl_slots_in_use(); \
if ( __r != 0 ) { \
fprintf(stderr, "FAIL %s\n", #__fn); \
(__failures)++; \
} else if ( __after != __before ) { \
fprintf(stderr, \
"FAIL %s (leaked %d error pool slot(s))\n", \
#__fn, __after - __before); \
(__failures)++; \
} else { \
fprintf(stderr, "ok %s\n", #__fn); \
} \
} while ( 0 )
#define AKSL_REPORT(__failures) \
do { \
fprintf(stderr, "%s: %d failure(s)\n", __FILE__, (__failures)); \
return (__failures) == 0 ? 0 : 1; \
} while ( 0 )
#endif // AKSL_TEST_CAPTURE_H