Andrew Kesterson
506af093a1
Ports libakstdlib's mutation harness (the newest of the three) to scripts/
mutation_test.py, adds the namespaced `mutation` CMake target the sibling
libraries have, and splits CI in two.
.gitea/workflows/ci.yaml keeps the push path: suite, ASan+UBSan, coverage, and a
mutation run bounded to src/convert.c and src/symtab.c -- about four minutes,
scoring 77.8% against a gate of 65.
.gitea/workflows/release.yaml is new and manual (workflow_dispatch). It carries
the doxygen gate, moved out of ci.yaml, and a whole-tree mutation run: 3675
mutants and hours of runner time, which is a release cost rather than a
per-commit one. Both artifacts a release wants -- api-documentation and
mutation-report -- come out of it. Two optional inputs narrow the run or change
the threshold; they arrive through the environment rather than being
interpolated into the shell, because ${{ }} substitution happens before the
shell sees the line.
The harness paid for itself immediately, which is the point of it. Three real
gaps, each checked to be a genuine bug rather than an equivalent mutant:
- errno was never asserted clear before a strtoll. Confirmed with a standalone
probe that strtoll leaves a stale errno untouched on success, so without the
`errno = 0` a valid conversion raises ERANGE.
- Nothing exercised a maximum-length symbol-table key, so every MAX_KEY - 1
off-by-one in a strncpy and its NUL terminator survived. The same hole exists
for strings in src/value.c and is filed.
- Nothing asserted a freshly initialised table was actually zeroed.
Closing the first two took the measured score from 73.1% to 77.8%.
I set the push-path threshold to 75 first, on an estimate. Measuring gave 73.1%
and the job would have failed on its first run -- the earlier per-file figure was
too high because the captured output had been truncated to its last lines and I
counted fewer survivors than there were. It is 65 now, and the gate was run as
written and confirmed to exit 0.
src/value.c is the file most worth mutating and is deliberately off the push
path: 368 mutants at ~11s each is about 70 minutes, because almost everything
links against it. A partial run over it found the same maximum-length-string
hole plus two genuinely equivalent mutants that only exist because of reference
defect section 6 item 5 -- adding both of the right operand's numeric fields
works only while the unused one is zero, so + and - are interchangeable there.
Recorded in TODO.md.
ctest 61/61; doxygen exits 0; both workflows' steps were executed locally, both
input paths included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
73 lines
2.6 KiB
C
73 lines
2.6 KiB
C
/**
|
|
* @file convert.c
|
|
* @brief Tests the strict string-to-number converters.
|
|
*
|
|
* These exist because libakstdlib's aksl_ato* family cannot report a conversion
|
|
* failure. Every assertion here is one that family would get wrong.
|
|
*/
|
|
|
|
#include <errno.h>
|
|
|
|
#include <akbasic/convert.h>
|
|
#include <akbasic/error.h>
|
|
|
|
#include "testutil.h"
|
|
|
|
int main(void)
|
|
{
|
|
int64_t i = 0;
|
|
double d = 0.0;
|
|
|
|
TEST_REQUIRE_OK(akbasic_error_register());
|
|
|
|
/* Valid decimal, hex and negative. */
|
|
TEST_REQUIRE_OK(akbasic_str_to_int64("1234", 10, &i));
|
|
TEST_REQUIRE_INT(i, 1234);
|
|
TEST_REQUIRE_OK(akbasic_str_to_int64("0xff", 16, &i));
|
|
TEST_REQUIRE_INT(i, 255);
|
|
TEST_REQUIRE_OK(akbasic_str_to_int64("-256", 10, &i));
|
|
TEST_REQUIRE_INT(i, -256);
|
|
|
|
/*
|
|
* The whole point: aksl_atoi returns success with 0 for each of these.
|
|
*/
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("", 10, &i), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("not a number", 10, &i), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("12abc", 10, &i), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("12 ", 10, &i), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("99999999999999999999999", 10, &i), ERANGE);
|
|
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64(NULL, 10, &i), AKERR_NULLPOINTER);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_int64("1", 10, NULL), AKERR_NULLPOINTER);
|
|
|
|
/* Floats. */
|
|
TEST_REQUIRE_OK(akbasic_str_to_double("123.456", &d));
|
|
TEST_REQUIRE_FEQ(d, 123.456);
|
|
TEST_REQUIRE_OK(akbasic_str_to_double("-2.5", &d));
|
|
TEST_REQUIRE_FEQ(d, -2.5);
|
|
TEST_REQUIRE_OK(akbasic_str_to_double("32", &d));
|
|
TEST_REQUIRE_FEQ(d, 32.0);
|
|
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_double("", &d), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_double("garbage", &d), AKBASIC_ERR_VALUE);
|
|
TEST_REQUIRE_STATUS(akbasic_str_to_double("1.5x", &d), AKBASIC_ERR_VALUE);
|
|
|
|
/*
|
|
* errno hygiene. strtoll and strtod do not clear errno on success, so a
|
|
* stale ERANGE left by some earlier call would make a perfectly valid
|
|
* conversion raise. That is what the `errno = 0` before each call is for.
|
|
*
|
|
* Found by mutation testing: deleting that line, or setting errno to
|
|
* something non-zero, survived the suite until these four assertions
|
|
* existed.
|
|
*/
|
|
errno = ERANGE;
|
|
TEST_REQUIRE_OK(akbasic_str_to_int64("42", 10, &i));
|
|
TEST_REQUIRE_INT(i, 42);
|
|
errno = ERANGE;
|
|
TEST_REQUIRE_OK(akbasic_str_to_double("2.5", &d));
|
|
TEST_REQUIRE_FEQ(d, 2.5);
|
|
|
|
return akbasic_test_failures;
|
|
}
|