Tests and CI#
Running the tests#
Every CI leg runs the full suite, so a plain ctest in any test-enabled tree reproduces what
CI runs; the tiers differ only in which configurations build. ctest -L test_scheduler selects
one binary (the label is the binary’s name).
cmake --preset desktop-tests-debug && cmake --build --preset desktop-tests-debug
ctest --preset desktop-tests-debug
# the benchmarks are tests too, one at a time so they do not compete for the cores
cmake --preset desktop-benchmark-tests-debug && cmake --build --preset desktop-benchmark-tests-debug
ctest --preset desktop-benchmark-tests-debug -R '^Benchmark' -j 1
Test layout#
test/ mirrors src/: test/<dir>/test_<Unit>.cpp covers src/<dir>/<Unit>.h, and the
directory decides the binary (test/CMakeLists.txt).
scheduler/,utils/andengines/(the adapters) map one to one;test/abi/covers the C ABI (include/anira/abi/, implemented undersrc/capi/) and holds the header gates.The
anira.hpphandler (test_InferenceHandler), the 2.x face (test_InferenceHandlerApi) and the cross-unit suites (test_OneSidedStreaming) are root-leveltest_*.cppfiles. A file’s binary follows the API it targets: the white-box cases of a black-box suite go to a white-box binary (scheduler/test_OneSidedStreamingInternals.cppintest_scheduler).test/contracts/checks the build, link and packaging contracts: engine linkage, the library-unload harness, the installed-package consumers.test/support/is shared infrastructure.A binary is white-box or black-box (
anira_add_test_binary(<target> [WHITE_BOX] ...)). The white-box ones (test_utils,test_scheduler,test_engines,test_abi) link the library’s objects withsrc/on the include path andANIRA_STATIC; every other binary linksanira::aniraand sees only what a consumer sees.The core’s C++ headers are private and never installed. What anira installs is the list of
cmake/public-headers.cmake; a header underinclude/anirathat is not listed fails the configure.Do not re-test what tanh-lib covers upstream: the threading primitives are aliases over
thl::core, and each ring holds athl::core::RingBuffer<T>. anira tests its own contract on top (the dtype tag, the refusals, the window pop); coverage of the container itself goes to tanh-lib.
The installed package#
test/contracts/install is an external project that consumes an installed anira through
find_package(anira). The build_install workflow runs it in the merge queue and on tags:
cmake --preset ci-install-shared
cmake --build build/ci/install-shared --target anira
cmake --install build/ci/install-shared --prefix $PWD/prefix
cmake -S test/contracts/install -B build-consumer -DCMAKE_PREFIX_PATH=$PWD/prefix
cmake --build build-consumer && ./build-consumer/consumer
Whatever a consumer needs (public headers, the exported target, the engine runtimes) must be in
that tree. The ci-install-* presets build ONNX Runtime only; desktop-install-release /
desktop-install-debug install the default engine set (without --prefix into
<build dir>/anira-<version>).
CI tiers#
Which tests run when is defined in four places, each what CI itself reads (three files and, last, the ruleset in the repository settings):
PR or merge queue: the
"pr"flags of.github/*_matrix.json; a row without"pr": trueruns in the queue only. A guard inbuild_test’sresultjob keeps the static and tflite legs (the ExecuTorch and desktop-TFLite coverage) in the PR tier.PR stubs: a workflow guarded by
if: github.event_name != 'pull_request'reports its status on a pull request without running; the merge queue is its gate.clang-tidy:
.github/workflows/clang_tidy.yml. A pull request checks the.cppfiles it changes (CHANGED_FILES_ONLY); the queue sweeps every source. A full sweep takes about an hour of CPU time, most of it in the test files.What the queue requires: the
<name> resultcontexts of the branch’s ruleset, each made by theresultjob at the bottom of its workflow. The ruleset is authoritative, and the branches’ rulesets need not be identical.
Sanitizers#
Three sanitizer presets gate the merge queue; each reproduces locally:
cmake --preset <name> && cmake --build --preset <name> && ctest --preset <name>
desktop-tests-rtsanRealtimeSanitizer over the full engine set (clang 20 or later), no suppressions: an allocation, lock, sleep, semaphore or stream syscall reached from an
ANIRA_NONBLOCKINGentry fails. The compile-time half isanira_abi_rt_contract(test/abi/test_rt_contract.c) under-Werror=function-effects: a nonblocking body calls onlyANIRA_NONBLOCKINGentries, and a function converted to a nonblocking callback type (anira_miss_fn) is declaredANIRA_NONBLOCKINGitself.desktop-tests-asanAddressSanitizer and UndefinedBehaviorSanitizer, including clang’s function sanitizer (hence
no_sanitize("function")on the DLPack trampoline ofsrc/capi/tensor.cpp). SetUBSAN_OPTIONS=halt_on_error=1:print_stacktrace=1by hand, as CI does on every leg; the default prints and continues.desktop-tests-tsanThreadSanitizer.
The ASan and TSan presets build no engines (the prebuilt runtimes are uninstrumented) and are
RelWithDebInfo with -DNDEBUG dropped. Each sanitizer is also an option on any
configuration (ANIRA_WITH_RTSAN, ANIRA_WITH_ASAN, ANIRA_WITH_UBSAN, ANIRA_WITH_TSAN,
ANIRA_WITH_LSAN), instrumenting the tanh-lib built beside anira too; TSan does not combine
with ASan or LSan.
Note
On macOS 26 the Apple clang ASan and TSan runtimes are broken (a hello-world hangs or crashes).
Use Homebrew LLVM with an explicit SDK
(-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++ -DCMAKE_OSX_SYSROOT=$(xcrun --show-sdk-path)).
LeakSanitizer does not exist on macOS.
The web package#
The wasm-abi preset is wasm-release plus ANIRA_WITH_ABI_TESTS, the one test option
WebAssembly allows. It needs the emsdk of Installation / Building and tsc on the
PATH (--emit-tsd runs it):
cmake -P extras/fetch-models.cmake # the model fixtures: anira_em_oom, the browser test and the model-file example read them
cmake --preset wasm-abi && cmake --build --preset wasm-abi
cmake --build --preset wasm-abi --target anira_abi_wasm_gates # the ABI gates under node
cd web && npm install
npm test # unit tests under node, no build needed
npm run build # tsc && vite build
npm run examples # type-check and build web/examples/*
node test/browser/run.mjs chromium # or firefox; needs Playwright (CI: 1.64.1)
The browser test renders the package’s pages and the examples’ check pages (processor-and-engine’s
also under Vite’s dev server), offline contexts of render sizes 256 and 480 where the browser takes
renderSizeHint, and in Chromium a live
AudioContext (live.html); headless Firefox keeps a live context suspended. The wasm gates
are listed in The C ABI registry and its gates.