Anira Documentation#
#
Anira is a high-performance library designed to enable easy real-time safe integration of neural network inference within audio applications. Compatible with five inference engines, ONNX Runtime, LibTorch, LiteRT, TensorFlow Lite and ExecuTorch, and with engines of your own, anira bridges the gap between advanced neural network architectures and real-time audio processing. In the paper you can find more information about the architecture and the design decisions of anira, as well as extensive performance evaluations with the built-in benchmarking capabilities.
Features#
Real-time Safe Execution: Ensures deterministic runtimes suitable for real-time audio applications
Thread Pool Management: One inference thread pool per process, shared by every handler, avoids oversubscription and runs inferences in parallel
Minimal Latency: Designed to minimize latency while maintaining real-time safety
Built-in Benchmarking: Includes tools for evaluating the real-time performance of neural networks
Comprehensive Inference Engine Support: Integrates ONNX Runtime, LibTorch, LiteRT, TensorFlow Lite and ExecuTorch, and runs engines you register yourself
Flexible Neural Network Integration: Supports a variety of neural network models, including stateful and stateless models
Cross-Platform Compatibility: Works seamlessly on macOS, Linux, Windows, Android and iOS, and in the browser via WebAssembly
Usage#
anira takes its configuration from two JSON files that travel with the model: the model file names the model’s exports per inference engine (paths relative to the file) and describes the tensors, the contract file says how it runs: the inference budget and the warm-up. By default the warm-up pass at prepare measures the budget on the machine; "budget": { "ms": 5.33 } pins it for a reproducible latency. The host blocks are patched in at prepare.
{
"models": [ { "engine": "onnxruntime", "path": "model.onnx" } ],
"inputs": [ { "name": "audio_in", "axes": [["time", 256], ["channel", 1], ["any", 1]] } ],
"outputs": [ { "name": "audio_out", "axes": [["time", 256], ["any", 1]] } ]
}
{ "hard": { "budget": "measured", "warmup": "until_stable" } }
The basic usage of anira is as follows:
#include <anira/anira.hpp>
// The configuration: the model file next to your model, and the contract file
anira::ModelConfig model_config = anira::ModelConfig::from_file("path/to/model.json");
anira::ContractHandle contract = anira::ContractHandle::from_file("path/to/contract.json");
// The context (the inference thread pool) and the handler over the model's inference stage,
// on every engine of this build the model file names
anira::Context context{anira::ContextConfig{}};
anira::Pipeline pipeline;
pipeline.add_inference(model_config);
anira::InferenceHandler inference_handler(context, pipeline);
// Complete the contract with the host blocks; prepare allocates, warms the model up, measures
// it and returns the plans, one per engine the model runs on here
contract.hard_host_blocks(buffer_size, buffer_size, sample_rate);
const anira::PlanReport report = inference_handler.prepare(contract);
// Whether the running plan keeps up on this machine (hold a NOT_REALTIME one and ask the user)
const anira_plan_info row = report.plans()[inference_handler.get_plan()];
const bool realtime = static_cast<anira_rt_verdict>(row.rt_verdict) != ANIRA_RT_NOT_REALTIME;
// Optionally select another plan (the default engine's plan runs first, plan 0 without one) and
// get the latency
inference_handler.set_plan(0);
uint32_t latency_in_samples = inference_handler.get_latency();
// The audio block as a tensor over the host's channel pointers, built once at prepare
std::array<float*, 2> channels{};
anira::Tensor audio = anira::Tensor::from_host_planar<float>(
std::span<float* const>(channels), std::array<int64_t, 2>{2, buffer_size});
// Real-time safe audio processing in process callback of your application
void process(float* const* audio_data, size_t num_samples) noexcept {
channels = {audio_data[0], audio_data[1]};
audio.shape[1] = static_cast<int64_t>(num_samples);
inference_handler.process(audio, 0, audio, 0);
}
// audio_data now contains the processed audio samples
Installation#
C++ Library#
Anira can be easily integrated into your CMake project. You can either add anira as a submodule, download the pre-built binaries from the releases page, or build from source.
Option 1: Add as Git Submodule (Recommended)#
# Add anira repo as a submodule
git submodule add https://github.com/anira-project/anira.git modules/anira
In your CMakeLists.txt:
# Setup your project and target
project(your_project)
add_executable(your_target main.cpp ...)
# Add anira as a subdirectory
add_subdirectory(modules/anira)
# Link your target to the anira library
target_link_libraries(your_target anira::anira)
# Only if your own code calls an engine (e.g. includes onnxruntime_cxx_api.h): link
# its anira::<engine> target too — the same file anira uses, so the process holds
# one copy of the engine. anira itself never needs this line.
target_link_libraries(your_target anira::onnxruntime)
Option 2: Use Pre-built Binaries#
Download pre-built binaries from the releases page.
In your CMakeLists.txt:
# Setup your project and target
project(your_project)
add_executable(your_target main.cpp ...)
# Add the path to the anira library as cmake prefix path and find the package
list(APPEND CMAKE_PREFIX_PATH "path/to/anira")
find_package(anira REQUIRED)
# Link your target to the anira library
target_link_libraries(your_target anira::anira)
# Only if your own code calls an engine: link its anira::<engine> target too
target_link_libraries(your_target anira::onnxruntime)
anira::anira carries anira’s headers but no engine header: the engines are linked privately. Inside
anira’s build tree it also carries the USE_<ENGINE> definitions the bundled examples select on; the
installed anira::anira does not, and code that needs to know which engines a build compiled in asks
anira_enabled_engines (anira::enabled_engines()). A target that calls an engine itself links the
matching engine target — anira::onnxruntime, anira::tflite, anira::litert, anira::libtorch or
anira::executorch — which exists both in the build tree and in the installed package, carries the
engine’s headers, and is the very file anira links, so the process never holds a second copy of the
engine.
Option 3: Build from Source#
git clone https://github.com/anira-project/anira.git
cd anira
cmake . -B build -DCMAKE_BUILD_TYPE=Release -DANIRA_WITH_INSTALL=ON
cmake --build build --config Release --target anira
cmake --install build --prefix /path/to/install/directory
Or with the bundled presets (desktop-install-release / desktop-install-debug):
cmake --preset desktop-install-release
cmake --build --preset desktop-install-release
cmake --install build/desktop/Install/Release --prefix /path/to/install/directory
Without --prefix, the tree is installed to <build dir>/anira-<version>.
The Apache-2.0 core component of tanh-lib that backs anira’s buffers is built into the library and installs nothing of its own: find_package(anira) looks up Threads and the engines alone, so consumers only add the anira prefix to CMAKE_PREFIX_PATH.
C++ Build Options#
By default, LibTorch, ONNXRuntime, LiteRT and ExecuTorch are enabled. You can disable specific engines as needed:
LibTorch:
-DANIRA_WITH_LIBTORCH=OFFOnnxRuntime:
-DANIRA_WITH_ONNXRUNTIME=OFFLiteRT (
LiteRt*C API):-DANIRA_WITH_LITERT=OFF— runs.tflitemodels through LiteRT’s native CompiledModel runtime. Enabled by default; it is the modern TensorFlow-Lite-family engine.TensorFlow Lite (legacy
TfLite*C API):-DANIRA_WITH_TFLITE=ON— the same runtime as LiteRT exposed through the older C API, so the two are mutually exclusive. To use it, disable LiteRT:-DANIRA_WITH_LITERT=OFF -DANIRA_WITH_TFLITE=ON.ExecuTorch:
-DANIRA_WITH_EXECUTORCH=OFF— runs.pteprograms exported ahead-of-time withtorch.export; PyTorch’s edge/mobile inference stack (CPU execution via XNNPACK). Enabled by default; static-only.
Platform / engine support#
anira builds on the targets below; the pre-built engines it downloads ship per target as shared
and/or static. Engine linkage follows BUILD_SHARED_LIBS: a shared anira links shared engines,
a static anira links static engines, and an engine that does not ship the required linkage is
disabled with a warning:
Target |
LibTorch |
ONNXRuntime |
LiteRT |
TFLite (legacy) |
ExecuTorch |
|---|---|---|---|---|---|
macOS |
shared |
shared · static |
shared · static |
shared · static |
static |
macOS |
shared |
shared · static |
shared · static |
shared · static |
static |
macOS |
shared |
shared · static |
shared · static |
shared · static |
static |
Linux |
shared |
shared · static |
shared · static |
shared · static |
static |
Linux |
shared |
shared · static |
shared · static |
shared · static |
static |
Windows |
shared |
shared · static |
shared · static |
shared · static |
static |
Windows |
shared |
shared · static |
shared · static |
shared · static |
static |
|
— |
static |
— |
— |
— |
LibTorch is shared-only (disabled in a static anira build); ExecuTorch is static-only (disabled in a
shared anira build). iOS and WebAssembly are static-only altogether (-DBUILD_SHARED_LIBS=OFF).
LiteRT and TFLite are the same runtime via two C APIs and are mutually exclusive (LiteRT is the
default). In a fully static anira build ExecuTorch cannot be combined with LiteRT or TFLite
(each bundles its own copy of XNNPACK, whose symbols collide in one static image) — ExecuTorch
is then auto-disabled; disable LiteRT/TFLite to build a static anira with ExecuTorch. On
WebAssembly only ONNX Runtime is supported. Backends for Android and iOS are also
published in the anira-project/backends release for
cross-builds. — = not provided.
Pre-built engine binaries are downloaded at configure time from the
anira-project/backends release pinned by
ANIRA_BACKENDS_VERSION. Integrity is checked live: when GitHub is reachable, anira fetches each
asset’s published SHA256 and re-downloads any engine whose archive changed upstream or downloaded
incompletely (the download is verified against that hash). Nothing is pinned in-repo. The source is
configurable (the linkage is not: it follows BUILD_SHARED_LIBS, see above):
Backends release tag:
-DANIRA_BACKENDS_VERSION=v2.4.0(the built-in default).Offline / reproducible builds:
-DANIRA_BACKENDS_SKIP_REMOTE_CHECK=ONskips the GitHub query and reuses whatever is already inmodules/.Bring your own engine (no fork):
-DANIRA_<ENGINE>_ROOTDIR=/path/to/prebuilt(a tree withinclude/+lib/), or a custom source via-DANIRA_<ENGINE>_URL=... -DANIRA_<ENGINE>_SHA256=....
Moreover, the following options are available:
Build the benchmark examples and their fixture (
examples/benchmark/, not part of the library):-DANIRA_WITH_BENCHMARK=ONBuild example applications, plugins and populate example neural models:
-DANIRA_WITH_EXAMPLES=ONBuild anira with tests:
-DANIRA_WITH_TESTS=ONBuild anira with documentation:
-DANIRA_WITH_DOCS=ONDisable the logging system (compiles every anira log call out; anira otherwise logs through tanh-lib’s
thl::Logger):-DANIRA_WITH_LOGGING=OFF
Anira Web (Web / JavaScript)#
Anira is available as the @anira-project/anira package for web applications: anira compiled
to WebAssembly with a thin typed layer over its C ABI. anira starts the page’s threads from C on
one shared memory (the context’s inference threads are Wasm Workers, the audio path a Wasm Audio
Worklet node whose render callback is C), processors and inference engines are written in
TypeScript without rebuilding the module, and the model and contract files of a native host load
unchanged. The 3.x pre-release is on the next dist-tag:
npm install @anira-project/anira@next
See the Anira Web guides.
Building @anira-project/anira from source#
If you want to build the WASM module and JavaScript bindings yourself, you need to provide your own Emscripten SDK (4.0.23, the version CI pins) and TypeScript on the PATH (emcc --emit-tsd runs tsc). The CMake presets expect the EMSDK environment variable to be set to the root of your emsdk installation.
git clone https://github.com/anira-project/anira.git
cd anira
export EMSDK=/path/to/your/emsdk
# Configure and build the WASM module (release); wasm-abi adds the node ABI gates
cmake --preset wasm-release
cmake --build --preset wasm-release
# Build the JavaScript package
cd web
npm install
npm run build
For packaging it locally, use
npm pack
in the web folder, which will create a .tgz file that can be installed with npm or yarn.
Then install the package in your project:
npm install path/to/anira/web/anira-project-anira-x.x.x.tgz
A debug preset is also available via cmake --preset wasm-debug / cmake --build --preset wasm-debug.
Examples#
Bundled examples#
Simple JUCE Audio Plugin: Demonstrates how to use anira in a real-time JUCE / VST3 plugin, with a “Try nonetheless” button for a backend too slow for the machine.
CLAP Plugin Example: Demonstrates how to use anira in a real-time CLAP plugin, with a “Try nonetheless” parameter.
Benchmark: Demonstrates how to use anira for benchmarking of different neural network models, backends and audio configurations, with the Google Benchmark fixture of
examples/benchmark/fixture/to copy into a project of your own.Minimal Inference: Demonstrates how minimal inference applications can be implemented in every engine.
Web examples: A model file in the browser, and a processor and an engine written in TypeScript.
Other examples#
nn-inference-template: A more elaborate JUCE / VST3 plugin that uses anira for real-time safe neural network inference, with a more appealing GUI than the simple JUCE example.
Real-time safety#
anira’s real-time safety is checked in this repository with the rtsan sanitizer.
Citation#
If you use anira in your research or project, please cite either the paper or the software itself:
@inproceedings{ackvaschulz2024anira,
author={Ackva, Valentin and Schulz, Fares},
booktitle={2024 IEEE 5th International Symposium on the Internet of Sounds (IS2)},
title={ANIRA: An Architecture for Neural Network Inference in Real-Time Audio Applications},
year={2024},
volume={},
number={},
pages={1-10},
publisher={IEEE},
doi={10.1109/IS262782.2024.10704099}
}
@software{ackvaschulz2024anira,
author = {Valentin Ackva and Fares Schulz},
title = {anira: an architecture for neural network inference in real-time audio application},
url = {https://github.com/anira-project/anira},
version = {x.x.x},
year = {2024},
}
Contributors#
Sponsor#
License#
This project is licensed under Apache-2.0.
Next Steps#
New to anira: Getting Started, then the Usage Guide.
A processor or an engine of your own: Custom Pre/Post Processing, Custom Engines.
In the browser: Anira Web (TypeScript / WASM).
In a plugin: Plugins and hosts. Something wrong: Troubleshooting & FAQ.
Coming from anira 2.x: Migrating from anira 2.x.