Models and tensors#
A model config lists the model’s exports, one entry per engine, and describes its tensors with tensor specs. It travels with the model; its file form is the model file (The three files).
#include <anira/anira.hpp>
// The tensors: your names, the model's dtypes, a role, tagged axes
anira::TensorSpec in("audio_in", ANIRA_DTYPE_F32, ANIRA_ROLE_STREAMED);
in.axis(0, ANIRA_AXIS_BATCH, 1)
.axis(1, ANIRA_AXIS_CHANNEL, 1)
.axis(2, ANIRA_AXIS_TIME, 15380)
.window(15380, 15380, 13332); // 15380 samples per inference, 2048 of them fresh: hop 2048
anira::TensorSpec out("audio_out", ANIRA_DTYPE_F32, ANIRA_ROLE_STREAMED);
out.axis(0, ANIRA_AXIS_BATCH, 1)
.axis(1, ANIRA_AXIS_CHANNEL, 1)
.axis(2, ANIRA_AXIS_TIME, 2048); // no window: the fixed Time extent is the window
anira::TensorSpec gain("gain", ANIRA_DTYPE_F32, ANIRA_ROLE_STATIC);
gain.axis(0, ANIRA_AXIS_ANY, 1);
// The exports: one entry per engine you ship
anira::ModelConfig cfg;
cfg.add_model_path(ANIRA_ENGINE_LIBTORCH, "model.pt"); // takes (batch, channel, time)
uint32_t i = cfg.add_model_path(ANIRA_ENGINE_ONNXRUNTIME, "model.onnx");
cfg.tensor_name(i, "audio_in", "input.1"); // the export calls it "input.1"
i = cfg.add_model_path(ANIRA_ENGINE_LITERT, "model.tflite");
cfg.tensor_layout(i, "audio_in", std::array{0u, 2u, 1u}); // the export holds (batch, time, channel)
cfg.input(in).input(gain).output(out); // copied: the specs may go out of scope
cfg.default_engine(ANIRA_ENGINE_ONNXRUNTIME);
Every builder method is one C call (The C API); one that fails throws anira::Error
(.status and what(); a std::runtime_error). The handles are move-only RAII objects, and
what you add to another handle is copied; a spec is move-only too, so a chain that starts on a
temporary goes straight into the config (as above) rather than into a named variable. Nothing is
checked across handles here: every semantic rule (does the window fit the axis, does the default
engine name an entry) is checked when the handler is created and again at prepare, the same way for
code and for the model file. native() hands out the C handle.
Tensor specs#
A spec has a canonical name, a dtype and a role. The canonical name is your name for the tensor: the rest of the configuration (tensor records, the anchor, contract keys, messages) refers to it, and it need not match anything in an export. Names are unique across a model’s inputs and outputs. The dtype is the model’s, one of the ten of Dtypes, and the tensor keeps it end to end.
Role |
What it is |
|---|---|
|
A stream with exactly one Time axis, consumed window by window: audio. At most one Channel axis, which maps onto the ring’s channels. |
|
One whole value per inference, no Time axis: a gain, a conditioning vector, a score. Set and read whole (Static tensors). |
|
One half of a declared state pair, an output fed back into an input (Declared state). No Hard entry carries it. |
|
The whole submitted buffer is one tensor (a frame, an image): the payload of an Async job. A
Hard contract refuses it at prepare ( |
Axes are set by index in the model’s memory order, each with a tag and an extent:
ANIRA_AXIS_BATCH, CHANNEL, TIME, HEIGHT, WIDTH, FEATURE and ANY (no
semantics); NCHW against NHWC is another order of tags. A Streamed spec’s Time extent may be
ANIRA_DYNAMIC. On a Static or Buffer spec a Channel axis (at most one) is just an axis of the
shape, and the tensor moves as the product of its extents.
Windows#
window(window_min, window_max, overlap) says how many Time elements one inference consumes and
how many of them it keeps from the previous one. The hop, the advance per inference, is the
window minus the overlap. The receptive-field model above takes 15380 samples and yields 2048 fresh
ones: a window of 15380 with an overlap of 13332.
window_min == window_maxis a fixed window;window_max = ANIRA_UNBOUNDEDan open one. A flexible window is pinned at prepare by the contract’s host blocks (Flexible windows and the anchor hop).The window is optional. Without one, a fixed Time extent is the window (overlap 0), and a dynamic Time axis makes the window flexible,
[1, ANIRA_UNBOUNDED]. A declared window always wins. An overlap without a window isANIRA_ERROR_CONFIG.A derived window never enters the spec:
SpecView::windowreads{0, 0, 0}andanira_model_config_to_jsonwrites back only declared windows. The window each slot runs at, its hop and whether it was derived are in the plan report (anira_plan_slot.window,hop,window_derived) and the prepare log.An output’s window is aligned like an input’s: the model writes the whole window, channel
cfrom elementc * window, its newest hop at the tail, and the defaultpost_processpushes that last hop of every channel. A model whose output has the shape of its input needs no processor; an output that overlap-adds does it in its own processor.
latency(elements) declares an output’s internal delay, which the reported latency adds
(Latency). time_ratio(num, den) declares a tensor whose Time axis advances at another
rate than the anchor’s; (0, 0), the default, derives it from a fixed window’s hop, and a
flexible window without a ratio advances with the anchor (1/1).
Model entries#
Add an entry only for the engines you ship. Whether an engine is in the build is decided at
anira_handler_create (and checked again at prepare), not here, so one config serves every
build.
From a file:
add_model_path(engine, path), oradd_model_path("com.example.npu", path)for a custom engine’s id. In C the pair:anira_model_config_add_model_path(cfg, ANIRA_ENGINE_CUSTOM, "com.example.npu", path, &i, &err), withNULLas the id for a built-in engine. anira never opens a custom entry’s path; the engine’sloaddoes. An entry whose id no engine of the pipeline serves isANIRA_ERROR_NOT_SUPPORTEDatanira_handler_create.From bytes:
add_model_bytes(engine, bytes, ownership, release, ctx)((id, bytes, …)for a custom engine) over astd::span<const std::byte>, a resource compiled into a plugin.ANIRA_BYTES_COPY(the default) copies;ANIRA_BYTES_BORROWkeeps your pointer until the config is destroyed, then callsrelease(bytes, ctx)if given.set_model_bytes(i, …)swaps the source of an entry loaded from a file: the JUCE example loads its model file’s text withfrom_jsonand swaps each entry for the compiled-in bytes, matched bymodel_engine(i).An entry point: a LibTorch or ExecuTorch file may carry several methods (RAVE’s
encodeanddecode); name one with theentryextension:cfg.model_ext(i, anira::ext::Entry{"decode"});
Binding the tensors to an export#
Each entry carries an optional record per tensor, keyed by canonical name, for what differs in that export:
tensor_name(i, canonical, export_name): the export’s name for the tensor, read off ONNX Runtime’s graph names, the TFLite / LiteRT signature key (args_0,output_0; the tensor name for a file without signatures), or LibTorch’s argument name (inputs only). A name the file lacks fails prepare withANIRA_ERROR_CONFIGand the names it has. A name on a side the engine binds by position (every ExecuTorch tensor, a LibTorch output) isANIRA_ERROR_NOT_SUPPORTEDat create.tensor_layout(i, canonical, axes): the order the export holds the spec’s axes in, as spec axis indices ({0, 2, 1}: the file’s axis 1 is spec axis 2).ANIRA_AXIS_INSERTstands for a unit axis the file has and the spec lacks; a spec axis left out must have extent 1. Moving only unit axes costs nothing; moving an axis of another extent is a transpose, refused at prepare (What this pre-release does not run yet).
Without a record, a tensor binds by its canonical name where the engine’s side has a tensor of
that name, else by position: ONNX Runtime’s graph order, the signature’s key order (the names sorted) on TFLite
and LiteRT, the method’s argument order on LibTorch and ExecuTorch. Either way prepare checks the
bound tensor’s dtype and every static extent against the spec (ANIRA_ERROR_CONFIG naming both;
a dynamic extent of the file matches anything), and every tensor of the file must be bound exactly
once. The plan report says how each slot was bound (anira_plan_slot.binding).
Providers#
A provider is the engine’s twin in a backend id: a value of anira_provider
(ANIRA_PROVIDER_CPU, CUDA, COREML, XNNPACK, …) or, for one the enum does not name, a
string in the engine’s own words (provider_id: an ONNX Runtime execution provider’s name, a
LiteRT accelerator’s "gpu" or "npu", a custom engine’s listed name).
The pair rule, wherever an engine or provider travels: an id is set exactly beside
ANIRA_ENGINE_CUSTOM/ANIRA_PROVIDER_CUSTOM. Any other combination isANIRA_ERROR_INVALID_ARGUMENT.ANIRA_PROVIDER_NONE(0) names no provider: it unpins an entry and clears the default; a candidate or a provider option set refuses it.ANIRA_PROVIDER_CPUis the engine’s plain CPU path, served by every built-in engine (ONNX Runtime’sCPUExecutionProvider, LibTorch, TFLite, ExecuTorch without a delegate, LiteRT’s CPU accelerator); XNNPACK is a provider of its own.An entry is neutral by default and runs on any provider of its engine; one built for a provider (an ExecuTorch export lowered for a delegate, an ONNX Runtime
.ortfile) is pinned:model_provider(i, ANIRA_PROVIDER_XNNPACK)ormodel_provider(i, "com.example.npu"),"provider"in the model file (no key: a neutral entry;"default"and"none"are refused withANIRA_ERROR_JSON). A pinned entry runs on its pin alone. Two entries of one engine are legal only when their pins differ (one export per provider); two with the same pin, or two neutral ones, areANIRA_ERROR_CONFIG. An ExecuTorch entry pinned to a provider its method does not use is refused at prepare.
Which providers serve here is the context’s to say (Capabilities), which one a plan runs on the candidate’s (The pipeline).
The starting plan#
default_engine(engine) (or (id)) and default_provider(provider) (or (id)) name where
the handler starts: the first plan in plan-table order of the default engine (of any engine, when
none is set) on the default provider; without a default provider, the default engine’s first plan;
with neither, plan 0. ANIRA_ERROR_CONFIG at anira_handler_create for what the config alone
decides: a default engine with no entry, a default provider that every entry of the default engine
(every entry, without one) is pinned away from (a neutral entry runs on any). When no plan runs the default here (not in the build, not a candidate, not usable), the
handler starts on the default engine’s first plan, else plan 0, and prepare logs one Warning; it is
never a refusal.
State, instances, anchor#
state(ANIRA_MODEL_STATEFUL): the model carries state across inferences (an RNN, RAVE), so its inferences run one at a time and in submission order. A declared State pair makes a model stateful whatever this says.max_instances(n): the ceiling of a stateless model’s parallel instances, shared by every handler that runs the model. The default,ANIRA_INSTANCES_AUTO, is the inference pool’s thread count (at least 1; 1 when the host brings its own threads), resolved at prepare; an explicit count is clamped to the pool’s size, and the read-back and the JSON keep the sentinel. Each instance is a full copy of the model in memory, so a large model may setmax_instances(1); fewer instances can mean more latency (Latency, \(n\)). How the instances are dispatched: Architecture.anchor(canonical): the Streamed tensor the host blocks count in (Host blocks), and the one every other stream’s time ratio is stated against. The default is the first Streamed input, else the first Streamed output (a generator). Name one when the host’s stream is another tensor: a decoder of latent frames anchors on its audio output. A name that is no Streamed tensor isANIRA_ERROR_CONFIGat prepare.
Reading a config back#
A config answers what its setters and loaders stored, a loaded one alike:
input_count() / output_count() (State specs included), input_spec(i) /
output_spec(i) (an anira::SpecView: name(), dtype(), role(), ndim(),
axis(k), window(), time_ratio(), latency(), state_source(); valid until the
config is changed, moved or destroyed), default_engine() / default_provider() (an
anira::EngineRef / anira::ProviderRef; {ANIRA_ENGINE_NONE, ""} /
{ANIRA_PROVIDER_NONE, ""} when none is set), model_engine(i), model_provider(i)
(ANIRA_PROVIDER_NONE for a neutral entry), model_path(i), model_bytes(i), state(),
max_instances(), anchor() (empty for the default), tensor_name(i, canonical) (empty
without a record), tensor_layout(i, canonical) (empty for the spec’s order) and
model_ext<anira::ext::Entry>(i) (std::nullopt without one). A view’s axis(k) reads
{ANIRA_AXIS_ANY, 0} for a hole and at or beyond ndim(). model_engine(i) and
model_provider(i) are noexcept and answer the NONE pair out of range; the other indexed
reads throw ANIRA_ERROR_INVALID_ARGUMENT out of range, and model_path(i) /
model_bytes(i) ANIRA_ERROR_INVALID_STATE on an entry of the other source. The counts and
scalars are noexcept. TensorSpec::view() is the same view over a spec being built.
Extensions (ext(value) / ext_json(kind, text), on every handle) attach a typed record by
kind; anira_registered_ext_kinds lists what a build understands, and a kind nobody consumes
fails prepare by name, so a typo never becomes a default.