Contracts#
A contract names the run. A Hard contract is the real-time stream: the host blocks, the inference budget, the warm-up, what a missed block delivers, the wait ratio. An Async contract is for jobs without a real-time deadline (refused at prepare in this pre-release, What this pre-release does not run yet). Its file form is the contract file (The three files).
// In code: an anira::Hard aggregate, the library's defaults for everything left out
anira::Hard hard{
.host_block_min = 1, .host_block_max = 512, // blocks of 1..512 samples
.host_rate = 48000.0, // of the anchor, per second
.on_miss = ANIRA_MISS_BYPASS, // budget: measured, warm-up: until stable
};
anira::ContractHandle contract(hard);
// From the contract file next to the model, with the host's blocks patched in
anira::ContractHandle loaded = anira::ContractHandle::from_file("model.contract.json");
loaded.hard_host_blocks(1, max_block, sample_rate);
anira::Hard back = loaded.hard(); // every field read back, to change and mint again
contract.hard() reads a Hard handle back into the aggregate (the budget rounded to the
nanosecond; an Async handle throws ANIRA_ERROR_WRONG_CONTRACT), and contract.kind() tells the
two kinds apart; a Hard setter on an Async contract throws ANIRA_ERROR_WRONG_CONTRACT too.
anira::Contract is the std::variant of anira::Hard and anira::Async, and the handle
is minted from either.
Host blocks#
host_block_min, host_block_max and host_rate count in elements of the model’s anchor
tensor (State, instances, anchor): the block range in samples and the rate in samples per second for
an audio anchor. A contract file usually carries none; the host patches them in once it knows its
device (hard_host_blocks).
host_block_min == host_block_maxis a fixed block, at the lowest latency, for a host that never changes it.host_block_minbelow the maximum lets the host deliver any smaller block; the latency anira reserves covers every size of the range held from call to call. A plugin declares the range its host promises: JUCE’sprepareToPlaynames the largest block only (hard_host_blocks(1, samplesPerBlock, sampleRate)), CLAP’sactivateboth ends. A host that changes the size between calls can still miss a block in rare sequences (a full block right after a short one that completed a hop, #228).The blocks are doubles, both 0 (unset) or both finite and above 0 with
min <= max; anything else isANIRA_ERROR_INVALID_ARGUMENT. Unset blocks are legal on the contract and refused at prepare (ANIRA_ERROR_CONFIG, “Hard host blocks missing”).host_block_maxis also the most one call may carry: a Hard entry refuses a larger block.
Flexible windows and the anchor hop#
The host blocks pin every flexible window (window_min < window_max, declared or derived) at
prepare. One inference consumes the same span of the stream on every tensor, so anira chooses one
anchor hop, the anchor elements one inference consumes: the anchor, every Streamed tensor with
a time ratio and every flexible window advance that hop times their ratio (1/1 for a flexible
window without one), and a fixed window without a ratio keeps its own hop.
A time ratio
num/denmakes the anchor hop a multiple ofden(reduced), and each window bounds it.Of the hops that leave every such tensor a whole hop within its window, anira takes the largest at or below
host_block_max(one inference per block, or several when the block is larger), else the smallest above it (one inference every few blocks).A fixed anchor window is the anchor hop whatever the block.
What no hop satisfies is
ANIRA_ERROR_CONFIG, naming the tensor, the hops it needs and the hops the tensors before it allow.
A model whose audio input is flexible and whose latent output runs at time_ratio(1, 2048) gets
an anchor hop of 2048 under a block of 3000 (one or two inferences per call), of 4096 under 4096,
and of 2048 under 512 (one inference every fourth call).
Fractional blocks#
A model with no stream at the host rate (a control-rate model whose streams move one frame per 2048 audio samples) has an anchor whose block is a fraction of a sample. Anchor on a stream at the host rate whenever the model has one; the fractional block is for the model that has none, and it is a fixed block: a plugin, whose host promises only a largest block, anchors on a stream at the host rate and declares its range there.
// a host block of host_block samples at sample_rate: 512 samples are 0.25 of a frame
const double block = host_block / 2048.0;
contract.hard_host_blocks(block, block, sample_rate / 2048.0);
// per call k = 1, 2, ...: the whole samples accumulated since the last call
const auto moved = static_cast<int64_t>(std::floor(k * block));
latent_in.shape[1] = params_out.shape[1] = moved - moved_so_far; // 0 or 1
moved_so_far = moved;
anira_handler_process(handler, &latent_in, 0, ¶ms_out, 0, &delivered);
anira takes the block as the rational it recovers from it (a relative tolerance of 1e-6: 1/3 and
1/4 exactly) and assumes the host moves a sample only once a whole one has accumulated: by its
k-th call, floor(k * b) samples of a stream whose block is b. The latency is computed for
the sample arriving on the last call of its accumulation. ceil(b) is the most one call may carry,
on every Streamed slot: a call handed more is refused with ANIRA_ERROR_INVALID_ARGUMENT before
anything moves; a pop within the bound that the stream cannot cover returns ANIRA_MISSED. A
flexible window under a fractional block gets the smallest anchor hop the windows and time ratios
allow: a latent input [1, unbounded] under 0.25 runs on one frame every fourth call, and an
output at time_ratio(2, 1) beside it gets a window of 2.
Budget and warm-up#
The warm-up pass also measures the inference time; a measured budget is the maximum of its last four inferences; under an explicit budget the timings are only reported.
using namespace std::chrono_literals;
anira::Hard measured{.host_block_min = 512, .host_block_max = 512, .host_rate = 48000};
anira::Hard explicit_budget{.host_block_min = 512, .host_block_max = 512, .host_rate = 48000,
.budget = 5ms, // a reproducible latency
.warmup = ANIRA_WARMUP_FIXED, .warmup_iterations = 8};
The budget (
anira::Hard::budget, ananira::Budget) is measured by default (anira::Budget{},Budget::measured(),ANIRA_BUDGET_MEASURED, a fresh C contract’s). Astd::chronoduration is an explicit budget (ANIRA_BUDGET_EXPLICIT): the worst-case time per inference at the pinned window.budget.kind()/budget.value()read it back. An inference that exceeds the budget produces a dropout.The pass runs at
prepare, once the session is sized (for the explicit budget, or for one clock step under a measured one) and the registered engines and the processor are prepared: ordinary inferences of every plan, in table order, through the handler’s session on the inference threads, each timed on the thread that runs it (the state’s feed and capture, the processor’sbefore_inferenceandafter_inference, the engine call; neverpre_process,post_processor the wait for a thread).The warm-up modes:
ANIRA_WARMUP_UNTIL_STABLE(the default) stops at the first inference whose last four times lie within 20 % of their median, or within one clock step when that is more (a browser coarsens its clock), at most 32 (ANIRA_WARMUP_OUTCOME_CAPPEDthen);ANIRA_WARMUP_FIXEDrunswarmup_iterationsper plan (fewer than four put the cold start into the figure: takeUNTIL_STABLEor five or more);ANIRA_WARMUP_NONEruns none and needs an explicit budget. A measured budget withNONEorFIXED0 isANIRA_ERROR_CONFIG.Per plan, the worst case: under a measured budget every plan row takes its own figure (
anira_plan_info.budget_ms, unrounded, without a margin) and the session runs the slowest plan’s. The session is then sized again for it, which starts the stream over, so nothing of the pass reaches the stream or the declared state; when that changes the entry count, the registered engines and the processor are unprepared and prepared again with the final record.A measured latency depends on the machine and its load at the prepare; an explicit budget gives a reproducible one. A model too slow for the stream is not refused: the session is sized for one host block’s burst, the stream misses blocks under the miss policy, and the plan’s real-time verdict says so (The plan report).
The pass needs an inference thread in its loop: a context without a pool needs a host thread in
anira_inference_thread_run_loop(on the web,await context.ready()first), elseANIRA_ERROR_INVALID_STATE, as for an inference of the pass not complete within 10 s (a hang guard). An inference of the pass that fails fails the prepare withANIRA_ERROR_ENGINEnaming the plan. A Hard prepare never returnsANIRA_ERROR_BUDGET(the Async contract’s). Apart from the pass, every executor of a built-in engine runs one inference over its own scratch when it is made (at load, or at an exclusive handler’s prepare).
The model’s figure. The inference time belongs to the model: the core keeps what every pass
measured, under either budget kind, keyed like its pool of loaded models (the engine, the provider
and its options, the model, its entry, its tensors at the pinned window, its instances), except that
a model given as bytes counts by a content hash of its bytes (two plugin instances borrowing one
embedded model are one model) and a registered engine by its id (anira.v2.custom by its engine
object), plus the
processor’s inference hooks (nothing for a processor without before_inference /
after_inference, else its id, anira_processor_desc.id / anira::Processor(id), else its
registration).
A later prepare of the model, by another handler or the same one at another block size or rate, reuses the figure and runs no pass (
ANIRA_WARMUP_OUTCOME_REUSED): identical plugins report identical latency, and a re-prepare is quick. A prepare whose every plan reuses its figure sizes the session once and needs no running inference thread. The real-time factor and verdict are still the contract’s.A figure answers a warm-up at least as thorough:
UNTIL_STABLEanswers every one,FIXED naFIXED mwithm <= n; a weaker one is measured again and replaced. Two prepares of one model on two threads measure it once;ANIRA_WARMUP_NONEneither measures nor reads it.The figures live as long as the core’s state (until
anira_release_core_if_idle;anira_shutdownalone keeps them). An idle machine’s figure can underestimate the time under later load (memory bandwidth, clock speed, SMT siblings); a marginal verdict’s headroom is the margin for that, not a guarantee. Give a processor with inference hooks an id, so that identical plugins share the figure; registered engines of one id share it, however many objects the instances create (anira.v2.customexcepted).
Miss policy#
A block whose inference has not completed when the host pops it is a miss: the entry returns
ANIRA_MISSED (a success) with a delivered count of 0, and on_miss decides what the buffers
hold. The samples it stood in for are discarded when they arrive, never delivered late: a delivered
block is the stream at exactly its latency, and a host that stays ahead of the stream keeps
missing.
|
A missed block, and a dropped hop |
|---|---|
|
The block the same call pushed into the anchored input ( |
|
The last block the output delivered (one block-sized buffer per channel, allocated at prepare). A dropped hop: the head of that block. |
|
Silence, what the 2.x runtime did (the bundled RAVE encoder and decoder files use it). |
|
A function of the host fills it (below). |
The policies speak about Streamed outputs; a Static output holds the handler’s stored value on a missed block too (Static tensors).
Dropped hops. When every inference slot of the handler is still busy (an inference ran far
over its budget, the machine stalled), the hops a call completes get no inference: each one costs
exactly one hop of the policy’s fill at its own stream position, behind the inferences submitted
before it. The stream stays on its hop grid and is back at its latency once the late inferences
are collected. rt_error reports a drop as ANIRA_ERROR_CAPACITY (Errors and logging). A dropped
hop never reaches the model: a stateful model’s state continues from the last hop it ran.
The miss callback is an anira_miss_fn, set with anira_contract_hard_set_miss_fn(contract,
fn, user_data) (Hard::miss_fn / miss_user_data, ContractHandle::hard_miss_fn); the
policy without a function is ANIRA_ERROR_CONFIG at prepare, and a contract file can say
"on_miss": "callback" but not carry the function.
static anira_status ANIRA_CALL fade_out(anira_handler* handler,
const anira_tensor* inputs, uint32_t num_inputs,
const anira_tensor* outputs, uint32_t num_outputs,
void* user_data) ANIRA_NONBLOCKING;
anira_contract_hard_set_on_miss(contract, ANIRA_MISS_CALLBACK);
anira_contract_hard_set_miss_fn(contract, fade_out, &my_state); /* either order */
A missed block: called once on the thread that called the entry, with one tensor per slot in slot order (the arrays the entry was handed): a slot the call did not carry, and every State slot, is an empty tensor (
shape[1] == 0), a pop passes empty inputs, and a process form’s input block is already pushed and intact. Fill every output whoseshape[1]is above 0 and returnANIRA_OK; any other status zero-fills them. The entry still returnsANIRA_MISSED. Static output elements are filled with the stored value first, and the function may overwrite them.A dropped hop: called once per hop, at the drop, inside the entry whose call dropped it, with hop-shaped arrays: every Streamed input slot carries the dropped hop (
[channels, hop]of the ring dtype, packed, the handler’s memory, valid for the call), every Streamed output slot a[channels, hop]tensor to fill, every other slot an empty tensor; a status other thanANIRA_OKplaces zeros. The entry’s own return and counts are not touched by a hop; a function that must tell a hop from a block reads the flags (a hop’s tensors are packed, withoutANIRA_TENSOR_PLANAR; a block’s are the caller’s own). The handler keeps the fills of the lastceil(latency / hop) + 2dropped hops and places an older one as zeros.The function is real-time code (no allocation, lock or system call), never calls a Hard entry,
resetorprepareof the same handler, and outlives the prepared handler with itsuser_data. Under clang it is declaredANIRA_NONBLOCKINGitself, since the conversion toanira_miss_fncannot add the attribute.
Wait ratio, ring dtypes, latencies, host domains, edge cost#
hard.wait_ratio = 0.5; // a _wait entry may wait half a block
hard.ring_dtypes = {{"audio_in", ANIRA_DTYPE_I16}}; // the host streams int16 into a float32 model
hard.latencies = {{"audio_out", 4096}}; // the reported latency of this output
hard.edge_cost = ANIRA_EDGE_COPY; // no plan that crosses a domain
contract.host_domain("audio_in", ANIRA_DOMAIN_HOST); // where the host end lives (both kinds)
Wait ratio (
wait_ratio, default 0): the share of a block period a_waitentry waits for its inference underANIRA_WAIT_CONTRACT(The _wait twins); theANIRA_NONBLOCKINGentries never wait. It selects the completion primitive at prepare (a semaphore above 0, a 1 ms poll otherwise), and the latency counts the wait credit whether or not the host calls a_waitentry (Latency).Ring dtypes (
ring_dtypes,hard_ring_dtype): the element type of the host’s samples per Streamed tensor, by canonical name, where it differs from the model’s. Nothing converts: it needs a processor that converts (Dtypes).Latencies (
latencies,hard_latency): the stream latency of one output, in its samples, replacing the computed figure and priming its receive ring.ANIRA_ERROR_CONFIGat prepare below the model’s internal latency, on an input, a Static output or an unknown name; aboveINT32_MAXthe setter refuses it. Every output never named keeps the computed figure.Host domains (
host_domain, both kinds): where the host end of one tensor lives, by canonical name,ANIRA_DOMAIN_HOSTfor every tensor never named. The ring, the model tensor, the Static store, the state buffers and every processor phase live there, and the planner joins it to the engine’s domain with an edge (the plan report’s slot row: the host domain is an input’sfrom_domainand an output’sto_domain, the engine’s the other end; no edge where the two coincide). An unknown name isANIRA_ERROR_CONFIG; any domain butANIRA_DOMAIN_HOSTisANIRA_ERROR_NOT_SUPPORTEDin this pre-release.Edge cost (
edge_cost, both kinds;contract.edge_cost(...)): the worst edge class a plan may take for any slot (Edges, movers and memory).std::nullopt, the default, is no limit, an unknown edge included (in Canira_contract_set_edge_cost(c, ANIRA_EDGE_UNKNOWN), andanira_contract_edge_costanswersANIRA_EDGE_UNKNOWNwhile no limit is set). Every limit refuses anANIRA_EDGE_UNKNOWNedge. At prepare a plan above the limit is dropped once its model has loaded (an Info record names each slot, its class, its mover and why), and the report holds the eligible plans alone; with none left prepare throwsANIRA_ERROR_EDGE_UNREACHABLE. The session is then made again over the remaining plans, so no model loads twice (an exclusive session’s own executor is made, and warmed up, a second time).
Async#
anira::Async holds an optional deadline, on_late, priority, lanes,
max_in_flight, delivery and the edge_cost above: jobs without a real-time deadline, the
offline posture. An Async contract is refused at prepare in this pre-release.