Basic Usage#
This guide walks through the smallest end-to-end Anira Web setup: a bundled model config
run by the module’s built-in ONNX Runtime engine on the context’s inference Worker, streamed
through the handler’s audio node. The code is web/examples/model-file/main.ts, which
npm run examples type-checks and builds and the browser test renders offline in Chromium and
Firefox. The steps are the native host’s (Usage Guide), each a
class over the C handle:
instantiate the module (
Anira) and route the log,a context (
Context): the pool of inference Workers,the model config and the contract, from the same files a native host loads (
ModelConfig,ContractHandle),the pipeline and the handler (
Pipeline,InferenceHandler),the audio node (
InferenceHandler.createAudioNode()),the teardown.
The classes are anira.hpp’s in TypeScript: the same class names and the same member words
(InferenceHandler.getLatency for get_latency, ContractHandle.hardHostBlocks for
hard_host_blocks), the same return values (prepare returns the PlanReport) and
defaults; where the web differs (Anira.create, the worker module, Context.ready, the
audio node, measureRenderGeometry, fromUrl, the log pump, destroy() instead of RAII,
a thrown AniraError instead of a returned status), the class’s doc comment says so.
Each class hands out its C handle with native(), and the module’s bindings are in abi
(AniraBindings, one method per anira_* entry, generated from the ABI registry),
so anything the classes do not cover is one anira.abi.anira_* call away. A failed call throws
an AniraError with the entry’s message and its status.
Note
The page must be cross-origin isolated (Cross-Origin-Opener-Policy: same-origin,
Cross-Origin-Embedder-Policy: require-corp): the module runs on a shared
WebAssembly.Memory. See Installation / Building.
1. The Module and the Log#
import {
Anira,
Context,
ContractHandle,
InferenceHandler,
ModelConfig,
Pipeline,
anira_engine,
anira_rt_verdict,
measureRenderGeometry,
} from '@anira-project/anira'
const anira = await Anira.create() // the module on this thread, its shared memory
anira.setLogSink() // records to the console (pass a callback for your own sink)
const stopLogPump = anira.startLogPump() // the web has no drain thread
The hook replaces the module’s platform sink (the console by level), so each record reaches the console once, through the console method of its level (Errors and logging).
Anira.create() makes the module’s shared memory: 128 MiB growing to 2 GiB as handlers
need it, { memory: { initial, maximum } } to size it, equal sizes to fix it
(the memory).
2. A Context#
const context = Context.create(anira) // the pool: half the cores, as natively
The context’s inference threads are Wasm Workers anira starts from C when the context is
created: as many as the context config says, by default ANIRA_THREADS_AUTO (half the cores, at
least one) as on a native host; threads sets a count. A Worker enters its loop only once the
page’s thread yields, so the Workers boot while the page awaits its next promise, and
Context.ready() resolves once every one of them is in its loop (without a pool, once one
of the page’s own threads is). A page with JavaScript engines names the module every Worker
imports first, workerModule (Custom Inference Engines); the built-in engines need
none.
3. Configs Once#
The model and contract files of a native host load unchanged (modelsUrl: the directory
that serves extras/models/model-pool/):
const model = await ModelConfig.fromUrl(anira, new URL('gain.model.json', modelsUrl))
// The contract file leaves the Hard host blocks to the host: one render quantum per call, and
// blocks up to the context's burst (a live context renders a device callback's worth of
// quanta back to back), so the latency the handler reports covers it.
const geometry = await measureRenderGeometry(audioContext)
const contract = (
await ContractHandle.fromUrl(anira, new URL('gain.contract.json', modelsUrl))
).hardHostBlocks(geometry.quantum, geometry.burst, audioContext.sampleRate)
ModelConfig.fromUrl() parses the file (anira_model_config_from_json) and, for every
model entry whose engine the module compiled in (the ONNX Runtime one) or a JavaScript engine’s,
fetches the entry’s file relative to the config’s URL and attaches it as bytes
(anira_model_config_set_model_bytes); the other entries (a LibTorch or a TFLite one) keep
their paths unfetched. A JavaScript engine reads its entry’s bytes from that memory in every
Worker (load(model), Custom Inference Engines; the ONNX Runtime Web engine does), so
each model file is fetched once. Serve the model files beside the config the way its paths say. A contract
file may leave the Hard host blocks to the host: measureRenderGeometry() gives them, blocks
from the render quantum up to the context’s burst at the context’s rate. A live context renders
a device callback’s worth of quanta back to back, so host_block_max must cover that burst for the
inferences to keep up; the latency the handler reports then covers it too, and a live context
must run when it is measured (Audio Nodes).
ModelConfig.fromJson() and ContractHandle.hard() build the same in code.
4. The Pipeline and the InferenceHandler#
// The contract's warm-up runs at prepare on the pool's Workers, which start with the context and
// enter their loops once this thread yields: wait for them first.
await context.ready()
const pipeline = Pipeline.create(anira).addInference(model)
const handler = InferenceHandler.create(context, pipeline)
// Every prepare starts the handler on the config's default plan and returns the report of the
// plans it compiled; a host that picks one looks it up in that report, here the ONNX entry on
// the built-in ONNX Runtime engine.
const selectPlan = (report: PlanReport) => {
const plan = report.find({ engine: anira_engine.ANIRA_ENGINE_ONNXRUNTIME })
if (plan === null) throw new Error('anira: no plan on the ONNX Runtime engine')
handler.setPlan(plan)
}
selectPlan(handler.prepare(contract))
pipeline.destroy() // the handler copied it
await context.ready() comes before the prepare: a contract warms up by default (a measured
budget, UNTIL_STABLE), and its warm-up pass runs every plan through the session on the
Workers, so a prepare before they run is refused with ANIRA_ERROR_INVALID_STATE naming
context.ready(). ContractHandle.hard() takes the same defaults as anira::Hard:
budgetMs left out is the budget the pass measures, warmup left out is UNTIL_STABLE;
budgetMs with warmup: ANIRA_WARMUP_NONE is an explicit budget without a pass, which needs no
running Worker. The report’s rows carry what the pass measured and the plan’s real-time factor and
verdict (warmupIterations, warmupOutcome, measuredMs, medianMs, rtFactor,
rtParallel, rtVerdict; The plan report); a re-prepare of the same model reuses its
figure and runs no pass.
Every prepare starts the handler on the config’s default plan. A host that runs another one
looks its index up in the report with PlanReport.find(), which takes the fields of
anira::PlanQuery (variant, engine, engineId, provider, providerId; a field
left out matches every plan): { engine: anira_engine.ANIRA_ENGINE_ONNXRUNTIME } for a built-in
engine, { engine: anira_engine.ANIRA_ENGINE_CUSTOM, engineId: 'org.example.double' } for a
JavaScript engine, { provider } for a provider whatever its engine. It returns null when no
plan matches and throws an AniraError for a query against the pair rule (an id beside
a built-in value, ANIRA_ENGINE_CUSTOM without one), and the index goes to
InferenceHandler.setPlan(). A re-prepare starts on the default again, so the example
picks its plan in the report of every prepare.
5. The Audio Node#
const audioNode = await handler.createAudioNode(audioContext, {
params: [GAIN_SLOT],
waitMs,
})
const gainParam = audioNode.params.get(GAIN_SLOT)!
gainParam.value = gain
A page that plays live acts on the verdict of the plan it runs: the example logs every plan’s
row after each prepare, holds the node (audioNode.hold(true): silent, no inference, the dry path
plays) while the plan is NOT_REALTIME until a “Try nonetheless” button releases it, keeps the
worst verdict the user accepted, and never holds an offline render (waitMs above 0):
// What each prepare measured, per plan: the budget, how the warm-up pass found it, and the
// real-time verdict, an estimate at prepare on this machine under its load of the moment. On a
// live context the plan the handler runs, when it is not real-time here, would drop out: the
// node is held (silent, no inference; the dry path plays) until the user agrees. An offline
// render (waitMs > 0) does not run against the clock and is never held. The example keeps the
// worst verdict the user accepted, and asks again only when a re-prepare finds a worse one.
const live = waitMs === 0
let accepted: number = anira_rt_verdict.ANIRA_RT_UNMEASURED
let held = false
const applyVerdict = () => {
const plans = handler.planReport().plans()
plans.forEach((row, index) =>
console.log(
`plan ${index}: budget ${row.budgetMs.toPrecision(3)} ms (measured ` +
`${row.measuredMs.toPrecision(3)} ms, median ${row.medianMs.toPrecision(3)} ms, ` +
`${row.warmupIterations} warm-up inferences, outcome ${row.warmupOutcome}); ` +
`real-time factor ${row.rtFactor.toPrecision(3)} of ${row.rtParallel}, verdict ` +
`${row.rtVerdict}`
)
)
const row = plans[handler.getPlan()]
held =
live &&
row.rtVerdict === anira_rt_verdict.ANIRA_RT_NOT_REALTIME &&
row.rtVerdict > accepted
audioNode.hold(held)
if (held)
console.warn(
`anira: the plan runs at ${row.rtFactor.toFixed(2)}x real time on this machine ` +
`(${row.rtParallel} parallel inferences): the node stays held until "Try nonetheless"`
)
}
applyVerdict()
const tryNonetheless = () => {
accepted = handler.planReport().plans()[handler.getPlan()].rtVerdict
held = false
audioNode.hold(false)
}
The handler’s latency, in samples, delays the dry path beside the model’s, so the two stay aligned:
// The wet path is the handler's latency late: the dry path waits as long.
const input = new GainNode(audioContext)
const output = new GainNode(audioContext)
const dry = new DelayNode(audioContext, { maxDelayTime: 1 })
input.connect(audioNode.node).connect(output)
input
.connect(dry)
.connect(new GainNode(audioContext, { gain: dryLevel }))
.connect(output)
const applyLatency = () => {
const latency = handler.getLatency() // samples
// The value setter is setValueAtTime(value, currentTime), and reads back at once.
dry.delayTime.value = latency / audioContext.sampleRate
console.log(`latency: ${latency} samples`)
}
applyLatency()
The node is a Wasm Audio Worklet node whose render callback is C: per quantum it hands the
node’s buffers to anira_handler_process as tensors and applies the miss rule
(Audio Nodes). Connect audioNode.node into the graph. params serves float32 Static scalars as AudioParams of the node;
staticInputs serves Static inputs of any shape through audioNode.setStaticInput. For an
OfflineAudioContext render that waits for every inference, pass waitMs (the Workers
are running already: the prepare awaited context.ready()).
6. Teardown#
const teardown = () => {
input.disconnect()
audioNode.node.disconnect()
audioNode.destroy() // the render callback stops calling the handler
handler.destroy()
contract.destroy()
model.destroy()
context.destroy() // the last context stops the pool
stopLogPump()
}
C objects are freed by their destroy, never by the garbage collector.
Complete Examples#
web/examples/model-file/ is the page of this guide: main.ts builds the chain above and
plays a tone through it, check.ts renders it offline with the gain automated. Its Vite config
serves the repository’s extras/models at /extras/models/.
web/examples/processor-and-engine/ is a complete page that writes its own processor and engine
in TypeScript (Custom Processors, Custom Inference Engines): a gain
processor in the worklet and a stand-in engine on the inference Worker, bundled by Vite. npm run
examples type-checks and builds it, and the browser test renders its check page offline in
Chromium and Firefox, built and under Vite’s dev server. Its main.ts:
// The example: a JavaScript processor that applies a gain before the model (gain-processor.ts, in
// the audio worklet), a JavaScript engine standing in for the model (double-engine.ts, on the
// context's inference Workers), and the gain as the model's Static input, the AudioParam of the
// handler's audio node. Every piece is TypeScript over the npm package; nothing rebuilds the wasm.
// Beside the model's (wet) path a dry path, delayed by the handler's latency.
import {
ANIRA_EM_PROCESSOR_HOOK_PREPARE,
ANIRA_EM_PROCESSOR_HOOK_PRE,
ANIRA_EM_PROCESSOR_HOOK_RESET,
ANIRA_EM_PROCESSOR_HOOK_UNPREPARE,
ANIRA_ENGINE_FLAG_NEEDS_NO_MODEL,
Anira,
Context,
ContractHandle,
InferenceHandler,
ModelConfig,
Pipeline,
anira_rt_verdict,
measureRenderGeometry,
type AniraAudioNode,
} from '@anira-project/anira'
// The processor's module on this thread too: its prepare and unprepare run here.
import { GAIN_SLOT, INPUT_GAIN_PROCESSOR } from './gain-processor'
import { DOUBLE_ENGINE } from './double-engine'
// The worker module and the worklet module, each bundled into one script by Vite; their URLs.
import workerModule from './double-engine.ts?worker&url'
import processorModule from './gain-processor.ts?worker&url'
// 128 samples per inference: the fixed Time extent is the model's window.
const WINDOW = 128
export type GainExample = {
/** Connect the source here: it feeds the model and the dry path. */
input: AudioNode
/** The wet output and the delayed dry signal, mixed. */
output: AudioNode
audioNode: AniraAudioNode
/** The dry path's delay: the handler's latency. */
dry: DelayNode
/** The handler's latency in samples: the wet output is the input this much later. */
getLatency: () => number
/** The gain's AudioParam: set, ramp or automate it. */
gain: AudioParam
/** Measure the context's geometry again and re-prepare: after the output device changed. */
remeasure: () => Promise<void>
/**
* The real-time verdict (`anira_rt_verdict`) of the plan the handler runs, from the last
* prepare: an estimate at prepare, measured on this machine under its load of the moment.
*/
verdict: () => number
/**
* Whether the node is held: on a live context, the plan is not real-time here and the user has
* not agreed to run it. A held node is silent and starts no inference; the dry path plays.
*/
waitsForConsent: () => boolean
/** The user's "Try nonetheless": the node streams, and the verdict counts as accepted. */
tryNonetheless: () => void
teardown: () => void
}
/**
* Build the example on `audioContext`. `waitMs` lets each quantum wait for its inference (an
* offline render); `dryLevel` is the dry path's level. A live context must be running.
*/
export async function createGainExample(
audioContext: BaseAudioContext,
{ gain = 1, waitMs = 0, dryLevel = 0.5 } = {}
): Promise<GainExample> {
// 1. The module, its log.
const anira = await Anira.create()
anira.setLogSink()
const stopLogPump = anira.startLogPump()
// 2. A context whose pool of Wasm Workers imports the engine's module, and the model
// config: a streamed audio input, the Static input `gain` (slot 1), a streamed output.
const context = Context.create(anira, { workerModule }) // the pool: half the cores
const streamed = (name: string) => ({
name,
dtype: 'float32',
role: 'streamed',
axes: [
['batch', 1],
['channel', 1],
['time', WINDOW],
],
})
const model = await ModelConfig.fromJson(
anira,
JSON.stringify({
// No path: the engine declares ANIRA_ENGINE_FLAG_NEEDS_NO_MODEL.
models: [{ engine: DOUBLE_ENGINE }],
inputs: [
streamed('audio_in'),
{ name: 'gain', dtype: 'float32', role: 'static', axes: [['any', 1]] },
],
outputs: [streamed('audio_out')],
})
)
// 3. The pipeline: the engine and the processor, under the names their modules define them by.
const pipeline = Pipeline.create(anira)
.addInference(model)
.addEngine(DOUBLE_ENGINE, { flags: ANIRA_ENGINE_FLAG_NEEDS_NO_MODEL })
.addProcessor(INPUT_GAIN_PROCESSOR, {
phases:
ANIRA_EM_PROCESSOR_HOOK_PRE |
ANIRA_EM_PROCESSOR_HOOK_RESET |
ANIRA_EM_PROCESSOR_HOOK_PREPARE |
ANIRA_EM_PROCESSOR_HOOK_UNPREPARE,
})
// 4. The handler and its Hard contract: one render quantum per call, at the context's rate, in
// bursts up to the context's (a live context renders a device callback's worth of quanta
// back to back; the latency the handler reports covers it). The library's defaults: the
// budget the warm-up pass measures at prepare, on the pool's Workers, which start with the
// context and enter their loops once this thread yields: wait for them first.
const geometry = await measureRenderGeometry(audioContext)
const contract = ContractHandle.hard(anira, {
hostBlockMin: geometry.quantum,
hostBlockMax: geometry.burst,
hostRate: audioContext.sampleRate,
})
await context.ready()
const handler = InferenceHandler.create(context, pipeline)
handler.prepare(contract)
pipeline.destroy()
// 5. The audio node: the processor's module loaded into the worklet, the gain an AudioParam.
const audioNode = await handler.createAudioNode(audioContext, {
params: [GAIN_SLOT],
waitMs,
workletModules: [processorModule],
})
const gainParam = audioNode.params.get(GAIN_SLOT)!
gainParam.value = gain
// 6. What each prepare measured, per plan: the budget, how the warm-up pass found it, and the
// real-time verdict, an estimate at prepare on this machine under its load of the moment.
// On a live context a plan that is not real-time here would drop out: the node is held
// (silent, no inference; the dry path plays) until the user agrees. An offline render
// (waitMs > 0) does not run against the clock and is never held. The example keeps the
// worst verdict the user accepted, and asks again only when a re-prepare finds a worse one.
const live = waitMs === 0
let accepted: number = anira_rt_verdict.ANIRA_RT_UNMEASURED
let held = false
const applyVerdict = () => {
const plans = handler.planReport().plans()
plans.forEach((row, index) =>
console.log(
`plan ${index}: budget ${row.budgetMs.toPrecision(3)} ms (measured ` +
`${row.measuredMs.toPrecision(3)} ms, median ${row.medianMs.toPrecision(3)} ms, ` +
`${row.warmupIterations} warm-up inferences, outcome ${row.warmupOutcome}); ` +
`real-time factor ${row.rtFactor.toPrecision(3)} of ${row.rtParallel}, verdict ` +
`${row.rtVerdict}`
)
)
const row = plans[handler.getPlan()]
held =
live &&
row.rtVerdict === anira_rt_verdict.ANIRA_RT_NOT_REALTIME &&
row.rtVerdict > accepted
audioNode.hold(held)
if (held)
console.warn(
`anira: the plan runs at ${row.rtFactor.toFixed(2)}x real time on this machine ` +
`(${row.rtParallel} parallel inferences): the node stays held until "Try nonetheless"`
)
}
applyVerdict()
const tryNonetheless = () => {
accepted = handler.planReport().plans()[handler.getPlan()].rtVerdict
held = false
audioNode.hold(false)
}
// 7. The dry path, delayed by the handler's latency, read again after every re-prepare.
const input = new GainNode(audioContext)
const output = new GainNode(audioContext)
const dry = new DelayNode(audioContext, { maxDelayTime: 1 })
input.connect(audioNode.node).connect(output)
input
.connect(dry)
.connect(new GainNode(audioContext, { gain: dryLevel }))
.connect(output)
// The value setter is setValueAtTime(value, currentTime), and reads back at once.
const applyLatency = () => {
dry.delayTime.value = handler.getLatency() / audioContext.sampleRate
}
applyLatency()
// After an output-device change: measure again with the node held (its rendering would hide
// the callback's run) and re-prepare the running node, then re-read the latency.
const remeasure = async () => {
const g = await measureRenderGeometry(audioContext, { hold: [audioNode] })
contract.hardHostBlocks(g.quantum, g.burst, audioContext.sampleRate)
audioNode.prepare(contract)
applyLatency()
applyVerdict() // the new figures: held again only when worse than what the user accepted
}
// 8. The teardown: the node stops calling the handler first.
const teardown = () => {
input.disconnect()
audioNode.node.disconnect()
audioNode.destroy()
handler.destroy()
contract.destroy()
model.destroy()
context.destroy()
stopLogPump()
}
return {
input,
output,
audioNode,
dry,
getLatency: () => handler.getLatency(),
gain: gainParam,
remeasure,
verdict: () => handler.planReport().plans()[handler.getPlan()].rtVerdict,
waitsForConsent: () => held,
tryNonetheless,
teardown,
}
}
/**
* The page: a tone through the example, its gain on a slider, started and stopped by a button;
* the verdict line, and "Try nonetheless" while the node waits for the user's consent.
*/
export function mountGainExample(
button: HTMLButtonElement,
slider: HTMLInputElement,
tryButton: HTMLButtonElement,
verdictLine: HTMLElement
): void {
let running: { audioContext: AudioContext; example: GainExample } | null = null
const showVerdict = () => {
const example = running?.example
tryButton.hidden = !example?.waitsForConsent()
verdictLine.textContent = !example
? ''
: example.waitsForConsent()
? 'The model is not real-time on this machine: the node stays silent (the dry path ' +
'plays) until "Try nonetheless".'
: example.verdict() === anira_rt_verdict.ANIRA_RT_MARGINAL
? 'The model is marginal on this machine: it may drop out under load.'
: ''
}
tryButton.onclick = () => {
running?.example.tryNonetheless()
showVerdict()
}
button.onclick = async () => {
button.disabled = true
if (running) {
running.example.teardown()
await running.audioContext.close()
running = null
button.textContent = 'Start'
} else {
// The page's AudioContext: renderSizeHint "hardware" where the browser has render sizes
// (Web Audio API, Render Quantum Sizes; renderQuantumSize is their feature test), so the
// context renders at the size the browser deems best for the device. Elsewhere one
// quantum is 128 frames.
const options: AudioContextOptions & { renderSizeHint?: 'hardware' } = {
sampleRate: 48000,
}
if ('renderQuantumSize' in AudioContext.prototype)
options.renderSizeHint = 'hardware'
const audioContext = new AudioContext(options)
await audioContext.resume() // the burst is measured on a running context
const example = await createGainExample(audioContext, {
gain: Number(slider.value),
})
// The page's call: it switches the output device and accepts the latency change.
audioContext.addEventListener(
'sinkchange',
() => void example.remeasure().then(showVerdict)
)
const tone = new OscillatorNode(audioContext, { frequency: 220 })
const level = new GainNode(audioContext, { gain: 0.1 })
tone.connect(level).connect(example.input)
example.output.connect(audioContext.destination)
tone.start()
running = { audioContext, example }
button.textContent = 'Stop'
}
showVerdict()
button.disabled = false
}
slider.oninput = () =>
running?.example.gain.setTargetAtTime(Number(slider.value), 0, 0.01)
}
Raw Calls#
Every C entry is a method of anira.abi under its C name, its out parameters returned, its
anira_error made for it:
const config = anira.abi.anira_context_config_create()
anira.abi.anira_context_config_set_threads(config, 2, anira_wait_strategy.ANIRA_WAIT_SPIN_BACKOFF)
const raw = anira.abi.anira_context_create(config)
anira.abi.anira_context_config_destroy(config)
Records are read through layout (layout.anira_tensor, …), and enums and defines come
from the package by their C names.