Audio Nodes#
A handler streams audio through a Wasm Audio Worklet node: await handler.createAudioNode(
audioContext, options) (InferenceHandler.createAudioNode(), the C entry
anira_em_audio_node_create). The node works on an AudioContext and an
OfflineAudioContext; connect audioNode.node into the graph.
The Render Callback#
The node’s render callback is C, on the browser’s audio thread, once per render quantum:
the node has one input per Streamed input slot it serves and one output per Streamed output slot, each with the slot’s channels; their buffers are the handler’s planar host tensors,
[channels, quantum], without a copy: an output tensor is the node’s output;the call is one
anira_handler_process_multiover every slot (withwaitMs,anira_handler_process_multi_wait: an offline render that waits for every inference), an empty tensor at each position the node does not serve;the miss rule:
ANIRA_MISSEDkeeps the miss policy’s fill (bypass, hold the last, the callback), a failure is silence, a success silences the samples an output was not delivered;a quantum other than the one the handler is prepared for is silence (the creation refuses a handler whose Hard block range excludes the context’s render quantum);
it allocates nothing and locks nothing.
Destroy the node (audioNode.destroy()) before the handler.
The Render Geometry#
The handler is prepared under a Hard contract whose block range is the context’s render
geometry, two numbers measureRenderGeometry() gives:
quantum, the smallest blockThe frames of every
process()call: the context’srenderQuantumSize, else 128 where the browser has no such attribute (Firefox and Safari today). It is fixed for the context’s life, an output-device change included (Rendering an Audio Graph); a context chooses it at construction (renderSizeHint, Supported Render Quantum Sizes). A"hardware"context reports 0 until it runs, andmeasureRenderGeometryrefuses it then.burst, the largest blockThe frames a live
AudioContextrenders back to back: each callback of the audio system is a run of quanta with no time between them. An inference started in a run cannot complete before the run’s next quantum, so a handler prepared for one quantum alone misses most quanta live; withhost_block_maxthe burst, the latency covers the run and every quantum is served. No attribute gives the callback’s size across browsers (the spec calls it “a varying number of sample-frames”), so the burst is measured: a probeAudioWorklettimes theprocess()calls and takes the longest run of back-to-back calls and the frames per device callback, whichever is more, in whole quanta (a resampled context counts in its own rate). AnOfflineAudioContextrenders quantum by quantum: its burst is one quantum.
The probe needs a running context (await context.resume() first) and costs time once, after the
page’s start: a 100 ms lead-in, then 300 ms to 2 s until the frames per callback are known (about
1 s in headless Chromium). The geometry is explicit: the page measures it and declares it.
const model = await ModelConfig.fromUrl(anira, new URL('gain.model.json', modelsUrl))
// The contract file leaves the Hard host blocks to the host: one render quantum per call, and
// blocks up to the context's burst (a live context renders a device callback's worth of
// quanta back to back), so the latency the handler reports covers it.
const geometry = await measureRenderGeometry(audioContext)
const contract = (
await ContractHandle.fromUrl(anira, new URL('gain.contract.json', modelsUrl))
).hardHostBlocks(geometry.quantum, geometry.burst, audioContext.sampleRate)
The handler’s latency then covers the burst. A dry path beside the model’s waits as long:
// The wet path is the handler's latency late: the dry path waits as long.
const input = new GainNode(audioContext)
const output = new GainNode(audioContext)
const dry = new DelayNode(audioContext, { maxDelayTime: 1 })
input.connect(audioNode.node).connect(output)
input
.connect(dry)
.connect(new GainNode(audioContext, { gain: dryLevel }))
.connect(output)
const applyLatency = () => {
const latency = handler.getLatency() // samples
// The value setter is setValueAtTime(value, currentTime), and reads back at once.
dry.delayTime.value = latency / audioContext.sampleRate
console.log(`latency: ${latency} samples`)
}
applyLatency()
Nothing re-prepares on its own, and InferenceHandler.createAudioNode() cannot tell a
block range too small for the context: measure first.
Tip
Create the context with renderSizeHint: 'hardware' where the browser has render sizes
(feature-detect: 'renderQuantumSize' in AudioContext.prototype): it renders at the size
the browser deems best for the device instead of a fixed 128 (Chromium chooses 128; Firefox’s
patches for it, bug 2034508, the audio system’s smallest callback). A quantum of the
callback’s size is one process() call per callback, a burst of one quantum
(host_block_min == host_block_max).
// The page's AudioContext: renderSizeHint "hardware" where the browser has render sizes
// (Web Audio API, Render Quantum Sizes; renderQuantumSize is their feature test), so the
// context renders at the size the browser deems best for the device. Elsewhere one
// quantum is 128 frames.
const options: AudioContextOptions & { renderSizeHint?: 'hardware' } = {
sampleRate: 48000,
}
if ('renderQuantumSize' in AudioContext.prototype)
options.renderSizeHint = 'hardware'
const audioContext = new AudioContext(options)
Changing the Geometry of a Running Node#
audioNode.prepare(contract) (anira_em_audio_node_prepare) re-prepares the node’s handler
while the node streams and its context runs; handler.prepare refuses then, since the node’s
render callback may be inside a process call. The node’s callback is held off first (it outputs
silence for those quanta and leaves the handler alone), the main thread waits until the quantum
in flight returned (bounded; the audio thread never waits), the handler re-prepares, the node’s
state that depends on the prepare is rebuilt off the audio thread, and the callback resumes.
The hold is short (about 2 ms in headless Chromium, under one burst), longer when the re-prepare
runs a warm-up pass. It returns the
PlanReport of that prepare, as InferenceHandler.prepare() does; read the
handler’s latency again afterwards (handler.getLatency()).
- Device changes
An output-device change may change the burst (the callback’s size), never the quantum. The page measures again with its streaming nodes held (
hold: each outputs silence and leaves its handler alone while the probe watches, then streams on, whatever happens), declares the new geometry and re-prepares through the node:const g = await measureRenderGeometry(audioContext, { hold: [audioNode] }) contract.hardHostBlocks(g.quantum, g.burst, audioContext.sampleRate) audioNode.prepare(contract)
A node’s rendering on the audio thread hides the callback’s runs from the probe, hence the hold. The probe’s hold is the page’s own (
audioNode.hold) and ends with the measurement, so a page that holds a node for its verdict holds it again after the re-prepare; the hold insideaudioNode.prepareis separate and leaves the page’s alone. The model-file example:// After an output-device change the burst may differ (the quantum does not: it is the // context's for life): measure again with the node held, so its rendering does not hide the // callback's run, re-prepare the running node with the new geometry, pick the plan in the new // report and re-read the latency for the dry path. const applyGeometry = (g: RenderGeometry) => { contract.hardHostBlocks(g.quantum, g.burst, audioContext.sampleRate) // Holds the node off (silence) while the handler re-prepares, and returns the new report. selectPlan(audioNode.prepare(contract)) applyLatency() applyVerdict() // the new figures: held again only when worse than what the user accepted } const remeasure = async () => applyGeometry(await measureRenderGeometry(audioContext, { hold: [audioNode] }))
It is the page’s call, not anira’s: the page switches the device (
setSinkId), and the page decides whether a latency change mid-stream is acceptable. anira does not listen forsinkchange; a page that does:// The page switches the output device, so the page decides when to re-measure: the // latency changes mid-stream, and an OS default-device change may fire no event at all. audioContext.addEventListener( 'sinkchange', () => void example.remeasure().then(showVerdict) )
A change of the operating system’s default device may fire no event at all.
A Model with Several Streams#
A model with several Streamed inputs or outputs (a mixture and a control signal, a stereo pair as two tensors) runs through one node with an input and an output per stream, which is the default:
const audioNode = await handler.createAudioNode(audioContext) // every Streamed slot
voice.connect(audioNode.node, 0, 0) // node input 0: Streamed input slot 0
sidechain.connect(audioNode.node, 0, 1) // node input 1: Streamed input slot 1
audioNode.node.connect(left, 0) // node output 0: Streamed output slot 0
audioNode.node.connect(right, 1) // node output 1: Streamed output slot 1
One node calls anira_handler_process_multi once per quantum over every stream, which keeps
the guarantees of the multi form. Two nodes on one handler, one slot each, are not the
supported form: each calls the single-slot entry, so the streams stay aligned (submission waits
for every stream’s ring), but
ANIRA_MISS_BYPASSpasses the input through only on the call that carries the anchored slot: the other node gets zeros on a miss;the wait credit of
ANIRA_WAIT_CONTRACT(the_waitforms’ budget from the anchored tensor) is never taken for the other call;an
anira_miss_fnis called once per node, each time with one slot, instead of once per block with all of them.
On a miss under ANIRA_MISS_BYPASS, every output carries the anchored input’s block (the
contract refuses BYPASS when an output’s channels are not the anchored input’s).
A Model of Another Dtype#
An audio node streams float32, Web Audio’s samples. A model whose Streamed tensors are of another
dtype (an int16 or a float16 export) runs behind a node only when the contract names float32 as
those tensors’ ring dtype and a processor of the pipeline converts between the float32 rings and
the model’s tensors in preProcess and postProcess (Custom Processors);
nothing in anira converts. Without that, createAudioNode is refused with
ANIRA_ERROR_CONFIG, and the message names the ring dtype and the converting processor:
const contract = ContractHandle.hard(anira, {
hostBlockMin: geometry.quantum,
hostBlockMax: geometry.burst,
hostRate: audioContext.sampleRate, // measured budget, UNTIL_STABLE: the defaults
ringDtypes: { audio_in: ANIRA_DTYPE_F32, audio_out: ANIRA_DTYPE_F32 },
})
A handler that no node drives streams the model’s dtype as it is: the rings hold the spec’s dtype by default, and the Hard process entries take tensors of that dtype.
Options#
inputSlots/outputSlotsThe Streamed slots the node’s inputs feed and its outputs play, input (output) k the k-th, each with the slot’s channels (the node’s
channelCountis the largest input slot’s, explicit and discrete; a slot reads its first channels). Default: every Streamed slot of the side, in slot order, so input 0 feeds the first Streamed input.inputSlots: []is a node without an input.partialStreamsWithout it the creation refuses a Streamed slot of the model that no input or output of the node serves; with it (
ANIRA_EM_AUDIO_NODE_FLAG_PARTIAL_STREAMS) such a slot stays an empty element of the multi call.paramsStatic float32 scalar inputs (rank 1, one element) as AudioParams of the node, by slot:
audioNode.params.get(slot)is a k-rate AudioParam (automatable, default 0); the callback pushes its value to the handler each quantum it changed.staticInputsStatic inputs of any shape and dtype the main thread writes whole,
audioNode.setStaticInput(slot, values)(in the spec’s dtype, asInferenceHandler.setStaticInput()takes it: a typed array of the spec’s element type, any view of exactly the spec’s bytes, or plain numbers converted to it): a seqlock block per slot, so the callback never pushes a torn value and never waits.waitMs0 (the default): never wait, a live context. Above 0: wait up to this long per quantum for its inference, an offline render; the Workers must run, so
await context.ready()first (a prepare that warms up awaited it already).workletModulesModules the context’s
audioWorkletloads before the node: the worklet side of the handler’s JavaScript processor (Custom Processors), bundled into one file each.
The Worklet Scope#
The first node of an audio context starts the context’s worklet thread, a node of n AudioParams
registers the processor anira-node-n once per context, and the creation completes on the main
thread’s event loop (the promise resolves then). Any number of nodes may be created at once on one
context (Promise.all, any interleaving, also while another node of the context is destroyed):
they share the context’s first-node setup, and each creation completes on its own. Before the first
node, createAudioNode loads worklet-scope.js into the scope: the scope has no
performance, which the module’s clock needs on the audio thread. A JavaScript processor’s
worklet phases run on the scope’s registry, which anira boots over the worklet’s instance at every
create; the code there must not allocate, lock or wait (the processor promises it:
realtimePhases names preProcess, postProcess and reset by default).
The module is built with -sASSERTIONS=0: Emscripten’s Web Audio library asserts a
window.AudioContext, which an OfflineAudioContext is not.