Custom Processors#
A JavaScript processor is the pipeline’s one processor written in TypeScript: the counterpart of a C
anira_processor_desc (Custom Pre/Post Processing), defined by name in every scope that runs
one of its phases, with no rebuild of the module.
Defining a Processor#
import { defineProcessor } from '@anira-project/anira/scope'
defineProcessor('org.example.my-processor', (env) => ({
preProcess(ctx, prepared) { /* ... */ },
}))
The factory builds the scope’s instance over its env: env.abi (the bindings over the
scope’s module instance) and env.module (its heap views: env.module.HEAPF32, …). Each
method receives pointers into the shared memory: ctx an anira_processor_ctx (layout),
prepared what this handler’s prepare stored. A method returns an anira_status
(nothing is ANIRA_OK); one that throws fails its call. A processor without preProcess or
postProcess runs anira’s default body there.
The phases run in different scopes (Architecture): preProcess, postProcess and
reset in the audio worklet under a Hard contract, beforeInference and
afterInference on the inference Workers, init, prepare, unprepare and
release on the main thread. Define the processor in each scope its phases need: import the
module on the main thread, list it in the context’s workerModule for the Workers’ phases,
and in createAudioNode’s workletModules for the worklet’s. One name is one processor
(keyOf()).
Adding It to the Pipeline#
pipeline.addProcessor('org.example.my-processor', {
phases:
ANIRA_EM_PROCESSOR_HOOK_PRE | ANIRA_EM_PROCESSOR_HOOK_RESET | ANIRA_EM_PROCESSOR_HOOK_PREPARE | ANIRA_EM_PROCESSOR_HOOK_UNPREPARE,
})
phases names the phases that cross into JavaScript; a phase without its bit runs anira’s default
and never crosses. Each bit is 1 << ANIRA_PHASE_*, one index space with
anira::Processor::k_* of anira.hpp (ANIRA_EM_PROCESSOR_HOOK_AFTER is
Processor::k_after_inference, 16), the lifecycle phases (prepare, release, reset, unprepare,
init) above the data phases. realtimePhases is the processor’s real-time promise per phase,
stated, never derived from phases: an OR of ANIRA_PHASE_BIT(phase), the same bits, over the
hooks that allocate nothing, lock nothing and wait for nothing. A Hard contract requires the bit of
every preProcess, postProcess and reset hook (they run on the audio thread), and the
default is those three bits; beforeInference and afterInference run on the Workers and
need none. Where the C++ realtime_phases() defaults to no promise, the web’s defaults to the
three phases of the audio thread, since a JavaScript processor’s usual hook runs there:
pipeline.addProcessor('org.example.heavy-hooks', {
phases: ANIRA_EM_PROCESSOR_HOOK_PRE | ANIRA_EM_PROCESSOR_HOOK_BEFORE | ANIRA_EM_PROCESSOR_HOOK_AFTER,
realtimePhases: ANIRA_PHASE_BIT(anira_phase.ANIRA_PHASE_PRE_PROCESS), // the hooks may allocate
})
The warm-up. Under any warm-up but ANIRA_WARMUP_NONE, unless the model’s figure is
reused, the prepare’s pass runs
beforeInference and afterInference on the Workers (never preProcess or
postProcess), timed with the engine call, so a hook must accept an entry preProcess never
filled. Under a measured budget prepare may run twice in one prepare: prepare, the pass, then
unprepare and prepare again when the pass moved the entry count (The lifecycle).
The processor’s id, for the model’s measured figure, is org.anira.web.processor.<key>, the key of
its name: processors defined under one name share the figure (Budget and warm-up).
The Example’s Processor#
INPUT_GAIN_PROCESSOR of web/examples/processor-and-engine/ scales the model’s audio input by
the Static input gain in preProcess, ramping from the previous block’s gain across the
block. Its prepare and unprepare run on the main thread and allocate its per-handler memory
in the shared heap; preProcess and reset run in the worklet and reach that memory through
prepared; preProcess reads the tensors through the processor accessors
(_anira_processor_input_tensor, anira_tensor_data_f32). The same module is imported by the
page (main.ts) and bundled into the worklet:
// The JavaScript processor of the example: it scales the model's audio input by the Static input
// `gain` in pre_process, ramping from the previous block's gain to the new one across the block.
// One module, two scopes: the main thread imports it for prepare and unprepare (they run there),
// the audio worklet loads it (main.ts: workletModules) for preProcess and reset, which run inside
// the C render callback. Both define the processor under one name; each scope builds its own
// instance.
import { anira_status, defineProcessor, layout } from '@anira-project/anira/scope'
/** The processor's name: the pipeline adds it under this name (Pipeline.addProcessor). */
export const INPUT_GAIN_PROCESSOR = 'org.anira-example.input-gain'
/** The model config's input slots the processor reads. */
export const AUDIO_SLOT = 0
export const GAIN_SLOT = 1
// What prepare allocates per handler, in the shared memory, so the worklet's instance reaches it:
// a tensor record the accessors fill, the gain the last block ended on, and whether that gain is
// known yet (0 after prepare and after a reset: the next block starts on its own gain).
const TENSOR = 0
const CURRENT_GAIN = layout.anira_tensor.size
const PRIMED = CURRENT_GAIN + 4
const PREPARED_BYTES = PRIMED + 4
defineProcessor(INPUT_GAIN_PROCESSOR, ({ abi, module }) => ({
// Main thread, at the handler's prepare: the per-handler memory, freed in unprepare.
prepare(_prepareInfo: number, outPrepared: number): number {
const prepared = module._malloc(PREPARED_BYTES)
module.HEAPU8.fill(0, prepared, prepared + PREPARED_BYTES)
module.HEAPU32[outPrepared >>> 2] = prepared
return anira_status.ANIRA_OK
},
unprepare(prepared: number): void {
module._free(prepared)
},
// The audio thread, once per quantum inside the render callback: no allocation, no object
// literal, no closure (the processor promises it: addProcessor's default realtimePhases names
// pre_process, post_process and reset).
preProcess(ctx: number, prepared: number): number {
// Pop the input ring into the model's audio tensor (and materialise the Static inputs).
const popped = module._anira_processor_default_pre_process(ctx)
if (popped < 0) return popped
const tensor = prepared + TENSOR
if (
module._anira_processor_input_tensor(ctx, GAIN_SLOT, tensor) !==
anira_status.ANIRA_OK
) {
return anira_status.ANIRA_ERROR_INVALID_ARGUMENT
}
const f32 = module.HEAPF32
const target = f32[abi.anira_tensor_data_f32(tensor) >>> 2]
if (
module._anira_processor_input_tensor(ctx, AUDIO_SLOT, tensor) !==
anira_status.ANIRA_OK
) {
return anira_status.ANIRA_ERROR_INVALID_ARGUMENT
}
const audio = abi.anira_tensor_data_f32(tensor) >>> 2
const count = abi.anira_tensor_num_elements(tensor)
const primed = module.HEAPU32[(prepared + PRIMED) >>> 2] !== 0
const current = primed ? f32[(prepared + CURRENT_GAIN) >>> 2] : target
const step = (target - current) / count
for (let i = 0; i < count; i++) f32[audio + i] *= current + step * (i + 1)
f32[(prepared + CURRENT_GAIN) >>> 2] = target
module.HEAPU32[(prepared + PRIMED) >>> 2] = 1
return anira_status.ANIRA_OK
},
// The audio thread, on anira_handler_reset: the history is gone, so is the ramp.
reset(_ctx: number, prepared: number): void {
module.HEAPU32[(prepared + PRIMED) >>> 2] = 0
},
}))