Custom Processors#

A JavaScript processor is the pipeline’s one processor written in TypeScript: the counterpart of a C anira_processor_desc (Custom Pre/Post Processing), defined by name in every scope that runs one of its phases, with no rebuild of the module.

Defining a Processor#

import { defineProcessor } from '@anira-project/anira/scope'

defineProcessor('org.example.my-processor', (env) => ({
  preProcess(ctx, prepared) { /* ... */ },
}))

The factory builds the scope’s instance over its env: env.abi (the bindings over the scope’s module instance) and env.module (its heap views: env.module.HEAPF32, …). Each method receives pointers into the shared memory: ctx an anira_processor_ctx (layout), prepared what this handler’s prepare stored. A method returns an anira_status (nothing is ANIRA_OK); one that throws fails its call. A processor without preProcess or postProcess runs anira’s default body there.

The phases run in different scopes (Architecture): preProcess, postProcess and reset in the audio worklet under a Hard contract, beforeInference and afterInference on the inference Workers, init, prepare, unprepare and release on the main thread. Define the processor in each scope its phases need: import the module on the main thread, list it in the context’s workerModule for the Workers’ phases, and in createAudioNode’s workletModules for the worklet’s. One name is one processor (keyOf()).

Adding It to the Pipeline#

pipeline.addProcessor('org.example.my-processor', {
  phases:
    ANIRA_EM_PROCESSOR_HOOK_PRE | ANIRA_EM_PROCESSOR_HOOK_RESET | ANIRA_EM_PROCESSOR_HOOK_PREPARE | ANIRA_EM_PROCESSOR_HOOK_UNPREPARE,
})

phases names the phases that cross into JavaScript; a phase without its bit runs anira’s default and never crosses. Each bit is 1 << ANIRA_PHASE_*, one index space with anira::Processor::k_* of anira.hpp (ANIRA_EM_PROCESSOR_HOOK_AFTER is Processor::k_after_inference, 16), the lifecycle phases (prepare, release, reset, unprepare, init) above the data phases. realtimePhases is the processor’s real-time promise per phase, stated, never derived from phases: an OR of ANIRA_PHASE_BIT(phase), the same bits, over the hooks that allocate nothing, lock nothing and wait for nothing. A Hard contract requires the bit of every preProcess, postProcess and reset hook (they run on the audio thread), and the default is those three bits; beforeInference and afterInference run on the Workers and need none. Where the C++ realtime_phases() defaults to no promise, the web’s defaults to the three phases of the audio thread, since a JavaScript processor’s usual hook runs there:

pipeline.addProcessor('org.example.heavy-hooks', {
  phases: ANIRA_EM_PROCESSOR_HOOK_PRE | ANIRA_EM_PROCESSOR_HOOK_BEFORE | ANIRA_EM_PROCESSOR_HOOK_AFTER,
  realtimePhases: ANIRA_PHASE_BIT(anira_phase.ANIRA_PHASE_PRE_PROCESS),   // the hooks may allocate
})

The warm-up. Under any warm-up but ANIRA_WARMUP_NONE, unless the model’s figure is reused, the prepare’s pass runs beforeInference and afterInference on the Workers (never preProcess or postProcess), timed with the engine call, so a hook must accept an entry preProcess never filled. Under a measured budget prepare may run twice in one prepare: prepare, the pass, then unprepare and prepare again when the pass moved the entry count (The lifecycle). The processor’s id, for the model’s measured figure, is org.anira.web.processor.<key>, the key of its name: processors defined under one name share the figure (Budget and warm-up).

The Example’s Processor#

INPUT_GAIN_PROCESSOR of web/examples/processor-and-engine/ scales the model’s audio input by the Static input gain in preProcess, ramping from the previous block’s gain across the block. Its prepare and unprepare run on the main thread and allocate its per-handler memory in the shared heap; preProcess and reset run in the worklet and reach that memory through prepared; preProcess reads the tensors through the processor accessors (_anira_processor_input_tensor, anira_tensor_data_f32). The same module is imported by the page (main.ts) and bundled into the worklet:

// The JavaScript processor of the example: it scales the model's audio input by the Static input
// `gain` in pre_process, ramping from the previous block's gain to the new one across the block.
// One module, two scopes: the main thread imports it for prepare and unprepare (they run there),
// the audio worklet loads it (main.ts: workletModules) for preProcess and reset, which run inside
// the C render callback. Both define the processor under one name; each scope builds its own
// instance.
import { anira_status, defineProcessor, layout } from '@anira-project/anira/scope'

/** The processor's name: the pipeline adds it under this name (Pipeline.addProcessor). */
export const INPUT_GAIN_PROCESSOR = 'org.anira-example.input-gain'

/** The model config's input slots the processor reads. */
export const AUDIO_SLOT = 0
export const GAIN_SLOT = 1

// What prepare allocates per handler, in the shared memory, so the worklet's instance reaches it:
// a tensor record the accessors fill, the gain the last block ended on, and whether that gain is
// known yet (0 after prepare and after a reset: the next block starts on its own gain).
const TENSOR = 0
const CURRENT_GAIN = layout.anira_tensor.size
const PRIMED = CURRENT_GAIN + 4
const PREPARED_BYTES = PRIMED + 4

defineProcessor(INPUT_GAIN_PROCESSOR, ({ abi, module }) => ({
  // Main thread, at the handler's prepare: the per-handler memory, freed in unprepare.
  prepare(_prepareInfo: number, outPrepared: number): number {
    const prepared = module._malloc(PREPARED_BYTES)
    module.HEAPU8.fill(0, prepared, prepared + PREPARED_BYTES)
    module.HEAPU32[outPrepared >>> 2] = prepared
    return anira_status.ANIRA_OK
  },

  unprepare(prepared: number): void {
    module._free(prepared)
  },

  // The audio thread, once per quantum inside the render callback: no allocation, no object
  // literal, no closure (the processor promises it: addProcessor's default realtimePhases names
  // pre_process, post_process and reset).
  preProcess(ctx: number, prepared: number): number {
    // Pop the input ring into the model's audio tensor (and materialise the Static inputs).
    const popped = module._anira_processor_default_pre_process(ctx)
    if (popped < 0) return popped

    const tensor = prepared + TENSOR
    if (
      module._anira_processor_input_tensor(ctx, GAIN_SLOT, tensor) !==
      anira_status.ANIRA_OK
    ) {
      return anira_status.ANIRA_ERROR_INVALID_ARGUMENT
    }
    const f32 = module.HEAPF32
    const target = f32[abi.anira_tensor_data_f32(tensor) >>> 2]
    if (
      module._anira_processor_input_tensor(ctx, AUDIO_SLOT, tensor) !==
      anira_status.ANIRA_OK
    ) {
      return anira_status.ANIRA_ERROR_INVALID_ARGUMENT
    }
    const audio = abi.anira_tensor_data_f32(tensor) >>> 2
    const count = abi.anira_tensor_num_elements(tensor)

    const primed = module.HEAPU32[(prepared + PRIMED) >>> 2] !== 0
    const current = primed ? f32[(prepared + CURRENT_GAIN) >>> 2] : target
    const step = (target - current) / count
    for (let i = 0; i < count; i++) f32[audio + i] *= current + step * (i + 1)
    f32[(prepared + CURRENT_GAIN) >>> 2] = target
    module.HEAPU32[(prepared + PRIMED) >>> 2] = 1
    return anira_status.ANIRA_OK
  },

  // The audio thread, on anira_handler_reset: the history is gone, so is the ramp.
  reset(_ctx: number, prepared: number): void {
    module.HEAPU32[(prepared + PRIMED) >>> 2] = 0
  },
}))