Class anira::InferenceThread

class InferenceThread : public anira::HighPriorityThread

Inheritance diagram for anira::InferenceThread:

Collaboration diagram for anira::InferenceThread:

Thread class for executing neural network inference operations.

The InferenceThread class provides a dedicated thread for executing neural network inference operations in real-time audio processing contexts. It manages a concurrent queue of inference requests and processes them with minimal latency while maintaining thread safety and real-time performance guarantees.

On native builds, this inherits from HighPriorityThread and owns its own OS thread. Under Emscripten there is no owned OS thread — a JS Worker drives the loop externally by calling run_loop(), and start()/stop() simply flip an atomic flag. This is required because each WASM worker instance shares memory with the main instance; spawning OS threads from C++ inside a worker would interact badly with the shared allocator.

A moodycamel::ConsumerToken is pre-allocated in the constructor (which must run on the thread that owns the allocator — the main WASM instance in browser builds). Using an explicit token makes execute() fully allocation-free.

Public Functions

InferenceThread(moodycamel::ConcurrentQueue<InferenceData> &next_inference)

Constructor that initializes the inference thread with a task queue.

Creates an inference thread that will process inference requests from the provided concurrent queue. The thread is not started automatically and must be explicitly started using the start() method.

Constructor that initializes the inference thread with a task queue

Creates an inference thread that will process inference requests from the provided concurrent queue. The thread is not started automatically and must be explicitly started using the start() method.

Parameters:

• next_inference – Reference to a thread-safe concurrent queue containing inference data structures to process

• next_inference – Reference to a thread-safe concurrent queue containing inference data structures to process

~InferenceThread() override

bool execute()

Executes a single iteration of inference processing.

Attempts to dequeue and process one inference request from the queue. This method is designed to be called repeatedly in a loop and provides efficient processing with automatic backoff when no work is available.

The method handles:

• Dequeuing inference data from the concurrent queue

• Processing the inference request through the appropriate session

• Managing CPU usage through exponential backoff strategies

• Thread-safe access to shared data structures

Note

This method is real-time safe and designed for repeated calls in a high-frequency processing loop.

Returns:

True if an inference operation was executed, false if no work was available

void run_loop()

Run the main processing loop with exponential backoff.

Natively, this is invoked by the inherited HighPriorityThread via the run() override. Under Emscripten, JS Workers call this directly. Returns when should_exit() becomes true.