Class anira::Context

class Context

Collaboration diagram for anira::Context:

Singleton context class managing global inference resources and session coordination.

The Context class serves as a singleton manager for all neural network inference resources, including thread pools, backend processors, and session management. It provides centralized coordination for multiple inference sessions while maintaining efficient resource sharing and thread safety across the entire inference system.

Key responsibilities:

• Managing singleton instance lifecycle and configuration

• Coordinating inference thread pool with configurable size

• Managing backend processor instances (LibTorch, ONNX, TensorFlow Lite)

• Session creation, management, and cleanup

• Thread-safe concurrent queue management for inference requests

• Resource pooling and efficient allocation/deallocation

The Context uses a singleton pattern to ensure:

• Global resource coordination across multiple inference instances

• Efficient sharing of expensive resources (thread pools)

• Centralized configuration and lifecycle management

• Thread-safe access to shared components

See also

ContextConfig, SessionElement, InferenceThread, PrePostProcessor, BackendBase

Note

This class is thread-safe and manages its own lifecycle. All access should be through the static interface methods rather than direct instantiation.

Public Functions

Context(const ContextConfig &context_config)

Constructor that initializes the context with specified configuration.

Creates a new context instance with the provided configuration settings. This constructor is should not be called directly. Use get_instance() to obtain a context instance.

Parameters:

context_config – Configuration settings for thread pool size, backend preferences, etc.

~Context() = default

Destructor that cleans up all context resources.

Properly shuts down the thread pool, releases all backend processors, and cleans up any remaining sessions or inference data.

void prepare_session(const std::shared_ptr<SessionElement> &session, HostConfig new_config, std::vector<long> custom_latency = {})

Prepares a session for processing with new audio configuration.

Configures the specified session with new audio host settings and optional custom latency values. This method handles buffer allocation, latency calculation, and session state updates.

Parameters:

• session – Shared pointer to the session to prepare

• new_config – New host configuration with audio settings

• custom_latency – Optional vector of custom latency values for each tensor

void new_data_submitted(const std::shared_ptr<SessionElement> &session)

Notifies the context that new data has been submitted for a session.

Signals to the inference system that new audio data is available for processing by the specified session. This triggers the inference pipeline to begin processing the submitted data.

Parameters:

session – Shared pointer to the session that has new data available

void new_data_request(const std::shared_ptr<SessionElement> &session)

Requests new data processing for a session.

Requests that the inference system process data for the specified session. This is used for scheduling and managing inference operations. The request is processed immediately.

Parameters:

session – Shared pointer to the session requesting data processing

void new_data_request(const std::shared_ptr<SessionElement> &session, std::chrono::steady_clock::time_point wait_until)

Requests new data processing for a session at a specific time.

Requests that the inference system process data for the specified session, but waits for the data until the given time point before processing.

Parameters:

• session – Shared pointer to the session requesting data processing

• wait_until – Time point at which to begin processing the data request

void reset_session(const std::shared_ptr<SessionElement> &session)

Resets a session to its initial state.

Clears all internal buffers, resets the inference pipeline, and prepares the session for a new processing session. This method is typically used to reinitialize a session without releasing it completely.

Parameters:

session – Shared pointer to the session to reset

Public Static Functions

static std::shared_ptr<Context> get_instance(const ContextConfig &context_config)

Gets or creates the singleton context instance.

Returns the existing context instance or creates a new one with the specified configuration if none exists. This is the primary method for accessing the global inference cntext.

Note

If a context already exists, the provided configuration is ignored. The configuration is only used when creating a new instance.

Parameters:

context_config – Configuration settings for the context (used only on first creation)

Returns:

Shared pointer to the singleton context instance

static std::shared_ptr<SessionElement> create_session(PrePostProcessor &pp_processor, InferenceConfig &inference_config, BackendBase *custom_processor)

Creates a new inference session with specified components.

Creates and registers a new inference session with the provided preprocessing/ postprocessing pipeline, inference configuration, and optional custom backend. The session is automatically assigned a unique ID and integrated into the global resource management system.

Parameters:

• pp_processor – Reference to the preprocessing/postprocessing pipeline

• inference_config – Reference to the inference configuration

• custom_processor – Pointer to custom backend processor (nullptr for default backends)

Returns:

Shared pointer to the newly created session

static void release_session(const std::shared_ptr<SessionElement> &session)

Releases an inference session and its resources.

Properly shuts down and releases the specified session, including cleanup of associated backend processors, buffers, and other resources.

Parameters:

session – Shared pointer to the session to release

static void release_instance()

Releases the singleton context instance.

Shuts down and releases the global context instance, including all sessions, thread pools, and backend processors. This should be called during application shutdown to ensure proper cleanup.

static void release_thread_pool()

Releases the inference thread pool.

Shuts down all inference threads and releases thread pool resources. This is typically called as part of context cleanup or reconfiguration.

static int get_num_sessions()

Gets the number of active inference sessions.

Returns the current count of active inference sessions managed by the context. This is useful for monitoring and debugging purposes.

Returns:

Number of currently active sessions

static std::vector<std::shared_ptr<SessionElement>> &get_sessions()

Gets a reference to all active sessions.

Returns a reference to the vector containing all currently active inference sessions. This method is primarily used for internal management and debugging.

Note

This method provides direct access to internal data structures and should be used carefully to avoid disrupting session management.

Returns:

Reference to the vector of active session shared pointers

static moodycamel::ProducerToken &get_producer_token()

Get producer token for the next inference request Returns a producer token that can be used to enqueue inference requests into the concurrent queue.

Note

The producer token is used to ensure thread-safe and non-blocking access to the concurrent queue for submitting inference requests.

Returns:

Shared pointer to the producer token

static moodycamel::ConcurrentQueue<InferenceData> &get_static_inference_queue()

Get a reference to the static inference queue Returns a reference to the global concurrent queue used for inference requests. This is used by InferenceThread to pre-allocate a ConsumerToken on the main thread, enabling allocation-free dequeue from worker threads.

Returns:

Reference to the static inference queue

static std::unique_ptr<InferenceThread> make_inference_thread()

Factory for a user-owned InferenceThread bound to the static inference queue.

Returns a new InferenceThread whose lifecycle is fully managed by the caller. The thread is not started automatically — call start() on the returned object to begin processing. The caller must also call stop() (or simply destroy the object) before program exit.

This is purely additive: the auto-managed thread pool sized via ContextConfig::m_num_threads continues to work unchanged. Users who want full control over threading typically construct Context with ContextConfig(0) so that no auto-pool threads exist, then create and manage threads themselves via this factory.

The returned thread references the static inference queue, which has static storage duration — so the thread remains valid even after all sessions and the Context singleton itself are released.

Returns:

Unique pointer to a new user-owned InferenceThread.