Class anira::v2::InferenceHandler#
-
class InferenceHandler#
Collaboration diagram for anira::v2::InferenceHandler:
![digraph {
graph [bgcolor="#00000000"]
node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
edge [color="#1414CE"]
"2" [label="anira::Context" tooltip="anira::Context"]
"6" [label="anira::ModelConfig" tooltip="anira::ModelConfig"]
"3" [label="anira::v2::InferenceConfig" tooltip="anira::v2::InferenceConfig"]
"1" [label="anira::v2::InferenceHandler" tooltip="anira::v2::InferenceHandler" fillcolor="#BFBFBF"]
"5" [label="anira::v2::PrePostProcessor" tooltip="anira::v2::PrePostProcessor"]
"4" [label="anira::v2::ProcessingSpec" tooltip="anira::v2::ProcessingSpec"]
"3" -> "4" [dir=forward tooltip="usage"]
"1" -> "2" [dir=forward tooltip="usage"]
"1" -> "3" [dir=forward tooltip="usage"]
"1" -> "5" [dir=forward tooltip="usage"]
"1" -> "6" [dir=forward tooltip="usage"]
"5" -> "3" [dir=forward tooltip="usage"]
}](../../_images/graphviz-e1cb639aef2e3523a7341695a2e1ae786d2ee78a.png)
The 2.x InferenceHandler over the C handler of anira/abi/handler.h: a context (from the ContextConfig), a pipeline built from a private copy of the configuration’s model config, the custom engine and a LegacyProcessor over the processor, and one anira_handler created from them. The processor and the configuration are the caller’s and outlive the handler, as in 2.x.
What differs from 2.x: nothing loads at construction (a model that does not load fails prepare); a prepare with the settings of the last one only resets the stream, one with other host blocks loads the models again, one with another reference tensor rebuilds the C handler; the single forms carry their one slot per side and nothing else (2.x resent what the last call left in the other slots); a block above the HostConfig buffer size (ceil of it, scaled to each stream) is refused, nothing moves and rt_error() reports ANIRA_ERROR_INVALID_ARGUMENT (2.x kept its tail); push_data never waits, pop_data with a deadline polls on a handler without a blocking ratio, and set_non_realtime is never refused (a wait without an inference loop fails the call instead, which rt_error() reports); set_inference_backend of a backend without a plan keeps the selection and logs; get_num_inference_threads is the pool size, 0 until the first prepare in the process; every failure is an anira::Error.
Real time: reset, get_latency, get_available_samples, set_inference_backend, get_inference_backend and rt_error never wait and carry ANIRA_NONBLOCKING; the process, push and pop forms are noexcept but carry no attribute, since a blocking ratio or set_non_realtime makes them wait for the inference (the 2.x declarations claimed the attribute and waited anyway). Without either, on the thread that drives the handler, they allocate nothing and wait for nothing. No member runs concurrently with prepare() or the destructor: set_inference_backend and get_inference_backend may be called from any other thread while audio runs, but not while prepare runs (the C ABI’s rule for anira_handler_set_plan).
Public Functions
-
InferenceHandler() = delete#
-
InferenceHandler(const InferenceHandler&) = delete#
-
InferenceHandler &operator=(const InferenceHandler&) = delete#
-
InferenceHandler(InferenceHandler&&) = delete#
-
InferenceHandler &operator=(InferenceHandler&&) = delete#
-
inline InferenceHandler(PrePostProcessor &pp_processor, InferenceConfig &inference_config, const ContextConfig &context_config = ContextConfig())#
A CUSTOM row runs a PassthroughEngine (the 2.x base backend). Loads no model.
- Throws:
Error – with the status of the context, the pipeline or anira_handler_create (ANIRA_ERROR_CONFIG for a configuration whose every entry names an engine this build lacks).
-
inline InferenceHandler(PrePostProcessor &pp_processor, InferenceConfig &inference_config, anira::Engine &custom_engine, const ContextConfig &context_config = ContextConfig())#
The CUSTOM row runs the caller’s engine, which carries k_custom_engine_id (the id the CUSTOM row names; a 2.x custom backend is an anira::Engine constructed with it) and outlives the handler, as 2.x required of a BackendBase. Its providers() decide where the plan runs: the CPU path without a list, else its first listed provider.
- Throws:
Error{ANIRA_ERROR_INVALID_ARGUMENT} – for an engine of another id, Error{ANIRA_ERROR_CONFIG} for a configuration without a CUSTOM row, and as the other constructor.
-
inline ~InferenceHandler()#
Destroys the C handler (in-flight inferences drain first), then the processor and the context. A plugin destroys its handlers before its host unloads it, never from its own static destructors (they run inside the unload, under the loader lock).
-
inline void set_inference_backend(InferenceBackend inference_backend) noexcept#
The backend the next block runs on; before prepare the request is kept and applied at prepare. A backend without a plan keeps the selection and logs once through the real-time queue (group anira.compat); rt_error() is untouched. Any thread, but not while prepare()
or the destructor runs on another thread: prepare rebuilds the plan table and may replace the C handler this reads, the rule anira_handler_set_plan states (“not while
anira_handler_prepare runs”). 2.x promised this member while audio runs, never alongside prepare; a host that switches from a UI thread and re-prepares from another serialises the two itself.
-
inline InferenceBackend get_inference_backend() const noexcept#
The backend of the selected plan; before prepare the request, else the 2.x initial rule (CUSTOM with a caller’s engine, else the first entry this build runs, else CUSTOM). The same thread rule as set_inference_backend: not while prepare() or the destructor runs.
-
inline void prepare(HostConfig new_audio_config)#
Prepares for the host’s stream: the configuration’s legacy contract with the host blocks (host_block_max m_buffer_size, a fractional one included; host_block_min 1 when smaller blocks come, half the block when it is one sample or less), the reference tensor as the model’s anchor. The settings of the last successful prepare only reset the stream (the models stay loaded); other host blocks prepare again and load the models again; another reference tensor rebuilds the C handler. The selected backend is applied again.
- Throws:
Error – ANIRA_ERROR_CONFIG for a block size or rate that is not finite and above 0, ANIRA_ERROR_INVALID_ARGUMENT for a reference tensor out of range or not streamable, else the status of anira_handler_prepare (a model that does not load, a rule of the configuration); the handler is then unprepared.
-
inline void prepare(HostConfig new_audio_config, unsigned custom_latency, size_t tensor_index = 0)#
prepare() with the stream latency of one output declared (the 2.x custom latency): what get_latency reports for it and what its receive ring is primed with, in place of the computed figure; raised to the model’s internal latency with a Warning when below it.
- Throws:
Error{ANIRA_ERROR_INVALID_ARGUMENT} – for an index out of range or an output that is not streamable, and as prepare().
-
inline void prepare(HostConfig new_audio_config, std::vector<unsigned> custom_latency)#
prepare() with a stream latency per output; a non-streamable output carries 0.
- Throws:
Error{ANIRA_ERROR_INVALID_ARGUMENT} – for a vector of another size or a figure on a non-streamable output, and as prepare().
-
inline size_t process(float *const *data, size_t num_samples, size_t tensor_index = 0) noexcept#
In place: the block of slot tensor_index is pushed and replaced by the output.
-
inline size_t process(const float *const *input_data, size_t num_input_samples, float *const *output_data, size_t num_output_samples, size_t tensor_index = 0) noexcept#
One block of input slot tensor_index in, one of output slot tensor_index out; no other slot is carried. A non-streamable input slot takes min(num_input_samples, its size) values into the processor; a non-streamable output slot is read back likewise.
-
inline size_t *process(const float *const *const *input_data, size_t *num_input_samples, float *const *const *output_data, size_t *num_output_samples) noexcept#
Every slot at once: data[slot][channel][sample] with one count per slot (0 leaves the slot out). The delivered counts are written into num_output_samples, which is returned; num_input_samples is not written.
-
inline void push_data(const float *const *input_data, size_t num_input_samples, size_t tensor_index = 0) noexcept#
One block of input slot tensor_index; no other slot is carried. Never waits.
-
inline void push_data(const float *const *const *input_data, size_t *num_input_samples) noexcept#
Every input slot at once. Never waits (2.x waited under set_non_realtime).
-
inline size_t pop_data(float *const *output_data, size_t num_output_samples, size_t tensor_index = 0) noexcept#
One block of output slot tensor_index; no other slot is carried.
-
inline size_t pop_data(float *const *output_data, size_t num_output_samples, std::chrono::steady_clock::time_point wait_until, size_t tensor_index = 0) noexcept#
pop_data() that waits for the block’s inference until wait_until (polling on a handler without a blocking ratio, where 2.x neither waited nor collected); under set_non_realtime without limit, as in 2.x.
-
inline size_t *pop_data(float *const *const *output_data, size_t *num_output_samples) noexcept#
Every output slot at once; the counts are written into num_output_samples.
-
inline size_t *pop_data(float *const *const *output_data, size_t *num_output_samples, std::chrono::steady_clock::time_point wait_until) noexcept#
The deadline form of the multi pop.
-
inline unsigned get_latency(size_t tensor_index = 0) const noexcept#
The latency of output slot tensor_index in samples: the declared figure of a custom latency, else the computed one; 0 out of range and for a non-streamable output.
-
inline std::vector<unsigned> get_latency_vector() const#
Every output’s latency, index-aligned. Allocates: main thread.
-
inline size_t get_available_samples(size_t tensor_index, size_t channel = 0) const noexcept#
Collects the completed inferences and reports the samples waiting in one channel of the output ring of tensor_index; 0 out of range.
-
inline void set_non_realtime(bool is_non_realtime) noexcept#
Whether the process and pop forms wait without limit for their inference (an offline render). Never refused: without an inference loop each such call fails instead (ANIRA_ERROR_INVALID_STATE in rt_error(), the counts of the nonblocking stem returned).
-
inline size_t drain_log() noexcept#
anira_drain_log: delivers the queued real-time records to the sinks.
-
inline void reset() noexcept#
Restarts the stream: the rings, the declared state and rt_error() (anira_handler_reset).
-
inline anira_handler *native() const noexcept#
The C handler, for a host that mixes in a 3.x entry; valid until the next prepare that changes the reference tensor, or the destruction.
-
inline anira_status rt_error() const noexcept#
The last real-time failure (anira_handler_rt_error).
Public Static Functions
-
static inline unsigned get_num_inference_threads() noexcept#
The size of the core’s inference thread pool: 0 until the first handler is prepared.
-
InferenceHandler() = delete#