Class anira::InferenceHandler#

class InferenceHandler#

An anira_handler with its lifetime: the runtime of one pipeline on a context. Move-only; the destructor is anira_handler_destroy, which drains the handler’s in-flight work.

Created from a Pipeline, whose processors and engines the handler copies (anira_handler_create): the Pipeline may die afterwards, and so may the Engine and Processor objects the program holds (the handler keeps what it runs alive). prepare takes the contract and returns the PlanReport, a view of the handler’s report, valid until the next prepare or the handler’s end.

The Hard entries are one-line forwarders onto the tensor entries of anira/abi/handler.h: noexcept, ANIRA_NONBLOCKING, a status returned and never thrown (the reason of a failure is rt_error()), nothing copied but the samples. They take anira::Tensor descriptors only, as the C surface does; a float host builds its planar tensors once per prepare with Tensor::from_host_planar and sets the block length (shape[1]) per call. The forms over one slot take the slot’s tensor; the forms over every slot (the _multi entries) take one tensor per slot of the model’s list: a Streamed slot’s block, a Static slot’s whole tensor, or an empty element (a rank of 1 or more with an extent of 0; a value-initialised Tensor{} has rank 0 and is not empty) that leaves its slot out. delivered receives the samples written per output (one count for the single forms, one per output slot for the multi forms), or is NULL. The _wait twins wait for the inference the call needs, up to timeout_ms or with ANIRA_WAIT_CONTRACT / ANIRA_WAIT_FOREVER; they are not nonblocking.

Every member is [main-thread] except: the Hard entries, the Static pair, reset and get_available_samples, which are the driver thread’s ([driver-thread], nonblocking); rt_error, set_plan, get_plan, get_latency and get_latencies, which are [thread-safe] (set_plan and get_plan nonblocking); the _wait twins ([any-thread, blocking]).

Public Functions

inline InferenceHandler(Context &context, const Pipeline &pipeline)#
Throws:

Error – with the status of anira_handler_create (an engine the pipeline does not carry, a structural rule of the model config).

inline ~InferenceHandler()#
InferenceHandler(const InferenceHandler&) = delete#
InferenceHandler &operator=(const InferenceHandler&) = delete#
inline InferenceHandler(InferenceHandler &&other) noexcept#
inline InferenceHandler &operator=(InferenceHandler &&other) noexcept#
inline PlanReport prepare(const ContractHandle &contract)#

anira_handler_prepare: the blocking quiescence point, never from the driver thread. An Async contract is ANIRA_ERROR_NOT_SUPPORTED until the Async pump lands.

Throws:

Error – with the status and the reason; the handler is unprepared after a refusal.

inline PlanReport prepare(const Hard &hard)#
inline PlanReport prepare(const Async &async)#
inline PlanReport prepare(const Contract &contract)#
inline PlanReport plan_report() const noexcept#

The report of the last successful prepare (a report without plans before it).

inline uint32_t num_entries() const noexcept#

anira_handler_num_entries: the chunks that can be in flight at once.

inline anira_status process(const Tensor &in, uint32_t in_slot, const Tensor &out, uint32_t out_slot, size_t *delivered = nullptr) noexcept#
inline anira_status process(std::span<const Tensor> inputs, std::span<const Tensor> outputs, size_t *delivered = nullptr) noexcept#
inline anira_status push_data(const Tensor &in, uint32_t slot) noexcept#
inline anira_status push_data(std::span<const Tensor> inputs) noexcept#
inline anira_status pop_data(const Tensor &out, uint32_t slot, size_t *delivered = nullptr) noexcept#
inline anira_status pop_data(std::span<const Tensor> outputs, size_t *delivered = nullptr) noexcept#
inline anira_status process_wait(const Tensor &in, uint32_t in_slot, const Tensor &out, uint32_t out_slot, double timeout_ms, size_t *delivered = nullptr) noexcept#
inline anira_status process_wait(std::span<const Tensor> inputs, std::span<const Tensor> outputs, double timeout_ms, size_t *delivered = nullptr) noexcept#
inline anira_status pop_data_wait(const Tensor &out, uint32_t slot, double timeout_ms, size_t *delivered = nullptr) noexcept#
inline anira_status pop_data_wait(std::span<const Tensor> outputs, double timeout_ms, size_t *delivered = nullptr) noexcept#
inline anira_status set_static_input(uint32_t slot, const Tensor &value) noexcept#

anira_handler_set_static_input: a whole Static tensor in the spec’s dtype and shape.

inline anira_status get_static_output(uint32_t slot, const Tensor &out) noexcept#

anira_handler_get_static_output: the latest collected value of a Static output.

inline uint32_t get_latency(uint32_t slot = 0) const noexcept#

anira_handler_get_latency: one output’s latency in samples of that output.

inline std::vector<uint32_t> get_latencies() const#

anira_handler_get_latencies: one per output slot, 0 for a slot that is not Streamed; empty before a successful prepare.

inline anira_status get_available_samples(uint32_t slot, uint32_t channel, size_t *out) noexcept#

anira_handler_get_available_samples: the samples waiting in one channel of an output ring (it collects finished inferences, hence non-const).

inline void reset() noexcept#

anira_handler_reset: restarts the stream (the rings, the declared state, rt_error()).

inline anira_status rt_error() const noexcept#

anira_handler_rt_error: the last real-time failure, ANIRA_OK when there was none.

inline anira_status set_plan(uint32_t plan) noexcept#

anira_handler_set_plan: the dense plan index of the report; effective at the next chunk.

inline uint32_t get_plan() const noexcept#
inline const anira_handler *native() const noexcept#
inline anira_handler *native() noexcept#