Class anira::Processor#

class Processor#

Inheritance diagram for anira::Processor:

digraph {
    graph [bgcolor="#00000000"]
    node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
    edge [color="#1414CE"]
    "1" [label="anira::Processor" tooltip="anira::Processor" fillcolor="#BFBFBF"]
    "2" [label="anira::v2::LegacyProcessor" tooltip="anira::v2::LegacyProcessor"]
    "2" -> "1" [dir=forward tooltip="public-inheritance"]
}

The pipeline’s processor (its custom pre- and post-processing) as a class: the registration. Subclass it, hand it to a Pipeline with Pipeline::add_processor(std::make_shared<YourProcessor>(…)). It is anira_processor_desc with virtual functions in place of the function pointers, and the semantics are those of anira/abi/processor.h, whose lifecycle it follows: the registration is one object shared by the pipeline and every handler created from it (phases(), realtime_phases() and consumed_kinds() are read once, when the processor is added); init() is called once per registration, by the first anira_handler_prepare of a handler of the pipeline, with the facts of the core in effect (InitInfo); prepare() is called once per anira_handler_prepare of each handler and returns that handler’s Processor::Prepared, the C lifecycle’s prepared pointer as a class: the four phases and reset run on it, so what a processor keeps per handler (its per-entry scratch, sized by PrepareInfo::num_entries) is a member of the Prepared and never of the registration, which two handlers of one pipeline share. anira owns the Prepared from prepare on and deletes it at the C unprepare: at the next anira_handler_prepare of that handler, once the old session is released, and at anira_handler_destroy, after the last phase call; release() runs once, when the last carrier dies, after every Prepared was deleted, whether or not init ever ran.

A Pipeline holds at most one processor, and the processor owns every phase it fills for every slot. The phases of one chunk: pre_process takes the host end of every slot (the ring of a Streamed tensor) to the model tensor, the chunking and every conversion, on the thread where the host end is produced (the thread that drives the Hard entries); then, on an inference thread, anira feeds the State inputs, before_inference runs, anira crosses the edge into the backend’s domain, the engine runs, anira crosses back, after_inference runs, anira captures the State outputs; then post_process takes the model tensors back to the host end. A processor never crosses a domain: every phase works in the host domain of the slot. reset runs for the first chunk of a new stream (after prepare, after anira_handler_reset), before that chunk’s pre_process and on its thread, with the chunk’s context in ANIRA_PHASE_RESET, where the accessors answer the role only: what the Prepared keeps between chunks (a resampler, an overlap buffer, a filter’s history) starts over there.

A processor states the phases it fills in phases(), which a subclass must define. The C descriptor tells a filled phase from a NULL one: a NULL pre_process or post_process slot means anira’s default body runs (the window pop or push of every Streamed slot), a filled slot means this processor owns the phase and composes what it does not handle itself by calling the default through the base class (“call super”): Prepared::pre_process(ctx) is anira_processor_default_pre_process, Prepared::post_process(ctx) is anira_processor_default_post_process. A virtual function is never NULL, so the mask says which of the four phases and reset this processor takes part in: only those reach the descriptor, the others stay NULL and are never called (a Prepared that overrides reset names k_reset, else its reset is never called).

realtime_phases() is the processor’s real-time promise, per phase (anira_processor_desc::realtime_phases), in the bits of phases(): the bit of a phase says that phase allocates nothing, locks nothing and blocks on nothing; the default promises nothing. anira_handler_prepare checks the promise against the placement: every filled phase that the contract runs on the driving thread must carry its bit, else prepare fails with ANIRA_ERROR_CONFIG naming the phase. Under a Hard contract those are pre_process, post_process and reset, so a processor of inference hooks alone needs no promise; a promised phase that is not filled is no error. A processor whose bodies are all real-time says so in one line: realtime_phases() returns phases(). The phase functions and reset are noexcept (an override that throws does not compile without it, and terminates with it) and carry no real-time attribute of their own; everything ProcessorContext, RingView and Tensor offer is nonblocking, so a body that keeps its promise is composed of those calls. A status other than ANIRA_OK fails the chunk: the status goes into anira_handler_rt_error with one latched record naming the phase, and the chunk delivers zeros at its stream position.

Why the ownership spelling: the Prepared is polymorphic and lives on the heap; prepare returns a std::unique_ptr because anira is its one owner from then on (the trampoline releases it into the C out_prepared on success; a refused prepare destroys nothing that was not returned); the registration is a std::shared_ptr because it IS shared, by the pipeline and every handler created from it.

A processor has no name: a Pipeline holds one, so a name would identify nothing; every record about it says “the processor”, and the consumer column of its anira_plan_ext rows reads “processor”. Its optional id (the constructor’s) names its inference hooks for the model’s figure, which a prepare’s warm-up pass measures with them: processors of one id share it.

Subclassed by anira::v2::LegacyProcessor

Public Functions

Processor() = default#
inline explicit Processor(std::string id)#

A processor with an id (anira_processor_desc::id), a reverse-URI name: what its before_inference and after_inference are for the model’s figure. A prepare’s warm-up pass times those hooks with the engine call, so every handler whose processor gives the same id shares one figure of the model (identical plugins report identical latency); without an id, a processor that fills an inference hook measures its own. A processor without inference hooks needs none.

virtual ~Processor() = default#
Processor(const Processor&) = delete#
Processor &operator=(const Processor&) = delete#
Processor(Processor&&) = delete#
Processor &operator=(Processor&&) = delete#
inline const std::string &id() const noexcept#

The id the constructor took; empty for none.

virtual uint32_t phases() const noexcept = 0#

The phases this processor fills: an OR of k_pre_process, k_post_process, k_before_inference, k_after_inference and k_reset (a Prepared that overrides reset names k_reset, else its reset is never called). Read once, when the processor is added to a Pipeline; 0 is a processor of prepare, unprepare and release alone.

inline virtual uint32_t realtime_phases() const noexcept#

The processor’s real-time promise, per phase, written into anira_processor_desc::realtime_phases by Pipeline::add_processor: an OR of the k_* bits of phases() whose bodies allocate nothing, lock nothing and block on nothing; 0 (the default) promises nothing. Every filled phase the contract runs on the driving thread must carry its bit, else anira_handler_prepare refuses the processor with ANIRA_ERROR_CONFIG naming the phase: under a Hard contract k_pre_process, k_post_process and k_reset, so one of before_inference and after_inference alone needs no promise there. A promised phase that is not filled is no error; a processor whose bodies are all real-time returns phases(). Read once, when the processor is added to a Pipeline.

inline virtual std::span<const char*const> consumed_kinds() const noexcept#

The extensions the processor reads at prepare, as “<host>:<kind>” strings (hosts: tensor_spec, model, model_config, context, contract); they join the consumed-or-fail walk. Read once, when the processor is added; the strings are copied.

inline virtual void init(const InitInfo&)#

Called once per Pipeline::add_processor of this processor, by the first anira_handler_prepare of a handler of that pipeline, on its caller’s thread ([main-thread]; it may allocate), before that handler’s prepare, with the facts of the core in effect (the level, the thread count, the handler’s context, whose capabilities are legal to query). What the processor keeps for its whole registration (a lookup table, a thread of its own) is built here, as a member of the object. It may throw, as prepare may: the status fails that anira_handler_prepare, the registration counts as uninitialised, and the next prepare calls init again. The base class does nothing.

virtual std::unique_ptr<Prepared> prepare(const PrepareInfo &info) = 0#

Called by anira_handler_prepare on its caller’s thread after the plan report is built, once per prepare of each handler ([main-thread]; it may allocate): returns the Prepared that runs the phases for that handler, sized by the record (its num_entries for a per-chunk scratch, its templates for the tensors), which anira owns from then on. Of anira it may call the [callback-safe] entries, the handler’s getters and the two Static entries, never prepare, destroy or a Hard entry. It may throw: an anira::Error fails the prepare with its status, std::bad_alloc with ANIRA_ERROR_OUT_OF_MEMORY, anything else with ANIRA_ERROR_INTERNAL, and what() goes to the log (group “anira.hpp”), since the C callback has no error record; a null return fails it with ANIRA_ERROR_INTERNAL, since nothing could run the phases. A refused prepare leaves the handler unprepared and deletes nothing that was not returned.

inline virtual void release() noexcept#

Called once per Pipeline::add_processor of this processor, when the last pipeline or handler that carries it dies, on the thread of that destroy ([main-thread]), after every Prepared of it was deleted, whether or not init ever ran; no callback of that chain runs afterwards. The object itself lives as long as a shared_ptr to it does.

Public Static Functions

static inline constexpr uint32_t phase_bit(anira_phase phase) noexcept#

The bit of one phase in phases() and realtime_phases(), ANIRA_PHASE_BIT(phase).

Public Static Attributes

static constexpr uint32_t k_pre_process = uint32_t{1} << ANIRA_PHASE_PRE_PROCESS#
static constexpr uint32_t k_post_process = uint32_t{1} << ANIRA_PHASE_POST_PROCESS#
static constexpr uint32_t k_before_inference = uint32_t{1} << ANIRA_PHASE_BEFORE_INFERENCE#
static constexpr uint32_t k_after_inference = uint32_t{1} << ANIRA_PHASE_AFTER_INFERENCE#
static constexpr uint32_t k_reset = uint32_t{1} << ANIRA_PHASE_RESET#

reset: its Prepared overrides reset, which is called only with this bit in phases().

static constexpr uint32_t k_all_phases = k_pre_process | k_post_process | k_before_inference | k_after_inference | k_reset#

Every bit phases() and realtime_phases() may carry.

class Prepared#

Inheritance diagram for anira::Processor::Prepared:

digraph {
    graph [bgcolor="#00000000"]
    node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
    edge [color="#1414CE"]
    "1" [label="anira::Processor::Prepared" tooltip="anira::Processor::Prepared" fillcolor="#BFBFBF"]
    "2" [label="anira::v2::LegacyProcessor::Prepared" tooltip="anira::v2::LegacyProcessor::Prepared"]
    "2" -> "1" [dir=forward tooltip="public-inheritance"]
}

One handler’s prepared processor: what Processor::prepare returns, the C prepared pointer as a class. The four phases and reset run on it, for that handler’s chunks alone, and what the processor keeps per handler lives here. Created by prepare on the main thread (it may allocate there), deleted by anira at the C unprepare (the next anira_handler_prepare of the handler, once the old session is released, or anira_handler_destroy), after the last phase call; never copied, never moved.

Subclassed by anira::v2::LegacyProcessor::Prepared

Public Functions

Prepared() = default#
virtual ~Prepared() = default#
Prepared(const Prepared&) = delete#
Prepared &operator=(const Prepared&) = delete#
Prepared(Prepared&&) = delete#
Prepared &operator=(Prepared&&) = delete#
inline virtual anira_status pre_process(ProcessorContext &ctx) noexcept#

ANIRA_PHASE_PRE_PROCESS, on the thread where the host end is produced (the driver thread under a Hard contract): takes the host end of every slot to its model tensor. The base class is the default fill (anira_processor_default_pre_process).

inline virtual anira_status post_process(ProcessorContext &ctx) noexcept#

ANIRA_PHASE_POST_PROCESS, on the thread where the host end is consumed: takes the model tensor of every output slot back to its host end. The base class is the default push (anira_processor_default_post_process).

inline virtual anira_status before_inference(ProcessorContext&) noexcept#

ANIRA_PHASE_BEFORE_INFERENCE, on an inference thread behind the State feed and ahead of the edge into the backend’s domain. The base class does nothing.

inline virtual anira_status after_inference(ProcessorContext&) noexcept#

ANIRA_PHASE_AFTER_INFERENCE, on an inference thread behind the edge out of the backend’s domain and ahead of the State capture. The base class does nothing.

inline virtual void reset(ProcessorContext&) noexcept#

ANIRA_PHASE_RESET: the first chunk of a new stream (after prepare, after anira_handler_reset), before that chunk’s pre_process, on its thread, never mid-stream; the context answers the role only (a ring or a tensor asked here is refused and recorded). What this object keeps between chunks starts over. The base class does nothing.