Class anira::v2::InferenceConfig#

class InferenceConfig#

Collaboration diagram for anira::v2::InferenceConfig:

digraph {
    graph [bgcolor="#00000000"]
    node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
    edge [color="#1414CE"]
    "1" [label="anira::v2::InferenceConfig" tooltip="anira::v2::InferenceConfig" fillcolor="#BFBFBF"]
    "2" [label="anira::v2::ProcessingSpec" tooltip="anira::v2::ProcessingSpec"]
    "1" -> "2" [dir=forward tooltip="usage"]
}

The 2.x InferenceConfig as a value over an anira::ModelConfig: the constructors write their arguments as a version-2 document and load it through ModelConfig::from_json, the one upgrade path (a bytes entry borrowed, its path the placeholder “anira:bytes”), and take the legacy Hard contract; the 2.x public fields and getters are read back from the handle.

What the constructors normalise as 2.x did: max_inference_time must be positive (ANIRA_ERROR_CONFIG, first); a session-exclusive configuration runs one processor; a processor count below 1 is 1 (with a Warning); a non-empty ProcessingSpec vector with the wrong entry count is dropped and defaulted (2.x did so silently, the shim with a Warning); a non-streamable tensor with more than one channel is refused (ANIRA_ERROR_CONFIG); a model_function on a backend other than LIBTORCH or EXECUTORCH is dropped (with the 2.x message as a Warning); a backend row of a backend without a model entry is dropped. What anira 3 refuses where 2.x ran: a backend row that is no axis permutation of the universal one (ANIRA_ERROR_JSON), a streamed tensor with channels > 1 and no axis of that extent (ANIRA_ERROR_JSON). A backend this build lacks stays an entry, which a handler skips.

Copyable (the copy loads its document again, the bytes borrowed again) and movable; operator== compares the configurations anira 3 holds (the model file, the three legacy scalars, the borrowed bytes by pointer and size), not the spelling.

Public Functions

InferenceConfig() = default#

An empty configuration: every getter answers empty, model_config() throws ANIRA_ERROR_INVALID_STATE.

inline InferenceConfig(std::vector<ModelData> model_data, std::vector<TensorShape> tensor_shape, ProcessingSpec processing_spec, float max_inference_time, unsigned warm_up = Defaults::k_warm_up, bool session_exclusive_processor = Defaults::k_session_exclusive_processor, float blocking_ratio = Defaults::k_blocking_ratio, unsigned num_parallel_processors = ANIRA_INSTANCES_AUTO)#
Throws:

Error – with the status of the normalisation or of ModelConfig::from_json.

inline InferenceConfig(std::vector<ModelData> model_data, std::vector<TensorShape> tensor_shape, float max_inference_time, unsigned warm_up = Defaults::k_warm_up, bool session_exclusive_processor = Defaults::k_session_exclusive_processor, float blocking_ratio = Defaults::k_blocking_ratio, unsigned num_parallel_processors = ANIRA_INSTANCES_AUTO)#
inline InferenceConfig(const InferenceConfig &other)#

Loads the document again: an equal configuration over its own handles.

inline InferenceConfig &operator=(const InferenceConfig &other)#
InferenceConfig(InferenceConfig&&) noexcept = default#
InferenceConfig &operator=(InferenceConfig&&) noexcept = default#
~InferenceConfig() = default#
inline std::string get_model_path(InferenceBackend backend) const#

The first entry of the backend: its path, or its bytes as a string (2.x); “” without one.

inline const ModelData *get_model_data(InferenceBackend backend) const#

The first entry of the backend (a pointer into m_model_data); nullptr without one.

inline std::string get_model_function(InferenceBackend backend) const#
inline bool is_model_binary(InferenceBackend backend) const#
inline const TensorShapeList &get_tensor_input_shape() const noexcept#

The universal shapes: anira 3’s one shape per tensor.

inline const TensorShapeList &get_tensor_output_shape() const noexcept#
inline const TensorShapeList &get_tensor_input_shape(InferenceBackend backend) const noexcept#

The shapes the backend’s first entry holds: the universal ones with that entry’s axis layout applied (a channels-last TFLite row reads channels last), the universal ones for a backend without an entry or a layout.

inline const TensorShapeList &get_tensor_output_shape(InferenceBackend backend) const noexcept#
inline const std::vector<size_t> &get_tensor_input_size() const noexcept#
inline const std::vector<size_t> &get_tensor_output_size() const noexcept#
inline const std::vector<size_t> &get_preprocess_input_channels() const noexcept#
inline const std::vector<size_t> &get_postprocess_output_channels() const noexcept#
inline const std::vector<size_t> &get_preprocess_input_size() const noexcept#
inline const std::vector<size_t> &get_postprocess_output_size() const noexcept#
inline const std::vector<size_t> &get_internal_model_latency() const noexcept#
inline bool operator==(const InferenceConfig &other) const#

The configurations anira 3 holds: the model files (ModelConfig::to_json), the three legacy scalars, the borrowed bytes by pointer and size.

inline bool operator!=(const InferenceConfig &other) const#
inline const anira::ModelConfig &model_config() const#

The model configuration, bytes entries borrowed (the caller’s buffer outlives this object and every handler built from it, as in 2.x).

Throws:

Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.

inline anira::ModelConfig model_config_copy() const#

A fresh ModelConfig of the same document, bytes entries borrowed again: a handler’s private copy, whose anchor it may set without touching this configuration.

Throws:

Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.

inline anira::Hard hard() const#

The legacy Hard contract read back: the explicit budget (max_inference_time), the fixed warm-up, the wait ratio (blocking_ratio), ANIRA_MISS_ZEROS; no host blocks, which the handler’s prepare sets.

Throws:

Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.

Public Members

std::vector<ModelData> m_model_data#

one per model entry, in entry order

std::vector<TensorShape> m_tensor_shape#

the universal row, the rows of the entries with a layout, a universal clone per other backend of an entry

ProcessingSpec m_processing_spec#
float m_max_inference_time = 0#

ms, the legacy contract’s explicit budget

unsigned m_warm_up = 0#

the legacy contract’s fixed warm-up count

bool m_session_exclusive_processor = false#

the model’s state is STATEFUL

float m_blocking_ratio = 0#

the legacy contract’s wait ratio

unsigned m_num_parallel_processors = 0#

the model’s max_instances (2.x’s default for ANIRA_INSTANCES_AUTO); 1 when exclusive

struct Defaults#

The defaults of the constructors’ trailing arguments, as in 2.x.

Public Static Functions

static inline unsigned num_parallel_processors() noexcept#

Half the hardware threads, at least 1: the count 2.x wrote, and what the shim reads back for a model that leaves the count to the pool (the constructors’ default, ANIRA_INSTANCES_AUTO; the library’s default pool has as many threads). A function where 2.x had a static member: a header has no out-of-line definition, and an inline variable would be bound STB_GNU_UNIQUE by GCC, which pins a plugin in memory.

Public Static Attributes

static constexpr unsigned k_warm_up = 0#
static constexpr bool k_session_exclusive_processor = false#
static constexpr float k_blocking_ratio = 0.F#