Class anira::v2::InferenceConfig#
-
class InferenceConfig#
Collaboration diagram for anira::v2::InferenceConfig:
![digraph {
graph [bgcolor="#00000000"]
node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
edge [color="#1414CE"]
"1" [label="anira::v2::InferenceConfig" tooltip="anira::v2::InferenceConfig" fillcolor="#BFBFBF"]
"2" [label="anira::v2::ProcessingSpec" tooltip="anira::v2::ProcessingSpec"]
"1" -> "2" [dir=forward tooltip="usage"]
}](../../_images/graphviz-3283fd810335753d8fc7035fb6f2c41cffdfa701.png)
The 2.x InferenceConfig as a value over an anira::ModelConfig: the constructors write their arguments as a version-2 document and load it through ModelConfig::from_json, the one upgrade path (a bytes entry borrowed, its path the placeholder “anira:bytes”), and take the legacy Hard contract; the 2.x public fields and getters are read back from the handle.
What the constructors normalise as 2.x did: max_inference_time must be positive (ANIRA_ERROR_CONFIG, first); a session-exclusive configuration runs one processor; a processor count below 1 is 1 (with a Warning); a non-empty ProcessingSpec vector with the wrong entry count is dropped and defaulted (2.x did so silently, the shim with a Warning); a non-streamable tensor with more than one channel is refused (ANIRA_ERROR_CONFIG); a model_function on a backend other than LIBTORCH or EXECUTORCH is dropped (with the 2.x message as a Warning); a backend row of a backend without a model entry is dropped. What anira 3 refuses where 2.x ran: a backend row that is no axis permutation of the universal one (ANIRA_ERROR_JSON), a streamed tensor with channels > 1 and no axis of that extent (ANIRA_ERROR_JSON). A backend this build lacks stays an entry, which a handler skips.
Copyable (the copy loads its document again, the bytes borrowed again) and movable; operator== compares the configurations anira 3 holds (the model file, the three legacy scalars, the borrowed bytes by pointer and size), not the spelling.
Public Functions
-
InferenceConfig() = default#
An empty configuration: every getter answers empty, model_config() throws ANIRA_ERROR_INVALID_STATE.
-
inline InferenceConfig(std::vector<ModelData> model_data, std::vector<TensorShape> tensor_shape, ProcessingSpec processing_spec, float max_inference_time, unsigned warm_up = Defaults::k_warm_up, bool session_exclusive_processor = Defaults::k_session_exclusive_processor, float blocking_ratio = Defaults::k_blocking_ratio, unsigned num_parallel_processors = ANIRA_INSTANCES_AUTO)#
- Throws:
Error – with the status of the normalisation or of ModelConfig::from_json.
-
inline InferenceConfig(std::vector<ModelData> model_data, std::vector<TensorShape> tensor_shape, float max_inference_time, unsigned warm_up = Defaults::k_warm_up, bool session_exclusive_processor = Defaults::k_session_exclusive_processor, float blocking_ratio = Defaults::k_blocking_ratio, unsigned num_parallel_processors = ANIRA_INSTANCES_AUTO)#
-
inline InferenceConfig(const InferenceConfig &other)#
Loads the document again: an equal configuration over its own handles.
-
inline InferenceConfig &operator=(const InferenceConfig &other)#
-
InferenceConfig(InferenceConfig&&) noexcept = default#
-
InferenceConfig &operator=(InferenceConfig&&) noexcept = default#
-
~InferenceConfig() = default#
-
inline std::string get_model_path(InferenceBackend backend) const#
The first entry of the backend: its path, or its bytes as a string (2.x); “” without one.
-
inline const ModelData *get_model_data(InferenceBackend backend) const#
The first entry of the backend (a pointer into m_model_data); nullptr without one.
-
inline std::string get_model_function(InferenceBackend backend) const#
-
inline bool is_model_binary(InferenceBackend backend) const#
-
inline const TensorShapeList &get_tensor_input_shape() const noexcept#
The universal shapes: anira 3’s one shape per tensor.
-
inline const TensorShapeList &get_tensor_output_shape() const noexcept#
-
inline const TensorShapeList &get_tensor_input_shape(InferenceBackend backend) const noexcept#
The shapes the backend’s first entry holds: the universal ones with that entry’s axis layout applied (a channels-last TFLite row reads channels last), the universal ones for a backend without an entry or a layout.
-
inline const TensorShapeList &get_tensor_output_shape(InferenceBackend backend) const noexcept#
-
inline const std::vector<size_t> &get_tensor_input_size() const noexcept#
-
inline const std::vector<size_t> &get_tensor_output_size() const noexcept#
-
inline const std::vector<size_t> &get_preprocess_input_channels() const noexcept#
-
inline const std::vector<size_t> &get_postprocess_output_channels() const noexcept#
-
inline const std::vector<size_t> &get_preprocess_input_size() const noexcept#
-
inline const std::vector<size_t> &get_postprocess_output_size() const noexcept#
-
inline const std::vector<size_t> &get_internal_model_latency() const noexcept#
-
inline bool operator==(const InferenceConfig &other) const#
The configurations anira 3 holds: the model files (ModelConfig::to_json), the three legacy scalars, the borrowed bytes by pointer and size.
-
inline bool operator!=(const InferenceConfig &other) const#
-
inline const anira::ModelConfig &model_config() const#
The model configuration, bytes entries borrowed (the caller’s buffer outlives this object and every handler built from it, as in 2.x).
- Throws:
Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.
-
inline anira::ModelConfig model_config_copy() const#
A fresh ModelConfig of the same document, bytes entries borrowed again: a handler’s private copy, whose anchor it may set without touching this configuration.
- Throws:
Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.
-
inline anira::Hard hard() const#
The legacy Hard contract read back: the explicit budget (max_inference_time), the fixed warm-up, the wait ratio (blocking_ratio), ANIRA_MISS_ZEROS; no host blocks, which the handler’s prepare sets.
- Throws:
Error{ANIRA_ERROR_INVALID_STATE} – on an empty configuration.
Public Members
-
std::vector<TensorShape> m_tensor_shape#
the universal row, the rows of the entries with a layout, a universal clone per other backend of an entry
-
ProcessingSpec m_processing_spec#
-
float m_max_inference_time = 0#
ms, the legacy contract’s explicit budget
-
unsigned m_warm_up = 0#
the legacy contract’s fixed warm-up count
-
bool m_session_exclusive_processor = false#
the model’s state is STATEFUL
-
float m_blocking_ratio = 0#
the legacy contract’s wait ratio
-
unsigned m_num_parallel_processors = 0#
the model’s max_instances (2.x’s default for ANIRA_INSTANCES_AUTO); 1 when exclusive
-
struct Defaults#
The defaults of the constructors’ trailing arguments, as in 2.x.
Public Static Functions
-
static inline unsigned num_parallel_processors() noexcept#
Half the hardware threads, at least 1: the count 2.x wrote, and what the shim reads back for a model that leaves the count to the pool (the constructors’ default, ANIRA_INSTANCES_AUTO; the library’s default pool has as many threads). A function where 2.x had a static member: a header has no out-of-line definition, and an inline variable would be bound STB_GNU_UNIQUE by GCC, which pins a plugin in memory.
-
static inline unsigned num_parallel_processors() noexcept#
-
InferenceConfig() = default#