Running 2.x code: anira/compat/v2.hpp#
The shim#
#include <anira/compat/v2.hpp> is the 2.x face of anira over the 3.x C ABI: the 2.x classes
under the names of the last 2.x release, v2.3.0, in namespace anira::v2, header-only C++20
over anira/anira.hpp and the entries of anira/abi/, with nothing of the 2.x tree. A 2.x
host compiles against it by including it in place of <anira/anira.h> and adding
using namespace anira::v2;, and then runs on the C handler of anira/abi/handler.h, the
3.x runtime, with the differences listed below. It is deprecated from the start and removed one
minor release after 3.0.0 (When the 2.x classes go); moving to the 3.x API is the
way forward, and the header lets that happen one class at a time
(anira::v2::InferenceConfig::model_config() hands out the 3.x handle,
anira::v2::LegacyProcessor runs a 2.x processor as the processor of a 3.x pipeline).
#include <anira/compat/v2.hpp>
using namespace anira::v2;
InferenceConfig config({ModelData("model.onnx", ONNX), ModelData("model.pt", LIBTORCH)},
{TensorShape({{1, 1, 2048}}, {{1, 1, 2048}})},
ProcessingSpec({1}, {1}, {2048}, {2048}),
42.66f);
PrePostProcessor pp(config);
InferenceHandler handler(pp, config, ContextConfig(2));
handler.prepare(HostConfig(2048, 48000));
handler.process(channels, 2048);
The shim, member by member.
2.x |
|
|---|---|
|
Present; a |
|
Present with an |
|
Present, wait-free, before |
|
Present; a fractional block size is a fractional Hard host block, as in 2.x; a reference tensor that is not streamable is refused. |
|
Present: the declared stream latency of the Hard contract
( |
The five |
Present with the 2.x signatures; no |
|
Present; |
|
Present over |
|
Present (The classes and what differs from 2.x). |
|
|
|
Not shimmed: an |
|
Present under the 2.x names; the values are |
Not source-compatible, a compile error where a stand-in would lie:
a custom backend as a
BackendBase(ananira::Engineunderk_custom_engine_id; Custom Engines);the owning
BufferF(channels, samples)andBufferF::swap_data/get_sample_rate(a host buffer is the host’s own container);RingBuffer::get_future_sample,get_past_sample,get_num_samples,swap_data,initialize_with_positionsandclear_with_positions(anira owns the rings);HostConfig::resolve_reference,get_reference_sizeand the relative size and rate getters;Contextbeyond those five statics (get_instance,make_inference_thread, the session statics; ananira::InferenceThreadon athreads(0)context is the successor ofmake_inference_thread);the
InferenceConfigsetters,get_tensor_shape(backend),clear_processing_spec/update_processing_specandDefaults::m_num_parallel_processors(the functionDefaults::num_parallel_processors());code that stored an
InferenceBackendas an integer or indexed an array by it: the shim’s values areanira_engine’s, and the 2.x values shifted with the engines of the build;a
catch (const std::invalid_argument&): every failure is ananira::Error, astd::runtime_error;a model file of 3.x through the
JsonConfigLoader(it reads version-2 documents; a 3.x file loads throughanira::ModelConfig::from_file).
Behaviour that differs is listed per class in The classes and what differs from 2.x; in short:
nothing loads before prepare, and a prepare with other settings loads again; the
single forms carry one slot per side; a block above the HostConfig buffer size is refused;
push_data never waits;
pop_data with a deadline polls on a handler without a blocking ratio;
set_non_realtime is never refused (a waiting call without an inference loop fails instead);
set_inference_backend of a backend without a plan logs and keeps the selection;
get_num_inference_threads is the pool size, 0 until the first prepare;
Context::shutdown() is refused while a handler lives; a processor that throws fails its chunk;
a Static input a custom processor leaves alone reads zeros; the JsonConfigLoader has no
leniency, resolves relative paths against the document’s directory and keeps rows of engines
this build lacks; the ContextConfig thread count defaults to k_threads_auto, and a
document without a log level reads Warning; the engine code of PassthroughEngine and every
2.x custom engine is the host’s, so a handler must be destroyed before the module that holds it
unloads, and never from that module’s static destructors (Teardown and unloading: a library
unloaded with an inference still running aborts with a message on Linux and macOS).
The classes and what differs from 2.x#
#include <anira/compat/v2.hpp> declares the 2.x configuration classes in
namespace anira::v2 under the names of the last 2.x release (v2.3.0):
InferenceBackend with LIBTORCH, ONNX, TFLITE, LITERT, EXECUTORCH and
CUSTOM, ModelData, TensorShape, ProcessingSpec, InferenceConfig,
JsonConfigLoader, ContextConfig with
LogConfig, LogLevel, LogDrain and WaitStrategy, and HostConfig. The header
is header-only C++20 over anira/anira.hpp and the C entries, includes nothing of the 2.x
tree, and compiles beside <anira/anira.h>. A 2.x configuration compiles unchanged with a
using namespace anira::v2;:
#include <anira/compat/v2.hpp>
using namespace anira::v2;
InferenceConfig config({ModelData("model.onnx", ONNX), ModelData("model.pt", LIBTORCH)},
{TensorShape({{1, 1, 2048}}, {{1, 1, 2048}})},
ProcessingSpec({1}, {1}, {2048}, {2048}),
42.66f);
const anira::ModelConfig& model = config.model_config(); // the 3.x handle
anira::Hard contract = config.hard(); // max_inference_time, warm_up, blocking_ratio
anira::v2::InferenceConfig is a value over the 3.x handles: its constructors write their
arguments as a version-2 document and load it through anira::ModelConfig::from_json, the one
upgrade path of JSON files (a bytes entry is borrowed, as in 2.x), take the legacy Hard
contract, and read the 2.x public fields and getters back from the handles.
get_tensor_input_shape(backend) answers the shape the backend’s first entry holds, the
canonical one with that entry’s axis layout applied (the channels-last TensorFlow rows read
channels last, as the 2.x HybridNN processor expects). model_config() hands out the handle,
model_config_copy() a fresh one of the same document, hard() the legacy contract
without host blocks. It is copyable (a copy loads its document again) and compares equal when
anira 3 holds the same configuration, whatever the spelling.
anira::v2::JsonConfigLoader reads the document once and builds both objects, the
ContextConfig read back from anira::ContextConfig::from_json; get_context_config()
and get_inference_config() hand each out once, as in 2.x.
What differs from 2.x:
Errors. Every failure is an
anira::Errorwith a status, astd::runtime_error(2.x threwstd::invalid_argumentor asserted): amax_inference_timethat is not positive and a non-streamable tensor with more than one channel areANIRA_ERROR_CONFIG, a nullModelDataor an emptyTensorShapeANIRA_ERROR_INVALID_ARGUMENT. The loader has no leniency: a file that does not open isANIRA_ERROR_NO_SUCH_FILE, malformed text, a wrong type or a word outside a vocabularyANIRA_ERROR_JSONwith the key path, a 3.x documentANIRA_ERROR_CONFIG, all from the constructor, where 2.x logged and returnednullptr;get_inference_config()on a document that carries only acontext_configthrowsANIRA_ERROR_CONFIG.The backend values are the
anira_enginevalues of the engines (ONNXisANIRA_ENGINE_ONNXRUNTIME,CUSTOMisANIRA_ENGINE_CUSTOM, and so on; every enumerator in every build): the 2.x names are what stays. The 2.x values shifted with the engines of the build, so code that stored a backend as an integer or indexed an array by it is not source-compatible.to_engine(backend)answers theanira::EngineRef(forCUSTOMwithanira::v2::k_custom_engine_id,"anira.v2.custom"),to_backend(engine)the backend (CUSTOMfor every custom engine),is_available(backend)whether this build has the engine.Shapes. A backend row must hold the universal shapes with their axes permuted or unit axes inserted, the per-entry layout of anira 3; a row that reshapes (the same element count, other extents) is
ANIRA_ERROR_JSON, and so is a streamed tensor with more than one channel and no axis of that extent. A backend row of a backend without a model entry is dropped.Normalisation. What the 2.x constructor normalised, the shim does: a session-exclusive configuration runs one processor, a processor count below 1 is 1; a
ProcessingSpecvector with the wrong entry count is defaulted, with a Warning of the groupanira.compat(2.x did it silently); amodel_functionon a backend other thanLIBTORCHorEXECUTORCHis dropped with the 2.x message as a Warning (2.x kept the string).The loader. A relative
model_pathresolves against the document’s directory (2.x left it to the working directory); an entry of a backend this build lacks is kept, and a handler skips it (2.x dropped the row). A field the document leaves out reads anira’s default:ContextConfig::k_threads_autothreads and theWarninglevel (2.x tookdefault_num_threads()anddefault_log_level()).ContextConfig defaults its thread count to
ContextConfig::k_threads_auto, the sentinel anira resolves (half the hardware threads, at least one), where 2.x stored the resolved count;to_context_config()mints theanira::ContextConfig.HostConfig keeps its fields and
operator==; the session’s arithmetic of 2.x (resolve_reference,get_reference_size, the relative size and rate getters) is not declared.Not declared: the
InferenceConfigsetters (set_tensor_input_shape,set_model_path, …),get_tensor_shape(backend),clear_processing_specandupdate_processing_spec, andDefaults::m_num_parallel_processors, which is the functionDefaults::num_parallel_processors().
The processor half of the runtime is in the header too. anira::v2::PrePostProcessor keeps
the four virtuals with their 2.x signatures and default bodies, the per-element atomics of the
non-streamable tensors (set_input, get_output and their twins, from any thread) and
the five helpers, over two views: anira::v2::RingBuffer is a view of the anira_ring of
one Streamed slot (the float face of the ring accessors of anira/abi/processor.h), and
anira::v2::BufferF a view of the model end of one slot, one channel of the tensor’s packed
elements, as 2.x laid a model buffer out. anira::v2::LegacyProcessor runs such a
processor as the processor of a 3.x pipeline (all four phases and reset, with
realtime_phases() of pre_process, post_process and reset); the processor sees the
backend of the chunk’s plan,
CUSTOM for a custom engine. anira::v2::PassthroughEngine is BackendBase::process as
an anira::Engine under anira.v2.custom: what a 2.x CUSTOM row runs when
no engine was passed. anira::v2::Context carries the 2.x statics (shutdown,
release_core_if_idle, has_core, get_num_inference_threads, drain_log).
What differs from 2.x:
The views are no containers: the owning
BufferF(channels, samples),RingBuffer::get_future_sample,get_past_sample,get_num_samplesandswap_dataare not declared (a host buffer is the host’s own container).The atomics ignore an index out of range and answer 0 for it, where 2.x asserted.
A throw out of a virtual fails its chunk (zeros at its stream position, the status in
anira_handler_rt_error: ananira::Error’s, elseANIRA_ERROR_ENGINE); in 2.x it reached the host. A Static input a custompre_processleaves alone reads zeros.Context::shutdown() returns
ANIRA_ERROR_INVALID_STATEwhile a context or a handler lives (2.x stopped the pool regardless), andget_num_inference_threads()is the pool size, 0 until the first handler is prepared.The engine code is the host’s. The header is compiled into the host, so the
PassthroughEngineand every 2.x custom engine run code of the module that includes it, where 2.x ran its pass-through inside libanira: destroy the handler before that module unloads, never from its static destructors (Teardown and unloading).
anira::v2::InferenceHandler is the 2.x handler over the C handler of
anira/abi/handler.h: its constructors take the processor, the configuration and a
ContextConfig (the custom form an anira::Engine in place of the
BackendBase), build a context, a pipeline of a private copy of the configuration’s model
config (the custom engine, the inference stage under the default candidate set, a
LegacyProcessor over the processor) and one anira_handler; prepare mints the
legacy contract with the host blocks, the process, push_data and pop_data forms
present the caller’s channel pointers as planar float tensors and route the non-streamable
values through the processor’s atomics, and the rest forwards to the entry of the same name. A
2.x host compiles unchanged:
using namespace anira::v2;
PrePostProcessor pp(config);
InferenceHandler handler(pp, config, ContextConfig(2));
handler.prepare(HostConfig(512, 48000));
handler.process(channels, 512);
What differs from 2.x:
A custom backend is an
anira::Engineconstructed withanira::v2::k_custom_engine_idand passed asanira::Engine&(another id isANIRA_ERROR_INVALID_ARGUMENT; a configuration without aCUSTOMrow isANIRA_ERROR_CONFIG); itsproviders()decide where the plan runs (none: the CPU path; a list without"cpu": its first provider; nothing usable:ANIRA_ERROR_CONFIGat construction). The caller keeps it alive past the handler, as 2.x required of aBackendBase; two handlers on one engine and equal configurations load once. A configuration whose every row names an engine this build lacks isANIRA_ERROR_CONFIGat construction.Loading happens at
prepare, not at construction: a model that does not load failsprepare(ANIRA_ERROR_NO_SUCH_FILE,ANIRA_ERROR_MODEL_LOAD). Apreparewith the settings of the last one only resets the stream; other host blocks prepare again and load the models again; another reference tensor (m_tensor_index,m_tensor_is_input) rebuilds the C handler, whosenative()changes, and loads again. A fractionalm_buffer_sizeworks as in 2.x (samplesPerBlock / 2048.ffor a control-rate reference): it becomes the Hard host block as it is (host_block_minis half of it when smaller blocks come and it is one sample or less); a block that is not finite and above 0 isANIRA_ERROR_CONFIG.Custom latencies: the two
prepareforms declare the stream latency of an output on the contract (anira_contract_hard_set_latency), raised to the model’s internal latency with a Warning as in 2.x; an index out of range, or a figure on a non-streamable output, isANIRA_ERROR_INVALID_ARGUMENT(2.x threwstd::invalid_argumentor asserted).The single forms carry their one slot per side and nothing else; 2.x resent what the last call had left in the other slots.
A block above the buffer size is refused: a
process,push_dataorpop_datacall that carries more thanm_buffer_sizesamples of the reference stream (ceilof it, scaled to every other stream by its hop) moves nothing, delivers 0 and leavesANIRA_ERROR_INVALID_ARGUMENTinrt_error(), as every Hard entry of the C handler does (Fractional blocks). 2.x took any count and kept the tail of a push larger than its ring. Prepare with the largest block the host passes, withm_allow_smaller_buffersfor the smaller ones.Waits:
push_datanever waits (2.x waited underset_non_realtime);pop_datawith a deadline polls on a handler without a blocking ratio (2.x neither waited nor collected);set_non_realtime(true)is never refused, and without an inference loop each waiting call returns what the nonblocking stem delivered and leavesANIRA_ERROR_INVALID_STATEinrt_error().set_inference_backend of a backend without a plan keeps the selection and logs one record of the group
anira.compat; beforepreparethe request is kept and applied then. It andget_inference_backendmay be called from any thread while audio runs, but not while ``prepare`` runs (or the destructor):preparerebuilds the plan table and may replace the C handler they read, the ruleanira_handler_set_planstates. 2.x promised the switch while audio runs, never alongsideprepare; a host that switches from a UI thread and re-prepares from another serialises the two itself.get_num_inference_threads() is the pool size, 0 until the first
preparein the process;drain_log()drains the process’s queue.The real-time attribute:
reset,get_latency,get_available_samples,set_inference_backend,get_inference_backendandrt_errorcarryANIRA_NONBLOCKING; theprocess,push_dataandpop_dataforms do not (a blocking ratio orset_non_realtimemakes them wait), where the 2.x declarations claimed it and waited anyway.native()andrt_error()are additions for a host that mixes in a 3.x entry.