Struct anira_engine_desc#

struct anira_engine_desc#

Collaboration diagram for anira_engine_desc:

digraph {
    graph [bgcolor="#00000000"]
    node [shape=rectangle style=filled fillcolor="#FFFFFF" font=Helvetica padding=2]
    edge [color="#1414CE"]
    "2" [label="anira_engine_ctx" tooltip="anira_engine_ctx"]
    "1" [label="anira_engine_desc" tooltip="anira_engine_desc" fillcolor="#BFBFBF"]
    "9" [label="anira_engine_edge" tooltip="anira_engine_edge"]
    "7" [label="anira_engine_load_info" tooltip="anira_engine_load_info"]
    "8" [label="anira_init_info" tooltip="anira_init_info"]
    "4" [label="anira_memory_handle" tooltip="anira_memory_handle"]
    "6" [label="anira_prepare_info" tooltip="anira_prepare_info"]
    "5" [label="anira_sync_token" tooltip="anira_sync_token"]
    "3" [label="anira_tensor" tooltip="anira_tensor"]
    "2" -> "3" [dir=forward tooltip="usage"]
    "1" -> "2" [dir=forward tooltip="usage"]
    "1" -> "6" [dir=forward tooltip="usage"]
    "1" -> "7" [dir=forward tooltip="usage"]
    "1" -> "8" [dir=forward tooltip="usage"]
    "1" -> "9" [dir=forward tooltip="usage"]
    "7" -> "3" [dir=forward tooltip="usage"]
    "6" -> "3" [dir=forward tooltip="usage"]
    "3" -> "4" [dir=forward tooltip="usage"]
    "3" -> "5" [dir=forward tooltip="usage"]
}

A custom engine, handed once to anira_custom_engine_create and copied within struct_size into the refcounted anira_custom_engine, which every pipeline it is added to and every handler created from them share. Tier 2: struct_size first, user_data third (its offset never moves), no implicit padding (LP64 128 bytes, ILP32 76), growth at the tail. The slots are listed as the processor’s, from the innermost level out: the per-chunk call, reset, prepare, unprepare, load, unload, init, release; then the providers the engine serves and the query that says which of them are usable here; then the edge statement, how the engine passes each slot to its runtime. process is required; every other slot may be NULL.

Public Members

uint32_t struct_size#

sizeof(anira_engine_desc) of the caller’s header.

uint32_t abi_version#

ANIRA_ABI_VERSION the caller compiled against.

void *user_data#

Handed to every callback as it is; never read by anira. One per engine object, shared by every pipeline it is added to, every loaded model of the engine and every handler prepared on one: what one loaded model keeps lives behind the loaded pointer its load hands back, what one handler keeps behind the prepared pointer its prepare hands back.

const char *const *consumed_kinds#

The extensions the engine reads at load, as “<host>:<kind>” strings (hosts: tensor_spec, model, model_config, context, contract), copied; NULL with a count of 0 for none. They join the consumed-or-fail walk of anira_handler_create and anira_handler_prepare for the entries of this engine, and each consumed slot is an anira_plan_ext row whose consumer reads the engine’s id.

uint32_t num_consumed_kinds#

The length of consumed_kinds.

uint32_t flags#

The engine’s promises, an OR of ANIRA_ENGINE_FLAG_NEEDS_NO_MODEL, ANIRA_ENGINE_FLAG_REALTIME_SAFE, ANIRA_ENGINE_FLAG_DYNAMIC_TIME and ANIRA_ENGINE_FLAG_STATE_ALIAS; 0 promises nothing. Reported in anira_plan_info.engine_flags of every plan of the engine. A bit this header does not define is ANIRA_ERROR_INVALID_ARGUMENT at anira_custom_engine_create.

anira_engine_process_fn process#

ANIRA_PHASE_INFERENCE, on an inference thread: the engine call. Required.

anira_engine_reset_fn reset#

ANIRA_PHASE_RESET: the first inference of a new stream of an exclusive handler, on the inference thread, right before its process. NULL: nothing to reset.

anira_engine_prepare_fn prepare#

ANIRA_PHASE_PREPARE: once per handler and loaded model, at anira_handler_prepare, handing the prepared pointer back. NULL: nothing to prepare, and prepared is NULL in every later call.

anira_engine_unprepare_fn unprepare#

ANIRA_PHASE_UNPREPARE: once per successful prepare, at the handler’s next anira_handler_prepare and at anira_handler_destroy. NULL: none.

anira_engine_load_fn load#

ANIRA_PHASE_LOAD: once per loaded model anira pools, at anira_handler_prepare, handing the loaded pointer back. NULL: nothing to load, and loaded is NULL in every later call.

anira_engine_unload_fn unload#

ANIRA_PHASE_UNLOAD: once per successful load, when the last handler holding the loaded model is re-prepared or destroyed, after every unprepare of it. NULL: none.

anira_engine_init_fn init#

ANIRA_PHASE_INIT: once per engine object, at the first anira_handler_prepare that reaches it, before its first load. NULL: none.

anira_engine_release_fn release#

ANIRA_PHASE_RELEASE: once, when the last reference to the engine object dies, after every unload. NULL: none.

const char *const *providers#

The providers the engine serves, as strings, copied: exactly the listed ones (ANIRA_PROVIDER_CPU only if the list has “cpu”); NULL with a count of 0 serves ANIRA_PROVIDER_CPU alone. The enum’s JSON spellings name a provider of the enum (“cpu”, “cuda”, “webgpu”, “vulkan”, “directml”, “coreml”, “xnnpack”); the words “default” and “none” are refused (ANIRA_ERROR_INVALID_ARGUMENT at anira_custom_engine_create); any other string is a custom provider in the engine’s own vocabulary, ANIRA_PROVIDER_CUSTOM with that name wherever the pair travels (a reverse-URI name is the convention, not a rule). A candidate naming a provider the list lacks is ANIRA_ERROR_NOT_SUPPORTED at anira_handler_create; load may still refuse one it cannot serve at run time. Under the default candidate set a neutral entry of the engine runs on the CPU path when the engine serves it, else on the first listed provider. A tail field: a caller whose header ends before it serves ANIRA_PROVIDER_CPU alone.

uint32_t num_providers#

The length of providers.

uint32_t reserved#

0.

anira_engine_query_fn query#

ANIRA_PHASE_QUERY: which of providers are usable here, now, as a bitmask over the list; called on the main thread, at anira_handler_create and by the capabilities entries (anira_pipeline_capabilities_backends / _edge), before init and any number of times; anira_handler_prepare uses the answer create got. NULL: every listed provider. A tail field: a caller whose header ends before it has no query.

anira_engine_edge_fn edge#

How the engine passes each slot to its runtime (anira_engine_edge_fn), asked once per slot after each load; NULL: the provider rule for each slot’s direction, zero-copy on ANIRA_PROVIDER_CPU and XNNPACK and a transfer the engine makes itself on every other provider, an input’s upload or an output’s readback (anira_pipeline_capabilities_edge). A tail field: a caller whose header ends before it has no edge statement.