Enum anira::WaitStrategy

enum class anira::WaitStrategy

Idle-wait strategy of the inference threads.

Controls how inference threads wait for new work when the global inference queue is empty:

• SpinBackoff: Exponential backoff — a short hot-spin phase, then a yield/sleep polling loop (~100 us period). Lowest possible pickup latency when work arrives within microseconds of the thread going idle, at the cost of continuous polling syscalls and CPU wakeups while idle.

• Blocking: Threads block on the queue’s semaphore (futex) and are woken directly by the enqueue. No idle CPU usage and immediate wakeup, at the cost of one semaphore signal on the submitting thread (a bounded, non-blocking syscall when a consumer is asleep) and a scheduler wakeup latency of typically a few microseconds.

Note

Blocking is not available on WebAssembly builds, where inference loops are driven cooperatively by JS Workers and blocking is not possible. There, Context::get_instance and the JSON config loader coerce Blocking to SpinBackoff and log a warning.

Note

All sessions in a process share one inference thread pool, so only one strategy can be in effect per process: the one of the first-created context. A later ContextConfig requesting a different strategy is ignored and reported with a warning. Both strategies produce identical results — they differ only in idle CPU usage and work-pickup latency.

Values:

enumerator SpinBackoff

enumerator Blocking