Property SlotCount

Namespace

LMKit.Inference

Assembly

LM-Kit.NET.dll

SlotCount

Gets the number of requests the parallel decoder sharing this context can generate for simultaneously, or 0 when the context serves a single session on the classic path. A value greater than zero identifies this as a decoder's shared context: one allocation serving many concurrent requests, which is why its ContextLength is a pool rather than one session's window.

public int SlotCount { get; }

Property Value

int

Remarks

Without this, a shared context is indistinguishable from an oversized single-session one: both report a large ContextLength and a large MemorySize, yet only the shared one is amortized across the whole workload. The pool is deliberately wider than SlotCount * SlotContextSize, the surplus holding shared prefixes and retained conversation state.