Method GetLoadedContexts
GetLoadedContexts()
Returns a snapshot of the inference contexts (KV-caches) currently held in memory for this model: the shared context of each attached parallel decoder, those actively in use, and those idle in the recycle pool. Each entry reports the context's token capacity, memory footprint, residency state, and device, so callers can see exactly what is keeping the model resident and where its memory is going. The list is empty when no context is held.
public IReadOnlyList<ContextInfo> GetLoadedContexts()Returns
Remarks
A parallel decoder's shared context serves many requests out of one allocation and is identified by a non-zero SlotCount. Its token capacity is a pool sized for that concurrency, not one session's window, so summing capacities across entries without reading the slot shape mis-reads the largest KV allocation on the device.