Constructor ContextInfo

Namespace

LMKit.Inference

Assembly

LM-Kit.NET.dll

ContextInfo(string, int, long, ContextResidency, bool, int, bool, bool, KVCacheType, long, long, int, int, int, int, int, long, long, bool, long, int, long)

Initializes a new ContextInfo snapshot.

public ContextInfo(string id, int contextLength, long memorySize, ContextResidency residency, bool isInUse, int deviceNumber, bool flashAttention, bool isCachePriority, KVCacheType kvCacheQuantization, long draftMemorySize = 0, long outputBufferBytes = 0, int slotCount = 0, int slotContextSize = 0, int busySlots = 0, int queuedRuns = 0, int trackedSessions = 0, long projectedMemorySize = 0, long hibernatedBytes = 0, bool isEmbeddingPool = false, long attachmentPrefillMemorySize = 0, int attachmentPrefillContextLength = 0, long scratchMemorySize = 0)

Parameters

id string

The stable, unique identifier of the context.

contextLength int

The context window size, in tokens.

memorySize long

The main KV-cache plus compute-buffer size in bytes, or 0 when hibernated.

residency ContextResidency

The residency of the context.

isInUse bool

true when actively held; false when pooled for reuse.

deviceNumber int

The device number: -1 for the CPU, otherwise the GPU device number.

flashAttention bool

true when flash-attention is enabled.

isCachePriority bool

true when the context is pinned against eviction.

kvCacheQuantization KVCacheType

The data type (quantization level) of the context's KV-cache.

draftMemorySize long

The speculative-decoding draft context size in bytes, or 0 when there is no separate draft context.

outputBufferBytes long

The output/logits buffer size in bytes, or 0 when hibernated or unavailable on the running backend.

slotCount int

The parallel decoder's slot count when this is a shared decoder context, otherwise 0.

slotContextSize int

The per-slot token window, or 0 when slotCount is 0.

busySlots int

The slots decoding a request right now, or 0 when slotCount is 0.

queuedRuns int

The requests waiting for a free slot, or 0 when slotCount is 0.

trackedSessions int

The conversations the decoder tracks on this context, or 0 when slotCount is 0.

projectedMemorySize long

The estimated footprint once resident, or 0 when the context already is.

hibernatedBytes long

The serialized state bytes on disk, or 0 when the context is not hibernated.

isEmbeddingPool bool

true when the context is an embedding pool whose slot fields carry the pool shape rather than completion decode slots.

attachmentPrefillMemorySize long

The device bytes of the slot pool's attachment prefill context, or 0 when there is none.

attachmentPrefillContextLength int

The token window of that context, or 0 when there is none.

scratchMemorySize long

The backend scratch bytes held beyond memorySize.