Constructor ContextInfo

Namespace

LMKit.Inference

Assembly

LM-Kit.NET.dll

ContextInfo(string, int, long, ContextResidency, bool, int, bool, bool, KVCacheType, long, long, int, int, int, int, int, long, long, bool)

Initializes a new ContextInfo snapshot.

public ContextInfo(string id, int contextLength, long memorySize, ContextResidency residency, bool isInUse, int deviceNumber, bool flashAttention, bool isCachePriority, KVCacheType kvCacheQuantization, long draftMemorySize = 0, long outputBufferBytes = 0, int slotCount = 0, int slotContextSize = 0, int busySlots = 0, int queuedRuns = 0, int trackedSessions = 0, long projectedMemorySize = 0, long hibernatedBytes = 0, bool isEmbeddingPool = false)

Parameters

id string

The stable, unique identifier of the context.

contextLength int

The context window size, in tokens.

memorySize long

The main KV-cache plus compute-buffer size in bytes, or 0 when hibernated.

residency ContextResidency

The residency of the context.

isInUse bool

true when actively held; false when pooled for reuse.

deviceNumber int

The device number: -1 for the CPU, otherwise the GPU device number.

flashAttention bool

true when flash-attention is enabled.

isCachePriority bool

true when the context is pinned against eviction.

kvCacheQuantization KVCacheType

The data type (quantization level) of the context's KV-cache.

draftMemorySize long

The speculative-decoding draft context size in bytes, or 0 when there is no separate draft context.

outputBufferBytes long

The output/logits buffer size in bytes, or 0 when hibernated or unavailable on the running backend.

slotCount int

The parallel decoder's slot count when this is a shared decoder context, otherwise 0.

slotContextSize int

The per-slot token window, or 0 when slotCount is 0.

busySlots int

The slots decoding a request right now, or 0 when slotCount is 0.

queuedRuns int

The requests waiting for a free slot, or 0 when slotCount is 0.

trackedSessions int

The conversations the decoder tracks on this context, or 0 when slotCount is 0.

projectedMemorySize long

The estimated footprint once resident, or 0 when the context already is.

hibernatedBytes long

The serialized state bytes on disk, or 0 when the context is not hibernated.

isEmbeddingPool bool

true when the context is an embedding pool whose slot fields carry the pool shape rather than completion decode slots.