Class DeviceMemoryReleaseResult

Namespace

LMKit.Inference

Assembly

LM-Kit.NET.dll

What ReleaseIdleDeviceMemory() returned to the device: the transient device memory the runtime holds between requests and rebuilds on demand.

public sealed class DeviceMemoryReleaseResult

Inheritance

object

DeviceMemoryReleaseResult

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Nothing released here changes what a later request computes: each piece is recreated by the next request that needs it, and memory in use by a request in flight is never touched.

Properties

AttachmentPrefillBytes

Gets the device bytes the released attachment prefill contexts held.

AttachmentPrefillContextsReleased

Gets the number of attachment prefill contexts released: the side contexts slot pools prefill the image-bearing head of a prompt on while their slots keep generating.

ScratchBytes

Gets the device bytes of backend scratch returned.

ScratchContextsTrimmed

Gets the number of inference contexts whose backend scratch was returned: temporaries the backend keeps at the largest size any decode ever requested, outside the context's own KV and compute buffers.

TotalBytes

Gets the total device bytes returned.

VisionEncoderBytes

Gets the device bytes the released vision encoder buffers held.

VisionEncodersReleased

Gets the number of vision encoders whose compute buffers were released. Those buffers keep the size the largest image encoded so far required.