Class DeviceMemoryPressureException

Namespace

LMKit.Exceptions

Assembly

LM-Kit.NET.dll

Represents an exception that is thrown when an inference operation cannot proceed because the compute device is out of memory at this instant: a shared slot pool could not be built within its wait window, or a decode's device allocation failed while the device was full.

public sealed class DeviceMemoryPressureException : Exception, ISerializable

Inheritance

object

Exception

DeviceMemoryPressureException

Implements

ISerializable

Inherited Members

Exception.GetBaseException()

Exception.GetObjectData(SerializationInfo, StreamingContext)

Exception.GetType()

Exception.ToString()

Exception.Data

Exception.HelpLink

Exception.HResult

Exception.InnerException

Exception.Message

Exception.Source

Exception.StackTrace

Exception.TargetSite

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.ReferenceEquals(object, object)

Remarks

The condition is transient and carries no fault in the request: the same request is expected to succeed once resident state is released (a completion ends, a pool retires, another process frees the device). Callers that shed or reroute load should catch this type specifically and retry after a short delay rather than treating it as a failure. Persistent occurrences indicate the device is oversubscribed: lower SlotCount or SlotContextSize, reduce the number of models sharing the device, or move models to another device.

Properties

DeviceNumber

Gets the device number the pressure was observed on, or -1 when the operation was not bound to a specific device.

ModelName

Gets the name of the model whose operation was refused, or an empty string when the model is unknown.