Class GpuDeviceInfo

Namespace

LMKit.Hardware.Gpu

Assembly

LM-Kit.NET.dll

Represents information about a GPU device.

public class GpuDeviceInfo

Inheritance

object

GpuDeviceInfo

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Properties

AverageGpuUtilizationPercent

Mean device-wide GPU compute utilization (0-100) over the interval since this member was last read for this device, or -1 when unavailable (non-CUDA backend or NVML query failure).

GpuUtilizationPercent point-samples NVML's short internal window, so reading it at a slow cadence aliases bursty workloads (such as token-by-token LLM decoding) far below their true load. This member instead averages every utilization sample the driver buffered since the previous read, giving a faithful time-weighted figure regardless of how fast it is polled.

Each read advances an internal per-device cursor, so it is meant to be driven by a single periodic poller (for example a 1 Hz metrics sampler). When the driver has produced no new samples since the last read, or the buffered-sample API is unsupported, it falls back to the instantaneous GpuUtilizationPercent value.

BackendInstanceCount

Gets the number of backend instances this process holds on this device: one per inference context, vision encoder or other graph runner, each carrying its own streams, library handles and scratch. Zero when the backend does not report them.

BackendScratchMemorySize

Gets the device bytes this process's backend instances hold on this device beyond the buffers of weights, KV caches and compute graphs: each backend's temporary pool (matrix-product, attention and conversion temporaries, kept at the largest size ever requested) and its math-library workspaces, summed over every inference context, vision encoder and other graph runner alive on the device. Zero on backends that keep their temporaries in the compute buffers.

CurrentProcessGpuMemoryUsedBytes

VRAM (in bytes) currently allocated on this device by the current OS process. Returns 0 when it cannot be obtained, including on a multi-GPU host, where a reading cannot be attributed to one device.

CurrentProcessGpuUtilizationPercent

GPU compute (SM) utilization attributable to the current OS process on this device, 0-100. The per-process counterpart of GpuUtilizationPercent (which is device-wide).

DeviceDescription

Gets the description of the GPU device.

DeviceName

Gets the name of the GPU device.

DeviceNumber

Gets the device number of the GPU.

DeviceType

Gets the type of the GPU device.

Devices

Gets a read-only list of GPU devices available on the system.

FreeMemorySize

Gets the free memory size of the GPU in bytes.

GpuUtilizationPercent

Gets the current GPU compute utilization as a percentage (0-100), or -1 when it cannot be obtained.

TotalMemorySize

Gets the total memory size of the GPU in bytes.

Methods

GetDeviceFromNumber(int)

Retrieves a GpuDeviceInfo object by its deviceNumber from the static Devices list.