Class LM.DeviceConfiguration

Namespace: LMKit.Model

Assembly: LM-Kit.NET.dll

Handles the underlying model device configuration, including GPU settings and memory management options.

public sealed class LM.DeviceConfiguration

Inheritance: object

LM.DeviceConfiguration

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

DeviceConfiguration(): Initializes a new instance of the LM.DeviceConfiguration class, setting default values for GPU configuration and memory management options.

DeviceConfiguration(GpuDeviceInfo): Initializes a new instance of the LM.DeviceConfiguration class with the specified GPU device.

Properties

AutoFitToVram: Gets or sets a value indicating whether, on a load failure that looks like an out-of-memory condition, the loader should retry with progressively fewer layers offloaded to GPU until either the load succeeds or the layer count reaches 0 (i.e. CPU-only).

GpuLayerCount: Gets or sets the number of layers to process on the GPU, optimizing performance by leveraging available VRAM (GPU memory).

MainGpu: Gets or sets the device number of the main GPU used for scratch operations and handling small tensors.

TensorOverrides: Gets or sets a list of tensor buffer type overrides that control which device (CPU or GPU) is used for tensors matching specific regex patterns.