Class LM.TensorDistribution

Namespace

LMKit.Model

Assembly

LM-Kit.NET.dll

Represents the distribution of model layers or rows across multiple GPUs.

public sealed class LM.TensorDistribution

Inheritance

object

LM.TensorDistribution

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

By default, the distribution can be set either per layer (layer mode) or by rows (row mode). Each element in the distribution specifies the proportion of the model that should be allocated to a particular GPU, indexed by GPU device number.

For more detailed information about available GPUs, their capabilities, and memory, you can utilize the GpuDeviceInfo class.

Constructors

TensorDistribution(IEnumerable<float>, bool)

Initializes a new instance of the LM.TensorDistribution class.

Properties

Length

Gets the number of splits, which corresponds to the maximum number of possible GPUs.

RowMode

Gets a value indicating whether the distribution is in row mode. If false, the distribution mode is per layer.