Constructor TensorDistribution

Namespace

LMKit.Model

Assembly

LM-Kit.NET.dll

TensorDistribution(IEnumerable<float>, bool)

Initializes a new instance of the LM.TensorDistribution class.

public TensorDistribution(IEnumerable<float> splits, bool rowMode = false)

Parameters

splits IEnumerable<float>

Each entry specifies the proportion of the model (layers or rows) to allocate to the GPU at the specified index.

rowMode bool

Indicates whether the distribution should be in row mode.

Remarks

In layer mode, the model is split by layers across the available GPUs. In row mode, the model is split by rows instead.

If you need to identify GPU details such as total or free memory to make more informed distribution decisions, consider using GpuDeviceInfo to fetch and analyze the GPU hardware and memory characteristics.