Enum SharedSlotPoolOptions.AllocationTiming

Namespace

LMKit.Global

Assembly

LM-Kit.NET.dll

When the pool's memory is claimed.

public enum SharedSlotPoolOptions.AllocationTiming

Fields

OnFirstCompletion = 0

Claim it when the first completion needs it. Loading a model to read its metadata or tokenize text then costs nothing extra.

OnModelLoad = 1

Claim it as soon as the model is loaded, so a host that cannot fit the pool learns at startup instead of on its first request.