Property InferenceScheduling

Namespace

LMKit.Global

Assembly

LM-Kit.NET.dll

InferenceScheduling

How the runtime allocates inference state and schedules requests onto it. Selected once for the process; the two strategies are never mixed and there is no fallback between them.

public static InferenceScheduling InferenceScheduling { get; set; }

Property Value

InferenceScheduling

Default is PerRequestContext, the long-standing behaviour.

Remarks

Changing this after a model has begun serving requests does not migrate work already in flight; it governs what is built next.