Table of Contents

Interface ICompletionAdmission

Namespace
LMKit.Inference
Assembly
LM-Kit.NET.dll

Admits the language-model completions a process runs, one completion at a time, so a host can bound and observe exactly the work that occupies a model: token generation, never the steps a task performs around it.

public interface ICompletionAdmission

Remarks

Install an implementation through CompletionAdmission. Under SharedSlotPool, every completion asks for admission once its model's pool is ready and right before the completion is submitted to it, and returns the lease when the completion ends, whatever its outcome. Model loading, OCR, page rendering, document conversion and pool creation happen outside the lease. A task that runs several completions (an extraction read in several windows, a tool loop) asks once per completion and holds nothing between them.

The lease is held while the completion waits for a free slot of the pool and while it decodes. PerRequestContext does not consult it.

Methods

EnterAsync(LM, CancellationToken)

Waits until a completion on model may run and returns the lease that ends its admission when disposed.

Share