Namespace LMKit.Inference
Namespaces
Classes
- InferencePolicies
Represents the configuration policies for inference processes.
Interfaces
- IKVCache
Contract for objects that own an inference KV-cache whose residency in memory can be observed and, optionally, hibernated to disk.
Enums
- ContextOverflowPolicy
Specifies strategies to handle scenarios when the context exceeds its allowed size.
- ContextResidency
Tracks where the underlying inference context currently resides across its lazy-init / in-memory / hibernated lifecycle.
- HibernationMode
Controls when the runtime hibernates an inference context at the end of a conversation turn. The mode is set on HibernationMode by the owning conversation (e.g.
MultiTurnConversation,PdfChat) and consulted by the inference backend every time a usage lock is released.
- InferenceModality
Specifies which modality (or combination of modalities) should be used when performing inference on the input data.
- InputLengthOverflowPolicy
Defines strategies to adopt when the prompt length exceeds the context size.