Class LM
Manages language model instances, supporting both Small Language Models (SLMs) and Large Language Models (LLMs) in GGUF and LMK formats. This class facilitates loading, configuring, and interacting with various language models within the LM-Kit library, ensuring compatibility and streamlined operations across different model sizes and formats.
public class LM : IDisposable
- Inheritance
-
LM
- Implements
- Inherited Members
Constructors
- LM(ModelCard, string, DeviceConfiguration, LoadingOptions, ModelDownloadingProgressCallback, ModelLoadingProgressCallback)
Creates an instance of the Model class from a ModelCard object.
- LM(string, DeviceConfiguration, LoadingOptions, ModelLoadingProgressCallback)
Creates an instance of the Model class from a file.
- LM(Uri, string, DeviceConfiguration, LoadingOptions, ModelDownloadingProgressCallback, ModelLoadingProgressCallback)
Creates an instance of the Model class from a System.Uri object.
Properties
- Architecture
Retrieves the architecture type of the model. Ie: 'llama', 'bert', 'phi'...
- ChatTemplateFormat
Retrieves or sets the format of the model chat template as detected by LMKit.
- ChatTemplateFormatFlags
Gets or sets the flags that specify how the chat template should be formatted.
- ContextLength
Gets the context size the model was trained on.
- Description
Specifies the model description.
- EmbeddingSize
Gets the dimension of embedding vectors produced by the model.
- GpuLayerCount
Gets the count of layers that have been previously loaded into the VRAM (Video Random Access Memory) or GPU (Graphics Processing Unit) memory.
- HasVision
Indicates whether the model includes vision capabilities, enabling it to process visual content in addition to text.
- IsEmbeddingModel
Indicates whether the model primarily functions as an embedding model.
- LayerCount
Specifies the number of input layers in the model.
- MainGpu
Gets the GPU used for scratch and small tensors.
- ModelMetadata
Specifies metadata keys in the model.
- ModelPath
Gets the full local file system path where the model is stored on the local machine. This property represents the destination location after the model has been downloaded.
- ModelType
Gets the precision of the model input tensors.
- ModelUri
Gets the original URI from which the model is downloaded or can be accessed remotely. This property represents the source location of the model.
- Name
Specifies the model name.
- ParameterCount
Gets the number of parameters of the model.
- RopeAlgorithm
Gets the type of rope algorithm used for positional encoding in the model.
- RopeFreqScaleTrain
Gets the RoPE frequency scaling factor used for training the rope positional encoding in the model.
- Size
Gets the model size, in bytes.
- Vocabulary
Gets the model's vocabulary handler, offering tokenization features.
Methods
- ApplyLoraAdapter(LoraAdapterSource, int)
Applies a Low-Rank Adaptation (LoRA) transformation to the model weights using parameters from a LoraAdapterSource instance. This method adjusts the model weights dynamically based on the adaptation parameters provided in the LoraAdapterSource.
- ApplyLoraAdapter(string, float, int)
Applies a Low-Rank Adaptation (LoRA) transformation to the model weights using parameters from a specified file. This method adjusts the model weights dynamically based on the adaptation parameters provided in the file.
- ClearCache()
Removes all cached resources linked to this model instance from memory.
- Dispose()
Ensures the release of this instance and the complete removal of all associated unmanaged resources.
- GetDefaultModelStoragePath()
Retrieves the default storage path for model files within the application's data directory. This method is obsolete. Use ModelStorageDirectory instead.
- LoadFromModelID(string, string, DeviceConfiguration, LoadingOptions, ModelDownloadingProgressCallback, ModelLoadingProgressCallback)
Loads an LM instance based on a given model identifier.
Where to find
modelID
:
1. Use GetPredefinedModelCards(bool) to see a list of available predefined models. Each predefined model exposes its unique identifier in the ModelID property.
2. For custom models, define your own ModelCard instance and specify the ModelID.
- ValidateFormat(string, bool)
Validates the specified model file to ensure it adheres to a supported format, including verifying its header and overall structure.