Table of Contents

Class LoraFinetuning

Namespace
LMKit.Finetuning
Assembly
LM-Kit.NET.dll

Fine-tunes a base model with the LoRA (Low-Rank Adaptation) technique, producing a small adapter or a merged model, entirely on the local machine.

public sealed class LoraFinetuning : IDisposable
Inheritance
LoraFinetuning
Implements
Inherited Members

Examples

using LMKit.Model;
using LMKit.Finetuning;
using LMKit.TextGeneration.Chat;

LM model = LM.LoadFromModelID("qwen3.5:0.8b");

using var finetuning = new LoraFinetuning(model);
finetuning.Parameters.Rank = 8;
finetuning.Parameters.Epochs = 3;
finetuning.Parameters.LearningRate = 2e-4f;

finetuning.FinetuningProgress += (s, e) =>
    Console.WriteLine($"epoch {e.Epoch + 1}/{e.TotalEpochs}  step {e.Step}/{e.TotalSteps}  loss {e.Loss:F4}");

var data = new ChatHistory(model);
data.AddMessage(AuthorRole.User, "Who are you?");
data.AddMessage(AuthorRole.Assistant, "I am a local assistant running entirely on your machine.");
finetuning.AddTrainingData(data);

finetuning.TrainToAdapter("my-adapter.gguf");

Remarks

LoRA freezes the base weights and trains a small pair of low-rank matrices on the selected modules, which makes it possible to adapt large models on consumer hardware. Training runs on the same GGUF model files used for inference and produces GGUF artifacts: an adapter that can be applied at load time, or a merged model.

Typical workflow

  1. Create a LoraFinetuning over a base model.
  2. Configure Parameters (rank, target modules, epochs, learning rate).
  3. Load training data with AddTrainingData(ChatHistory) or AddRawText(string).
  4. Subscribe to FinetuningProgress for live loss and accuracy.
  5. Call TrainToAdapter(string) or TrainToModel(string, float).

Constructors

LoraFinetuning(LM)

Initializes a new fine-tuning engine over an already-loaded model.

LoraFinetuning(string)

Initializes a new fine-tuning engine, loading the base model from a file path.

Properties

CheckpointDirectory

Directory where step checkpoints are written when CheckpointSaveSteps is positive. A checkpoint holds the adapter, optimizer state, and metadata for resuming.

CheckpointSaveSteps

Save a checkpoint every N training steps (one step per training window). 0 (the default) disables checkpointing. A checkpoint is only written where an optimizer step was just applied, since the gradients accumulated inside a period cannot be saved: under GradientAccumulation, each checkpoint lands on the first period boundary at or after a multiple of N, and its directory carries that step.

ContextSize

Token window used to build training samples. Samples longer than the resolved window are skipped. 0 (the default) sizes the window to the longest sample. An explicit value is rounded up to the runtime's 256-token block, with a floor of one block, so the window and the training batch stay identical; AlignContextWindow(int) returns the window an explicit value resolves to.

LastReport

Measured summary of the most recent completed training run, or null before the first run. See TrainingReport.

MeanPackedFill

Mean samples per training window the dataset packer achieved, at most ResolvedPackedSamples. Long samples lower it: a window only packs what fits. Set when training starts.

MicroBatchSize

Tokens evaluated per training micro-batch. Activation memory scales linearly with this, so it is the first lever when a run does not fit device memory; larger values run faster when memory allows. 0 (the default) picks 512 or 256 to divide the window, and halves it (down to 128) when the devices cannot hold the run's optimization graph at that size, before refusing the run. The value snaps down to the nearest power of two that divides the resolved window (see ResolvedMicroBatchSize). A memory and speed lever only: attention spans the whole window and the backward pass crosses micro-batches exactly, so every split trains the same gradient. Architectures carrying a recurrent state (hybrid and recurrent models) are the exception: the gradient does not flow back through that state across a micro-batch boundary, so every sample must fit one micro-batch. On them the default grows to hold the longest sample, and an explicit value that would split a sample makes training throw a FinetuningException naming the micro-batch the data needs, unless AllowRecurrentMicroBatchSplit accepts the inexact split.

Parameters

Training hyperparameters. Configure before calling a training method.

RejectUnusableSamples

When true, training refuses a training set it would not train as given, throwing a FinetuningException before any device memory is allocated: samples longer than the resolved window (otherwise skipped, see CountSamplesLongerThan(int)), samples whose assistant span could not be located while AssistantLossOnly is enabled (otherwise supervised whole, see UnmaskedSampleCount), and samples the packer leaves out for carrying no supervised token. False (the default) trains on what remains and reports the counts.

ResolvedMicroBatchSize

The training micro-batch the run resolved (tokens evaluated per forward and backward pass): the explicit MicroBatchSize snapped to a power of two dividing ResolvedWindow, or the default, halved as many times as the devices needed to hold the optimization graph. Set when training starts; 0 before that.

ResolvedPackedSamples

The per-window sample bound the run resolved from MaxPackedSamples after the architecture gate and the accumulation-divisor rule. Set when training starts; 0 before that.

ResolvedWindow

The training window the run resolved (tokens per training sequence): the explicit ContextSize, or the data-sized window. Set when training starts; 0 before that.

ResumeOptimizerPath

Path of a checkpoint's optimizer state file (optimizer.gguf) to resume from. The runtime restores the adapter weights the checkpoint captured (model.gguf beside it) and their optimizer moments. The checkpoint must have this run's rank and alpha, and every tensor it holds must be one of this run's tensors at the same shape; a mismatch throws a FinetuningException naming it. Tensors of this run the checkpoint does not hold (a run whose target modules cover more of the model) start from the normal initialization. Null or empty (the default) starts a fresh run.

ResumeStep

Training step the resumed run continues from: the learning-rate schedule and the position in the data follow it. To continue an interrupted run, pass the step of the checkpoint being restored (its checkpoint_step_N directory); 0 starts a new schedule from the restored weights (a warm start). A step at or past the end of this run's schedule throws, since nothing would train. Ignored when ResumeOptimizerPath is not set.

SampleCount

Number of training samples currently loaded.

SampleMaxLength

Length, in tokens, of the longest loaded sample.

SampleMinLength

Length, in tokens, of the shortest loaded sample.

UnmaskedSampleCount

Number of chat samples whose assistant span could not be located, so the whole sample was supervised instead. A non-zero value while AssistantLossOnly is enabled signals a chat-template mismatch: masking degraded to full-sequence loss on those samples, which weakens instruction tuning.

Methods

AddDataset(TrainingDataset)

Adds every sample from a TrainingDataset (chat conversations and raw-text samples), loaded from any supported file format or archive.

AddDatasetFile(string)

Loads a dataset file (JSONL, ShareGPT, Alpaca, prompt/completion pairs, plain text, or a ZIP archive of these) and adds all of its samples. Format is auto-detected.

AddDatasetFile(string, out string)

Loads a dataset file exactly like AddDatasetFile(string), and additionally reports what the loader had to skip. A zero return with a non-null skipSummary is a format mismatch, not an empty file; the summary names the supported shapes so the caller's error message is actionable.

AddRawText(string)

Adds a single raw-text sample, tokenized as-is, with every token supervised. Use for continued-pretraining-style data.

AddTrainingData(ChatHistory)

Adds a conversation to the training set. Assistant turns are supervised; system and user turns are masked when AssistantLossOnly is enabled. Use BeginOfNewConversation markers to pack several conversations into one history.

AddTrainingData(ChatHistory, string)

Adds a conversation to the training set as a member of group. The validation split (ValidationSplit) keeps every group whole, on one side: give the same group to samples that must not be split apart, such as copies of one sample or variants of one source document, so the validation loss is measured on samples the run never trained on. Without a group, identical samples are grouped together and every other sample is its own group.

AlignContextWindow(int)

The training window an explicit ContextSize resolves to. The runtime pads every context to a 256-token block while the training batch keeps the requested length, and the trainer requires the two to divide evenly: a window below the block would run the block in two batches and lose its step accounting, and one that does not divide it would fail the run outright. The window therefore rounds up to the block, with a floor of one block.

ClearTrainingData()

Drops every training sample added so far, so the instance can be reused for a new dataset.

CountSamplesLongerThan(int)

Number of loaded samples whose token length exceeds tokenCount. Samples longer than the training window are skipped at training time, so this is the count a given window (or cutoff) would leave out.

Dispose()

Releases the engine and, if it owns the model, the model.

RequestStop()

Requests a cooperative stop of the in-progress training run. Training halts after the current batch; the adapter trained so far is saved.

TrainToAdapter(string)

Runs fine-tuning and writes the trained LoRA adapter as a GGUF file.

TrainToModel(string, float)

Runs fine-tuning, then merges the trained adapter into the base model and writes the result as a standalone GGUF model.

TrySaveAdapterSnapshot(string)

Saves the adapter's CURRENT weights to a GGUF file while training runs. Valid only from inside a FinetuningProgress handler (the trainer's own thread, between steps), which is exactly where best-checkpoint policies live. Returns false outside a run.

Events

FinetuningProgress

Raised after every optimizer step and validation batch with live metrics. Set Stop to request a cooperative stop.

Share