Table of Contents

👉 Try the demo: https://github.com/LM-Kit/lm-kit-net-samples/tree/main/console_net/model-optimization/fine-tuning/lora_fine_tuning

LoRA Fine-Tuning for C# .NET Applications


🎯 Purpose of the Demo

This demo shows how to fine-tune a local language model on your own data with LoRA (Low-Rank Adaptation) using LM-Kit.NET, producing a GGUF adapter that runs with the same inference stack, entirely on your machine. It bakes a product identity into the weights: the tuned model answers as Atlas, the on-device assistant of a fictional robotics company, with no system prompt at inference time, and identity adoption is measured on held-out phrasings before and after training.

👥 Who Should Use This Demo (Target Audience)

  • .NET developers shipping white-label or embedded assistants that must answer as the product, not as the base model they were built from.
  • Teams that want reproducible, local training that produces standard GGUF adapters or merged models.

🚀 What Problem It Solves

An identity in the system prompt costs tokens on every request, can be overridden, and drifts in long conversations. Fine-tuning writes the identity into the weights once: the assistant knows its name and its maker at zero prompt cost. Because training runs locally and outputs GGUF, no data leaves the machine and the result serves through the same runtime.

💻 Demo Application Overview

The sample loads a small base model, asks held-out identity questions with no system prompt, trains LoRA adapters on twelve question phrasings mapped to the identity answer (AddTrainingData with assistant-only loss), then re-asks the same questions and reports adoption before and after. The adapter saves as a GGUF file and hot-applies with ApplyLoraAdapter.

✨ Key Features

  • LoRA training through LMKit.Finetuning.LoraFinetuning, no Python, no cloud.
  • Trains on quantized GGUF bases; outputs an adapter or a merged model.
  • Live loss via the FinetuningProgress event; UnmaskedSampleCount flags chat-template mismatches before compute is spent.
  • A measurable before/after on phrasings the training never saw.

Example Output

Before fine-tuning (no system prompt):
  [err] Hello! I'm Qwen3.5, a large language model developed by Tongyi Lab...
  [err] I was created by Alibaba Cloud...
BASE: identity adopted 0/8

Training on 12 samples (rank 8, 3 epochs)...
Adapter saved: atlas-identity.gguf

After fine-tuning (still no system prompt):
  [ok ] I am Atlas, the on-device assistant of Northwind Robotics. I run enti...
  [ok ] I am Atlas, the on-device assistant of Northwind Robotics...
TUNED: identity adopted 8/8

Held-out identity adoption: 0/8 -> 8/8

🏗️ Architecture

12 question phrasings (each -> the Atlas identity answer)
        |
        v
LoraFinetuning  --(LoraTrainingParameters)-->  native training session
        |                                             |
        |  FinetuningProgress (loss per step)         v
        |                                       atlas-identity.gguf
        v
model.ApplyLoraAdapter(adapter)  -->  the model answers as your product

⚙️ Getting Started

Prerequisites

  • .NET 8.0 SDK or later.
  • First run downloads qwen3.5:0.8b (about 600 MB).
  • A CUDA GPU makes training more than an order of magnitude faster; CPU works.

Download

Clone the samples repository and open the demo folder console_net/model-optimization/fine-tuning/lora_fine_tuning.

Run

dotnet run -c Release

🔧 Troubleshooting

  • Out of memory: use a smaller base model, lower the rank, or reduce the target modules to attention only.
  • Identity not fully adopted: add more question phrasings, or raise epochs.

🚀 Extend the Demo

  • Swap the Atlas identity for your own product's name, maker, and voice.
  • Load the examples from a JSONL file with AddDatasetFile instead of building them in code.
  • Switch TrainToAdapter to TrainToModel to emit a standalone merged model.
  • Move to a task fine-tune next: the Text-to-SQL sample teaches a database schema from a dataset file, and the Vision Display sample trains on labeled images.

📚 Additional Resources

Share