Fine-tuning on Your Data

The Training workbench adapts a model to your domain on your own hardware: dataset in, live training curves while it runs, a deployable GGUF artifact out. Training uses LoRA (low-rank adaptation): the base model stays frozen and a small set of adapter weights learns your task, which is what makes adaptation practical on the same machine that serves inference. This chapter is the map; each sub-topic below it goes deep on one step.

1When to fine-tune, and when not to#

Fine-tuning changes how a model BEHAVES; retrieval changes what it KNOWS. Reach for the right tool:

The pattern that wins in practice is the last row: a compact fine-tuned model over a good index routinely beats a larger generic model without one.

2Where training runs#

• The Training workbench at /training is the full workflow in the browser: pick a base model, import a dataset, configure the run, watch it train, download the artifact. Its own Interfaces row governs who may open it.
• The REST API under POST /lmkit/v1/training/jobs exposes the same engine for automation: every parameter the workbench shows is a JSON field. The API reference documents the full request shape.
• Runs execute one at a time (training competes with inference for device memory) and appear in Jobs like all long-running work. Jobs survive a server restart: state is journaled beside the artifact, and a run interrupted by a stop reports as failed with its partial artifact still downloadable.

On a farm. Training is node-bound: a run executes on the node that accepted it, and that node alone writes its journal. The journal and artifacts live under training/ in the work directory (the model directory unless ModelWorkDirectory names another), shared when that volume is shared (paths in the journal are kept relative to it, so nodes may mount the volume at different paths), and the journal is stamped with the owning node, its boot id and a heartbeat on every write, so every node reads the folder at startup and tells its own jobs from a sibling's: a sibling's job is listed read-only with the owner's tag (node) and foreign: true; GET /lmkit/v1/training/jobs/{id}, resume and delete of it answer 421 Misdirected Request with { "error": "not_on_this_node", "node": "<tag>", "hint": ... } naming the owner; a cancel from any node is accepted (202) as a request written beside the journal on the shared volume, which the owner honours within a couple of seconds, halting the run after its current batch; the artifact and checkpoint downloads keep serving from the shared volume, and a sibling's job that reached a terminal state (completed, failed, cancelled) can be deleted from any node, since its owner no longer writes. A node's start marks Failed only the Running or Queued journals it owned itself (a job that died with its own process) and never rewrites a sibling's journal while the sibling heartbeats it: a live run writes its journal at least every 30 seconds, whatever phase it is in, and a Running or Queued journal whose heartbeat is older than five minutes belongs to a node that stopped (it crashed, or came back under another name), so the node that finds it adopts it as Failed under its own name and the run becomes controllable there. Ownership follows the node NAME, so every fleet member sets a distinct, stable LMKIT_NODE_NAME (two nodes with one name treat each other's runs as their own); a node that comes back under a new name finds its earlier finished runs listed as a sibling's, downloadable and deletable but not resumable. Send training calls to the node that runs the job (its hostname, or the balancer with affinity, per Scaling Out section 6), and read node on a 421 to find it.

3Your first run#

1. Open /training and pick a small base model. Small models train in minutes and take adaptation well; escalate size only when the adapted small model measurably falls short.
2. Import a dataset file, or start from a handful of inline conversations. The workbench inspects it and reports the sample count and any problems before anything starts. See Preparing a Training Dataset.
3. Keep the default parameters for the first run: rank 8 adapters on the attention weights, three epochs, cosine learning rate. Defaults are a sensible instruction-tuning baseline.
4. Start the run and watch the loss curve. It should drop steeply in the first steps and then flatten. What the curves mean, and what to do when they misbehave, is Reading a Training Run.
5. Download the adapter and evaluate it on examples the training set never saw before deploying anything. See Deploying and Evaluating the Artifact.

A useful sanity check before any serious run: train on four or five examples with several epochs and confirm the model reproduces them exactly. If it cannot overfit a tiny set, the problem is the setup (dataset format, masking, learning rate), never the data volume.

4The chapters#

5Stated plainly#

• Everything runs locally: the dataset, the training, and the artifact never leave the machine.
• Fine-tuning is for behavior; retrieval is for knowledge; the strongest systems split the work.
• The output is a normal GGUF: everything in Choosing and Operating Models applies to it.