Deploying and Evaluating the Artifact

A finished run produces a standard GGUF artifact, downloadable from the workbench or GET /lmkit/v1/training/jobs/{id}/artifact. This guide covers the two artifact shapes, how to serve them, and the evaluation discipline that decides whether the artifact ships.

1Adapter or merged model#

The output choice at run time:

The working pattern: iterate on ADAPTERS while you experiment (small, fast to produce, easy to diff by run), and produce a MERGED model when a candidate is ready to be treated as a deployable unit. The GGUF format is the same either way, so every consumer that loads models loads these.

2Merge quantization#

A merged model can be re-quantized in the same step (merge_quantization: q4_k_m, q5_k_m, or q8_0; empty keeps the base's precision). Guidance:

• Trained over the quantized base you serve (the common case): keep the merge at base precision. The artifact is a drop-in replacement for the model you already run.
• Trained with the full_precision option (see Memory, Speed, and Hardware): merge at full precision first if you want an archival master, and quantize to the serving precision in the same run for the deployable copy. Quantize once, from the best source you have; never re-quantize a quantized merge.

3Serving the result#

• In this server: register the artifact in Models like any custom model. From that moment every dialect serves it: OpenAI-compatible, Anthropic-compatible, and Ollama clients all reach it by its model id (see API Compatibility), and it can take the default-model slot so existing clients switch without a config change.
• Anywhere else: the artifact is a normal GGUF file; copy it and load it. An adapter travels with the requirement that its exact base is present; a merged model travels alone.

Everything in Choosing and Operating Models applies to the artifact, because it IS a model.

4Prove it before it ships#

The evaluation set is the examples you held out entirely in Preparing a Training Dataset: inputs the run never saw, with the outputs you expect. The discipline:

1. Grade exact behaviors exactly. For format tasks (JSON, labels, fixed phrasings), score exact match, not vibes. Ten sharp held-out cases beat a hundred eyeballed ones.
2. Compare against the unmodified base on the same set, same prompts, same settings. The fine-tune must beat it by a margin that justifies operating a custom model.
3. Check what it must NOT have lost. Run a handful of general prompts (chat quality, refusals, out-of-scope handling). Aggressive small-dataset runs can trade general behavior for the task; if that shows, revisit epochs and early stopping in Reading a Training Run.
4. A/B interactively in the playground: load base and candidate, replay the same conversations side by side. Numbers decide, but reading real transcripts catches failure shapes no metric was watching.

5Reproducibility and versioning#

Every finished job journals its full request (parameters, dataset reference, seed), which is exactly the recipe for the artifact. Keep the trio under version control: dataset version, run request, artifact. Name artifacts by dataset version and run intent (support-router-ds3-r16.gguf says more than final-v2.gguf), pin the seed for comparison runs, and re-run the evaluation set whenever any of the three changes. That is the whole lifecycle: with it, any artifact can be rebuilt, explained, and defended months later.

6Stated plainly#

• Iterate on adapters, ship merged models, quantize exactly once from the best source.
• The artifact must beat the unmodified base on held-out exact-match cases AND keep the general behavior it had; otherwise it does not ship.
• The journaled request plus a versioned dataset is the artifact's recipe: keep both, and every deployment stays reproducible.