RAG, End to End

Retrieval-augmented generation is the pattern behind most production LLM systems: instead of trusting the model's memory, RETRIEVE the relevant passages from your own corpus and let the model answer FROM them, with citations. It is also the workload where local-first stops being a preference and becomes the requirement, because the corpus that makes RAG useful (contracts, tickets, records, knowledge bases) is exactly the data most organizations cannot ship to a third-party API. This server implements the entire pattern behind one API, in three consumption shapes on the same indexed corpus: a one-call answer, a stateful conversation, and the raw parts for building your own. This recipe runs all three; cookbook conventions apply.

1The corpus, in two calls#

RAG quality is decided at indexing time, so the corpus comes first: provision a tenant, then index each document with the metadata you will later filter by. The calls and the craft are the archive recipe's first two steps, and Indexing Well is the depth. The short version:

curl -s "$LMKIT_URL/lmkit/v1/search/tenants" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "display_name": "Knowledge base", "enable_semantic_search": true, "enable_full_text_search": true, "enable_ocr": true }'

Index with custom_metadata per document (owner, product, year: whatever answers must be scoped by later), and let content-hash idempotency make your corpus sync a re-runnable job. Everything below assumes $TENANT and a docs collection.

2Shape one: RAG as one call#

POST /lmkit/v1/search/answer is the whole pipeline (retrieve, rerank, generate, cite, verify) behind one request, with the honesty contract that makes it deployable (Grounded Answers):

curl -s "$LMKIT_URL/lmkit/v1/search/answer" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "tenant_id": "'$TENANT'", "collection_id": "docs",
        "query": "How do we rotate the signing certificate?" }'

{
  "answer": "Rotation is a two-step swap: stage the new certificate, then... [1][2]",
  "context_found": true,
  "citations": [ { "source_number": 1, "document_name": "ops-runbook.pdf", "source_uri": "https://kb.internal/runbook" } ],
  "groundedness_score": 0.95,
  "unsupported_claims": []
}

Branch on the contract: context_found: false renders as "not in the knowledge base" (refusal is a feature, not a failure), citations link every claim to its source, and a weak groundedness_score or non-empty unsupported_claims demotes the answer below its sources in your UI.

3Shape two: conversational RAG#

Real usage is a dialogue ("and on Linux?", "what about the old version?"), and follow-ups only retrieve well when the question is resolved against the conversation. POST /lmkit/v1/search/chat holds that state server-side:

curl -s "$LMKIT_URL/lmkit/v1/search/chat" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "tenant_id": "'$TENANT'", "collection_id": "docs",
        "session_id": "user-731-kb",
        "question": "And how often should we rotate it?" }'

{
  "session_id": "user-731-kb",
  "answer": "Every ninety days, per the runbook's schedule [1].",
  "context_found": true,
  "citations": [ ... ],
  "effective_query": "How often should the signing certificate be rotated?"
}

effective_query shows the follow-up REWRITTEN into the standalone question retrieval actually ran on: the mechanics that make multi-turn RAG work, visible so you can debug relevance instead of guessing. Sessions are keyed by YOUR session_id, expire when idle, stream over SSE with "stream": true, and idle session state tiers out of device memory transparently, so holding thousands of open conversations is cheap.

4Shape three: build your own#

When you own the prompt (an existing chat app, a bespoke agent, special formatting), take retrieval as parts and do the generation yourself. POST /lmkit/v1/search/search returns context-ready passages; feed them to any chat endpoint, including this server's OpenAI-compatible one:

curl -s "$LMKIT_URL/lmkit/v1/search/search" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "tenant_id": "'$TENANT'", "collection_id": "docs",
        "query": "rotate signing certificate", "search_type": "Hybrid",
        "top_k": 6, "rerank": true, "context_expansion": true }'

Then assemble and generate:

curl -s "$LMKIT_URL/v1/chat/completions" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "messages": [
        { "role": "system", "content": "Answer ONLY from the numbered sources. Cite as [n]. If the sources do not contain the answer, say so." },
        { "role": "user", "content": "SOURCES:\n[1] ...retrieved passage...\n[2] ...retrieved passage...\n\nQUESTION: How do we rotate the signing certificate?" }
      ] }'

The trade is explicit: you gain full control of the prompt, the model choice per call, and the message shape your app already uses; you take over what the managed shapes did for you (citation discipline, the refusal rule, groundedness checking). Most teams ship shape one or two and reach for shape three where the product demands it.

5Choosing the shape#

6Quality is measurable, and it is usually retrieval#

When a RAG answer disappoints, the failure is usually retrieval, not generation: the right passage never reached the model. That is why the harness ships in the box: golden query sets and persisted evaluation runs score retrieval directly, so you fix indexing and metadata or query-side levers with evidence, then re-run and compare. The method chapter is Measuring What Matters.

7Production notes#

• Scope answers per audience. Filters over custom_metadata, collections per corpus, and per-key cluster grants compose into "users only get answers from documents they may see", enforced at retrieval, where it belongs.
• Force async for corpus loads, not for queries: indexing is the jobs contract's natural customer; answers are interactive.
• The privacy story is the architecture. Corpus, embeddings, retrieval, and generation all run on this machine (Where Data Lives): RAG over confidential data without a single byte leaving is the point of running local.