An Archive That Answers

A folder of documents answers nothing; an indexed corpus answers with citations. This recipe stands up a tenant on the built-in store, indexes documents WITH the metadata that later becomes filters, and serves the two consumption shapes: ranked search results for a UI, and grounded answers for questions. Cookbook conventions apply.

1Provision a tenant: the quality decisions live here#

A tenant is an isolated workspace whose settings govern every collection inside it (the architecture). On the built-in store this is one call against the default cluster; the server generates the id:

curl -s "$LMKIT_URL/lmkit/v1/search/tenants" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "display_name": "Legal archive",
    "enable_full_text_search": true,
    "enable_semantic_search": true,
    "enable_ocr": true
  }'

Hold the returned tenant id as $TENANT. Both search modes stay on so hybrid retrieval is available; OCR on means scans index like born-digital files. Note the access rule if keys are in play: new API keys hold NO search clusters until granted.

2Index with the metadata you will need later#

One call per document; collections are just named buckets inside the tenant (contracts here). The high-leverage habit: decide custom_metadata NOW, because filters and facets run on it and retrofitting metadata means re-indexing (Indexing Well).

curl -s "$LMKIT_URL/lmkit/v1/search/documents" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "tenant_id": "'$TENANT'",
    "collection_id": "contracts",
    "input": "'$FILE_ID'",
    "input_format": "FileIdentifier",
    "name": "MSA Acme Fabric 2024.pdf",
    "source_uri": "https://dms.internal/contracts/8812",
    "custom_metadata": { "doc_type": "contract", "counterparty": "Acme Fabric", "year": 2024 }
  }'

Ingestion is idempotent by content hash (re-running a corpus sync only pays for real changes), and embedding runs in the background; pass wait_for_embedding: true when the next step must see the document semantically. Bulk loads are the textbook case for Prefer: respond-async plus a polling worker (the jobs contract), and the classifier from the IDP recipe is the natural source of doc_type.

3Search: three modes, one endpoint#

POST /lmkit/v1/search/search serves the ranked-results shape (Querying and Relevance covers every lever):

curl -s "$LMKIT_URL/lmkit/v1/search/search" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "tenant_id": "'$TENANT'",
    "collection_id": "contracts",
    "query": "termination notice period",
    "search_type": "Hybrid",
    "top_k": 8,
    "rerank": true
  }'

The response carries ranked results with scores and locations, plus facets and totals for building the filter UI, and rerank_applied so you know what actually ran. Full-text answers "where is this exact term", semantic answers "what talks about this idea", hybrid is the default choice for archives, and the reranker buys precision at the top of the list where readers actually look.

4From search results to answers: that is RAG#

The same corpus serves questions, not just result lists: one-call grounded answers with citations, multi-turn conversational sessions, and the parts for building your own, all with the honesty contract that refuses when the corpus cannot support an answer. That whole consumption side is the RAG recipe; this recipe's source_uri and metadata are exactly what its citations and filters run on.

5Prove the quality, then keep it#

Retrieval quality is measurable here without building anything: golden query sets and persisted evaluation runs are part of the Search API. Write fifteen real questions with their expected source documents, run the evaluation, and keep the run ids: after every tenant-setting change or model swap, re-run and compare. This is Measuring What Matters with the harness already built in.

6Production notes#

• Metadata is the product. doc_type, dates, counterparties: whatever your users will filter by must be in custom_metadata at index time; classification and keyword extraction generate it mechanically.
• Retention can be per document. ttl_seconds at index time makes expiring corpora (quotes, postings) self-cleaning.
• Scope keys per audience. A key for the intranet UI gets the archive's cluster granted and nothing else; the assistant from the support recipe can query the same tenant through its own key.