Chat with Documents over the API

POST /lmkit/v1/chat-with-document turns one document into a conversational Q&A session: load it once, then ask questions across turns, with answers grounded in the document and page-level citations attached. This is the API behind "chat with your PDF", for developers who want a working call in five minutes, before any indexing, tenants, or collections.

1One endpoint, two kinds of call#

The endpoint distinguishes the two calls by one field. No session_id: the request carries a document, the server loads it, and the response returns a new session_id. With session_id: the request carries a question, and the answer continues that session's conversation. A question on the first call is allowed too: the document loads and the question is answered in the same response.

The first call, document and question together:

curl -s "$LMKIT_URL/lmkit/v1/chat-with-document" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{
    "input": "'"$(base64 -w0 contract.pdf)"'",
    "input_format": "Base64EncodedFile",
    "question": "What is the termination notice period?"
  }'

{
  "session_id": "b41f0c2a9e764d5c8a31f7de20c48b17",
  "response": "The agreement requires ninety days of written notice before termination...",
  "source_references": [
    { "document_name": "document", "page_number": 12,
      "excerpt": "Either party may terminate this Agreement upon ninety (90) days...",
      "similarity_score": 0.83 }
  ],
  "document_name": "document",
  "indexing_info": { "indexing_mode": "PassageRetrieval", "page_count": 48,
                     "token_count": 30514, "exceeded_token_budget": true }
}

A follow-up turn is one field plus the question; the conversation context is maintained server-side, so "and what about..." questions resolve correctly:

curl -s "$LMKIT_URL/lmkit/v1/chat-with-document" \
  -H "Authorization: Bearer $LMKIT_API_KEY" -H "Content-Type: application/json" \
  -d '{ "session_id": "b41f0c2a9e764d5c8a31f7de20c48b17",
        "question": "What penalties apply if we terminate early?" }'

The remaining request fields, all optional:

Any format the document pipeline ingests works: PDF, the Office family, HTML, email, images. Scanned pages and image documents are read by the server's configured OCR engine automatically (OCR section); there is no separate OCR step. A base64 body carries no filename, so document_name in the response and its citations falls back to document; upload via POST /lmkit/v1/files/upload and pass the FileIdentifier to keep the real name. Large files should go through the file upload endpoint anyway, since an oversized base64 body is refused at the door with a 413 that names exactly that remedy. Models come from the catalog in Models.

2Citations and scores#

source_references lists the document passages the answer was generated from: the page number (1-based), the text excerpt, and a similarity score between 0.0 and 1.0 measuring how strongly the passage matched the question. This is what makes the endpoint deployable: a UI can render "where does it say that" as a link to page 12, and a low-scoring reference set is a signal the document may not contain the answer at all.

indexing_info appears only on the first response, when the document is loaded: the page count, the estimated token count, and the indexing mode, which decides whether citations exist at all (next section).

3Full document or passage retrieval#

At load time the server makes one decision, reported as indexing_mode:

• FullDocument: the document's full text fits a token budget (4,096 tokens), so it rides in the model's context whole. Every answer sees the entire document, which is the highest-fidelity mode for small documents, but there is nothing to retrieve, so source_references comes back empty. An empty citation list on a small document is this mode, not a failure.
• PassageRetrieval: the document exceeds the budget (exceeded_token_budget: true), so it is chunked, and each question retrieves the most relevant passages before answering. This is where citations and similarity scores come from.

The decision is automatic and per-session. If your product's contract is "every answer cites its pages", note that small documents will not produce citations on this endpoint; the Playground's document workbench forces retrieval for exactly that reason, and the search stack's grounded answers make citations unconditional.

4Streaming#

Set "stream": true on a question against an existing session and the response arrives as text/event-stream. Each event is a data: {json} line carrying a chunk:

data: {"session_id":"b41f...","delta":"The","thinking":false,"done":false}
data: {"session_id":"b41f...","delta":" agreement","thinking":false,"done":false}
...
data: {"session_id":"b41f...","delta":"","thinking":false,"done":true,"response":"The agreement requires...","source_references":[...]}
data: [DONE]

• delta is the incremental text; thinking: true marks tokens that belong to a reasoning model's internal chain-of-thought, so a UI can fold them away from the answer.
• The final chunk has done: true and carries the complete answer (thinking excluded) plus the source_references; the stream then closes with the data: [DONE] sentinel.
• Streaming applies to question turns. The first call, the one that loads the document, answers with a plain JSON response (or a job, section 6) regardless of the flag.

When a reasoning model is mid-deliberation and the user wants the answer now, POST /lmkit/v1/chat-with-document/{session_id}/skip-thinking signals the model to end its reasoning phase and start generating the visible answer; it is a no-op when the model is not currently thinking.

5Session lifecycle#

A session holds the loaded document, its index, and the conversation history, in memory on the server.

• Private to the key. A session is only visible to the API key owner that created it; another key asking for it gets the same 404 as for a session that never existed.
• One question at a time. A second question while one is generating waits briefly, then returns 429; serialize your turns per session.
• Idle expiry. A session unused for thirty minutes is expired, and the server caps concurrent sessions, evicting the oldest idle one at the ceiling. Treat a 404 on a known session_id as "expired": reload the document and continue.
• Reset without reloading. POST /lmkit/v1/chat-with-document/{session_id}/clear-history clears the conversation while keeping the loaded document, so a fresh line of questioning skips the loading cost.
• Delete when done. DELETE /lmkit/v1/chat-with-document/{session_id} releases the session and its memory immediately. Well-behaved clients delete rather than letting expiry collect.

Sessions do not survive a server restart, and they are not a document store: nothing about them persists.

6When loading takes long#

Loading a large or scanned document is real work (OCR per page, indexing), so the first call runs under the server's standard job contract: if loading outruns the configured synchronous window, the call returns 202 Accepted with a job_id, loading continues in the background with live page progress, and GET /lmkit/v1/jobs/{job_id} returns the same ChatWithDocumentResponse, session id included, once done. A client that prefers the job workflow from the start sends Prefer: respond-async on the first call. Question turns never become jobs; they answer or stream directly.

The failure answers are the standard ones: 415 for bytes no document pipeline ingests, 400 for a request that fails validation (a first call without input, a turn without question), and a failed load reports through the job's error and error_reason fields.

7When to graduate to the search stack#

This endpoint is deliberately scoped: one document, one session, in memory, gone on expiry. That is the right shape for "let the user interrogate the file they uploaded". It is the wrong shape the moment any of these appear:

The mental model: chat-with-document is RAG over one file with zero setup; the search stack is RAG over a corpus with an indexing step that pays for itself the second document in.

8Stated plainly#

• One endpoint, two calls: no session_id loads a document and returns one; with session_id, each call is a conversation turn.
• Citations exist in PassageRetrieval mode; a small document rides the context whole and returns an empty source_references list by design.
• Sessions are key-private, in-memory, serialized per question, and expire after thirty idle minutes; delete them when done.
• Slow loads become jobs under the standard contract; question turns answer or stream, never queue as jobs.
• One document per session is the boundary: a corpus belongs in the search stack.