Errors, Retries, and Jobs

The API reference documents each endpoint's shape; this page documents the CONTRACT that spans all of them: how the server fails, what each failure commits to, which failures deserve a retry, and how work that outgrows a request becomes a background job. Two principles govern everything here. First, a deterministic client fault is never a 500: if resending the same bytes must fail again, the status says so. Second, the same cause answers the same way everywhere: an oversized input reports identically from extraction, summarization, or splitting, synchronous or async.

1The job contract, precisely#

Every task endpoint (documents, PDFs, extraction, analysis, embeddings, transcription, search operations) runs under one execution contract with three modes. Chat is the exception: it streams instead, and never becomes a job.

The header is the mechanism to design around: a client that WANTS the job workflow (bulk processing, queue workers, anything that must not hold connections open) sends Prefer: respond-async on the first call and never depends on server configuration or on how long the work happens to take. A client that wants simplicity sends nothing and, on a default server, gets plain request-response.

The 202 body is minimal on purpose:

{ "job_id": "9f2c...", "status": "processing", "message": "Structured extraction in progress. Poll GET /lmkit/v1/jobs/{job_id} for status." }

2Polling: one endpoint, four terminal truths#

GET /lmkit/v1/jobs/{job_id} returns the job's full state:

DELETE /lmkit/v1/jobs/{job_id} cancels a job still processing; the job reports cancelled and, on the synchronous path, the caller receives status 499.

Three properties complete the contract:

• Jobs are private to their key. Polling and cancellation match the creating API key's owner: one key can neither observe nor cancel another's jobs. The operator sees and can cancel everything from the Admin Console's active-jobs view.
• Results are held for one hour, in memory. A completed job's result stays fetchable for an hour after completion, then is discarded; a server restart discards pending and completed jobs alike. Poll promptly, fetch the result once, and store it on your side. Treat a 404 from the poll as "unknown or expired", not as failure. On a load-balanced fleet a job id polls on the node that minted it, so route async bulk callers with affinity rather than round-robin per request (Scaling Out).
• Only real background work is a job. A request answered inline never registers as a job, so the admin's active-jobs count means detached work, not traffic.

A robust polling loop is short: poll on a modest interval (a few seconds, or scaled by progress_total when present), stop on any status other than processing, branch on error_reason when failed, and give up only on 404.

3Admission: the in-flight ceiling#

Before any work starts, synchronous or not, the server bounds how many task operations one API key may have in flight at once (MaxJobsPerOwner, default 100). At the ceiling the request is rejected immediately with 429, reason too_many_inflight_jobs, and Retry-After: 5; nothing was executed, so the retry is always safe. The slot frees when work COMPLETES, not when a 202 returns, so a bulk pipeline should submit, poll, and drain rather than fire thousands of accepts and hope.

4Failures: same cause, same answer, both channels#

Job failures and synchronous failures are classified by the same code, so a poller and a blocking caller learn identical truths. The reasons are stable, machine-readable snake_case; the important ones:

On the synchronous channel the failed body carries the same pair as { "error": "<reason>", "message": "<human text>" }. Branch your handling on the reason, never by parsing prose.

5The status map, and what each commits to#

• 400: the request itself is malformed or fails validation; field-level details are in the body. Includes unknown names that must refuse loudly (an unknown agent, for instance). Fix the request; never retry as-is.
• 401: no valid API key where one is required.
• 404: the addressed thing does not exist for you: an unknown or expired file_id or job_id. Remember jobs expire (section 2) and uploaded files expire with the configured retention.
• 413: two flavors, both actionable. An oversized REQUEST BODY is refused at the door, and the message names the remedy: upload the file first and pass its id instead of embedding base64. An oversized INPUT for inference is input_too_large above.
• 415 / PDF 4xx family: deterministic content refusals, reasons attached.
• 429: the in-flight ceiling (section 3). Retry-After is set.
• 499: the CALLER closed the connection or cancelled; recorded, never an error.
• 500: an unhandled server fault. The body is a sanitized generic message by design; the full cause is in the server logs, and the request record carries the exception summary for the operator.
• 503 + Retry-After: load shedding as policy, three known sources: every inference slot busy under the Reject saturation policy (retry after ~1s), transient device memory pressure (~5s), and a momentarily unavailable search database. All are "try again", never "your request is broken".

The envelope: errors arrive as the OpenAI-style object { "error": { "message", "type", ... } } on the OpenAI-compatible surface, and as the flat { "error": "..." } shape on Ollama-dialect routes, so existing clients of either dialect parse failures without adaptation. Task endpoints use the { error, message } pair shown above.

Streaming is its own channel: a streamed chat that fails after tokens began arrives as an error EVENT on the stream, because the HTTP status was already sent. A streaming client must handle both the status code (failures before start) and the error event (failures after).

6Retry discipline, stated once#

• Retry 503 (after Retry-After), 429 (after draining), and transport-level failures (connection reset, timeout on YOUR side), with bounded backoff.
• Never retry 4xx unchanged: they are deterministic verdicts on the request.
• Task submissions are not deduplicated: a retried submission runs the work again. Task endpoints do not mutate your inputs (an uploaded file is read, never changed), so the cost of a duplicate is compute, not corruption; still, on ambiguous transport failures after a 202, poll before resubmitting.
• Long work should not be retried into a synchronous window: force async with Prefer: respond-async and make the poll loop the single place that handles slowness.

7Stated plainly#

• Three execution modes, one header: Prefer: respond-async gives any client the job workflow on the first call, whatever the server's timeout setting.
• A job's result is byte-for-byte the synchronous response; failures carry an HTTP-equivalent code plus a stable machine reason on both channels.
• Results live one hour in memory and jobs are key-private: poll promptly, store the result yourself, and branch on reasons, not prose.
• 5xx means retry, deterministic 4xx means fix, and Retry-After is always honored by the well-behaved client you are writing.