Text Analysis
Ten endpoints under /lmkit/v1 answer questions ABOUT content: what it is, what it names, what
it reveals, how it reads. Two facts make them more useful than their one-line summaries
suggest: nearly all of them accept whole DOCUMENTS (PDF, HTML, EML, MBOX, Office, images with
OCR), not just strings, and the classification and entity endpoints are shaped by YOUR labels
at request time, so "custom NLP" here means writing a request, not training a model.
1The toolbox#
| Endpoint | Answers | The lever that makes it yours |
|---|---|---|
categorize |
Which of YOUR categories fits | Your taxonomy at request time; the full treatment is Document Classification |
ner |
Which entities are named | built_in_types selection plus custom_labels for domain entities |
pii-detection |
What personal data is present | Same shape: built-in types plus custom labels |
sentiment-analysis |
Tone: sentiment, emotion, sarcasm | Each dimension toggles independently, with neutral support |
keyword-extraction |
The terms that carry the content | Feeds tagging and lightweight retrieval |
language-detection |
What language this is | The router in front of translation and language-specific handling |
translation |
The content in another language | Whole documents, structure preserved |
summarization |
The short version | Length budgets, optional title generation, intent, and target_language for cross-lingual summaries |
text-correction |
Grammar and spelling fixed | Correction without rewriting |
text-rewriter |
The same content, said differently | Tone and style transformation |
Exact request shapes live in the API reference; models resolve like everywhere else
(the request's model, or the default slot for the capability).
2The two request patterns worth knowing#
Documents in, not strings. An invoice PDF, an email thread, or a screenshot goes into
categorization or pii-detection directly: the document pipeline
(extraction plus OCR) runs inside the endpoint. You only pre-convert when you want to reuse the
extracted text across several calls, in which case upload once and pass the file id everywhere.
Labels at request time. NER and PII detection take custom_labels beside the built-in
types: contract clause types, product codes, domain-specific identifiers, declared per
request, plus guidance for interpretation rules. Classification follows the same
philosophy with described categories and an honest open-set switch; as the router of the
IDP pipeline it has its own chapter.
3Pipelines these endpoints form#
- Inbox triage:
language-detectionroutes, classification files,nerfills the ticket fields,summarizationwrites the handler's brief. Four calls per message, all local. - Compliance sweep:
pii-detectionover a document folder, human review of the findings, then true redaction with the confirmed strings. Detection returns what and where; redaction destroys it. - Cross-language corpus:
language-detection, thensummarizationwithtarget_language, so a mixed-language archive reads in one language without a separate translation pass. - Search enrichment: classification and
keyword-extractionat ingest produce the metadata that search filters and facets run on. Decide metadata before bulk-indexing; retrofitting it means re-indexing.
4Choosing models for analysis#
Analysis tasks are the natural home of SMALL models: classification against described categories, entity tagging, and language detection are constrained tasks where a compact model is fast and reliably on-format, and throughput usually matters because these endpoints run per message or per document at volume. Start small, measure against a labeled sample of your own traffic, and escalate only where the small model measurably confuses your actual categories. For a category system with real volume, the escalation path is fine-tuning a compact model on your corrected outputs rather than renting a bigger generic one.
5Stated plainly#
- Every analysis endpoint reads whole documents through the same pipeline as everything else; OCR and format handling are not your problem.
- Custom taxonomies are request parameters, and the label DESCRIPTIONS are where the accuracy lives.
- These endpoints compose: triage, compliance-and-redact, cross-language reading, and search enrichment are each a few calls on one local server.