Table of Contents

Parse Documents into Structured Elements

DocumentParser turns a document into what a downstream system actually needs: every page as a list of elements in reading order, each with its role (title, text, table, formula, figure, caption, header, footer), its position on the page, and its typed content. Tables come back as HTML, formulas as LaTeX, and charts as the data table they plot. The same result renders as clean Markdown for LLM ingestion, RAG, or archiving.

The API is deliberately small: you choose an effort level, and optionally hand over the models you already hold. Everything else is the engine's decision. Everything runs on-device.


Prerequisites

Requirement Minimum
.NET SDK 8.0+
Disk About 2.5 GB free for the model download on first run
GPU Recommended (NVIDIA, AMD or Intel, or the integrated GPU of an Apple silicon Mac); on the CPU alone the parser reads at a lower resolution, see Step 4

The models are downloaded once, on first use, into Configuration.ModelStorageDirectory, and reused from there.


Step 1: Create the Project

dotnet new console -n ParseQuickstart
cd ParseQuickstart
dotnet add package LM-Kit.NET

Step 2: Parse a Document

using System.Text;
using LMKit.Document.Parsing;

Console.OutputEncoding = Encoding.UTF8;

using var parser = new DocumentParser(ParsingEffort.Medium);

ParsedDocument document = parser.Parse("report.pdf");

File.WriteAllText("report.md", document.ToMarkdown());

Console.WriteLine($"Pages   : {document.Pages.Count}");
Console.WriteLine($"Elapsed : {document.Elapsed.TotalSeconds:F1} s");

Parse accepts a file path or an Attachment (a PDF, an image, or any format Attachment opens). The parser loads its models when it is created and releases them when it is disposed, so create one parser and reuse it for every document.


Step 3: Choose the Effort Level

The effort level is the one decision you make: the trade-off between the time a document takes and the fidelity of the result.

Level Choose it for
ParsingEffort.Minimal Indexing at volume on any machine: no model, the cost of text extraction
ParsingEffort.Low High-volume ingestion where throughput matters most
ParsingEffort.Medium Most documents: a balance of time and fidelity (the default)
ParsingEffort.High Documents where every table, chart and heading counts

What the engine does at each level belongs to the engine and improves from one release to the next; your code does not change. A higher level spends more time on a typical document.

using var fast = new DocumentParser(ParsingEffort.Low);
using var thorough = new DocumentParser(ParsingEffort.High);

ParsingEffort.Minimal loads no model at all (DocumentParser.GetModelCards lists none for it): each page is read from the text the document carries, scanned pages and pictures through DocumentParser.OcrEngine, and Word documents, HTML pages and e-mails from their own structure. The result is the same ParsedDocument, so a pipeline can start at Minimal and parse the same documents again at a higher level later without changing its code.

using var minimal = new DocumentParser(ParsingEffort.Minimal)
{
    OcrLanguages = new[] { Language.French }   // a hint for scanned pages; null lets the engine decide
};
ParsedDocument document = minimal.Parse("contract.pdf");

Step 4: Parse on a Machine Without a GPU

The parser reads every page with a vision model. The model first turns the page image into patches and relates every patch to every other one, so its work grows faster than the number of pixels. The effort levels read pages at a document resolution, a few million pixels, and more for the pages that get a second look. On a GPU that takes seconds per page. On a processor alone it takes minutes.

So the parser adapts. When its page model runs its vision encoder on the CPU, it reads every page, figure and second reading at most at ImageDetail.Low (about one million pixels). A page is then read several times faster. A second reading that would ask for a larger image is skipped, since the ceiling would give it the same one. On a GPU nothing is lowered.

Where the vision encoder runs Resolution with the default MaximumImageDetail = null
A GPU through CUDA or Vulkan The effort level's own, unchanged
The integrated GPU of an Apple silicon Mac, through Metal The effort level's own, unchanged
The CPU (no GPU, or a model loaded with no GPU layers) At most ImageDetail.Low

The trade-off is the finest print. Small text, dense tables and tightly packed chart labels can lose detail at the lower resolution. Set MaximumImageDetail to decide for yourself; a stated value applies on every device:

using LMKit.Inference.Vision;

// On the CPU: read at the full resolution anyway, and accept the time it takes.
using var faithful = new DocumentParser(ParsingEffort.Medium) { MaximumImageDetail = ImageDetail.Maximal };

// On a GPU: trade detail for speed on a large batch of simple documents.
using var quick = new DocumentParser(ParsingEffort.Medium) { MaximumImageDetail = ImageDetail.Standard };

ImageDetail.Maximal lowers nothing. The setting is read when a parse starts, so it can change between two parses on the same parser.

Even at the lower resolution, the page model still writes out every page it reads, one token at a time. That is what remains on a processor alone: a minute or more per page on a small machine, depending on how much text the page holds. For large volumes, run the parser on a GPU.


Step 5: Read the Elements

Each page lists its elements in reading order. The concrete type of ParsedElement.Content says how its text is encoded.

Content type Content.Text holds
TextContent Running text, headings, captions, footnotes, as Markdown
TableContent The table as HTML
FormulaContent The formula as LaTeX, without delimiters
ChartContent The plotted data as a Markdown table, with Legend, Caption and one table per panel in Panels
FigureContent A picture; an optional description
FigureCaptionContent, FigureLabelContent, RepeatedTextContent Text that belongs to a figure or repeats another element; kept for layout views, left out of the page rendering 
using LMKit.Document.Layout;
using LMKit.Document.Parsing;

foreach (ParsedPage page in document.Pages)
{
    Console.WriteLine($"--- Page {page.PageNumber} ({page.Width} x {page.Height} {page.Unit})");

    foreach (ParsedElement element in page.Body)
    {
        switch (element.Content)
        {
            case TableContent table:
                Console.WriteLine($"Table at {element.Bounds}: {table.Text.Length} chars of HTML");
                break;

            case ChartContent chart:
                Console.WriteLine($"Chart '{chart.Caption}':\n{chart.Text}");
                break;

            case FormulaContent formula:
                Console.WriteLine($"Formula: {formula.Text}");
                break;

            default:
                if (element.Category == LayoutElementCategory.Title)
                {
                    Console.WriteLine($"# {element.Content.Text}");
                }
                break;
        }
    }
}

Positions (ParsedElement.Bounds) are in the page's own frame (ParsedPage.Unit: points for a document page, pixels for an image), origin at the top-left corner. page.Headers and page.Footers hold the running heads and feet, which the Markdown rendering leaves out. ParsedElement.ToMarkdown() renders one element exactly as it reads in the page, which is convenient for chunking.

Two optional layers describe the page's structure beyond its elements: ParsedPage.Blocks (elements the page sets as one group, such as the entries of a list or a label with its value) and ParsedPage.FigureTexts (the labels, legend entries and callouts a figure prints). Both may be empty, depending on the page and the effort level.


Step 6: Export to JSON, HTML and DocLang

A parse renders to the standard formats document pipelines exchange. Every rendering reads the same parse, so a heading, a table or a caption is the same in all of them.

Method Format Use it for
ToMarkdown() Markdown, tables as HTML, formulas as LaTeX LLM prompts, RAG chunks
ToJson() Lossless JSON: every element with its category, bounds, reading index, confidence and typed content Storage, pipelines, any language
ToHtml() Semantic HTML: headings, tables with their spans, figures with captions, each element carrying data-category and data-bbox Display, review, highlighting a source region
ToDocLang() DocLang 0.7, the open AI-native document markup of the LF AI and Data Foundation Exchange with DocLang tools and models 
File.WriteAllText("report.json", document.ToJson());
File.WriteAllText("report.html", document.ToHtml());
File.WriteAllText("report.dclg", document.ToDocLang());

ParsedPage has the same methods for one page; ParsedElement has ToMarkdown(), ToJson() and ToHtml() for one element.

The JSON is a contract: it follows a versioned schema (named by its $schema member), uses snake_case names, and reads back with ParsedDocument.FromJson. Parse once, store the JSON, and render or export it again later without the source document or the models:

ParsedDocument stored = ParsedDocument.FromJson(File.ReadAllText("report.json"));
string markdown = stored.ToMarkdown(); // identical to the original parse

The HTML is safe to display: a document's text is untrusted input, so any markup it carries is reduced to structural tags, and scripts, styles, event handlers and unsafe links are removed. Formulas are written as delimited LaTeX, which MathJax or KaTeX render as they stand.


Step 7: Keep the Figure Images

Set IncludeFigureImages to keep the picture of every figure and chart, cut from the page as it renders. ParsedElement.GetImage() returns it, and every export carries it.

using var parser = new DocumentParser(ParsingEffort.Medium) { IncludeFigureImages = true };
ParsedDocument document = parser.Parse("report.pdf");

foreach (ParsedPage page in document.Pages)
{
    foreach (ParsedElement figure in page.Elements)
    {
        using ImageBuffer picture = figure.GetImage();
        picture?.SaveAsPng($"figure-p{page.PageNumber}-{figure.ReadingIndex}.png");
    }
}

document.SaveHtml("report.html");            // report.html + report_files/ with the pictures
document.SaveDocLangArchive("report.dclx");  // one DocLang archive: markup and pictures

ToJson(), ToHtml() and ToDocLang() embed the pictures as data URIs. SaveHtml and SaveDocLang write them as files in a folder beside the page, and SaveDocLangArchive writes the DocLang package format (.dclx), which holds the markup and its pictures in one file. The pictures are held encoded as PNG and cost one page render per page that holds a figure; the setting is off by default.


Step 8: Ship the Model With Your Application

The parser runs on one model, the document parser bundle: a single archive holding everything it reads with. DocumentParser.GetModelCards(effort) lists it as a model catalog entry, and the same model serves every effort level. Created without a path, the parser loads it from the catalog, downloading it on first use.

To ship the file with your application instead of downloading it, pass its path. The parser checks the file, loads it, and releases it when disposed.

using var parser = new DocumentParser(ParsingEffort.Medium, @"models\document-parser.lmk");

A file that is not the bundle is refused with an InvalidModelException before anything is parsed.

To prepare a machine that parses offline, download the model ahead of time:

DocumentParser.GetModelCards(ParsingEffort.High).Single().Download();

Step 9: Select Pages, Report Progress, Parse in Parallel

Pass a page range to parse part of a document. The syntax is the SDK's own (PageRange): 1-based pages, ranges, open ends and lists, such as "3", "1-5", "5-" or "1-3, 7, 10-12".

ParsedDocument summary = parser.Parse("annual-report.pdf", "1-3");

PageParsed is raised each time a page has been read, with the page number and the progress of the parse:

parser.PageParsed += (_, e) =>
    Console.WriteLine($"{e.SourceName}: page {e.PageNumber} ({e.PagesParsed}/{e.PageCount}) in {e.Elapsed.TotalSeconds:F1} s");

ParseAsync has the same overloads as Parse, and every overload takes a CancellationToken that stops the parse between pages. One parser serves several documents at once from several threads.

using var parser = new DocumentParser(ParsingEffort.Medium);
using var cancellation = new CancellationTokenSource(TimeSpan.FromMinutes(10));

string[] files = Directory.GetFiles("inbox", "*.pdf");

ParsedDocument[] documents = await Task.WhenAll(
    files.Select(file => parser.ParseAsync(file, cancellation.Token)));

for (int i = 0; i < files.Length; i++)
{
    File.WriteAllText(Path.ChangeExtension(files[i], ".md"), documents[i].ToMarkdown());
}

Common Issues

Problem Cause Fix
The first parser takes minutes to create The model is downloading Download it ahead of time with GetModelCards(effort) and ModelCard.Download()
InvalidModelException when creating the parser The file is not the model the parser supports Pass the file GetModelCards lists, or none
ArgumentException when creating the parser Two models or files of the same kind were passed Pass each model once
Parsing is slow on a machine without a GPU The models run on the CPU The parser already lowers the resolution there (Step 4); for large volumes, run on a machine with a GPU
Fine print or dense tables come out incomplete on a machine without a GPU The CPU default reads at ImageDetail.Low Set MaximumImageDetail = ImageDetail.Maximal for those documents

DocumentParser or DocumentToMarkdown?

LM-Kit.NET offers two engines that turn documents into text, for different jobs.

DocumentParser DocumentToMarkdown
Result Typed elements with positions, reading order and confidence, in Markdown, JSON, HTML and DocLang Markdown per page
Inputs PDFs, scans, images Every format Attachment opens, including DOCX, XLSX, PPTX, HTML and email
Charts Read as data tables Transcribed as the model reads them
Configuration One effort level; the models are chosen and verified for you A strategy (text layer, vision model, hybrid) and your own models

Choose DocumentParser for the most faithful structure of PDFs and scans. Choose DocumentToMarkdown for fast Markdown across office and email formats, or to run a vision model of your own choice.


Next Steps

Share