Table of Contents

Extract Data from Charts and Graphs with VLM OCR

Business reports, quarterly earnings, market research, and operational dashboards are filled with bar charts, line graphs, pie charts, and scatter plots that communicate critical data visually. When these arrive as screenshots, PDFs, or printed slides, extracting the underlying data points requires either manual transcription or specialized tooling. LM-Kit.NET's VlmOcr engine, combined with PaddleOCR VL's Chart Recognition: instruction, reads chart elements (axes, legends, labels, data values) and outputs a structured textual representation. This tutorial shows how to extract data from charts embedded in images and PDF reports.


Why Vision-Based Chart Recognition

Two practical advantages of PaddleOCR VL's chart mode:

  1. No access to source data required. Charts in PDFs, screenshots, and printed reports are rasterized images. The original spreadsheet or database is often unavailable, lost, or held by a third party. PaddleOCR VL reconstructs the data directly from the visual representation.
  2. Handles diverse chart types without configuration. Bar charts, grouped bars, stacked bars, line charts, pie charts, and combination charts are all recognized by the same instruction. There is no need to select a chart type or configure parsing rules for each variant.

Prerequisites

Requirement Minimum
.NET SDK 8.0+
VRAM ~1 GB (PaddleOCR VL 1.5)
Disk ~750 MB free for model download

Input formats: scanned PDF, PNG, JPEG, TIFF, BMP, WebP.


Step 1: Create the Project

dotnet new console -n ChartExtraction
cd ChartExtraction
dotnet add package LM-Kit.NET

Step 2: Extract Data from a Chart Image

Load the PaddleOCR VL model and use the Chart Recognition: instruction:

using System.Text;
using LMKit.Data;
using LMKit.Extraction.Ocr;
using LMKit.Model;

LMKit.Licensing.LicenseManager.SetLicenseKey("");

Console.InputEncoding = Encoding.UTF8;
Console.OutputEncoding = Encoding.UTF8;

// ──────────────────────────────────────
// 1. Load PaddleOCR VL model
// ──────────────────────────────────────
Console.WriteLine("Loading PaddleOCR VL model...");
using LM model = LM.LoadFromModelID("paddleocr-vl:0.9b",
    downloadingProgress: (_, len, read) =>
    {
        if (len.HasValue) Console.Write($"\r  Downloading: {(double)read / len.Value * 100:F1}%   ");
        return true;
    },
    loadingProgress: p => { Console.Write($"\r  Loading: {p * 100:F0}%   "); return true; });
Console.WriteLine("\n");

// ──────────────────────────────────────
// 2. Extract chart data
// ──────────────────────────────────────
var ocr = new VlmOcr(model, VlmOcrIntent.ChartRecognition);

var attachment = new Attachment("quarterly_revenue_chart.png");

VlmOcr.VlmOcrResult result = ocr.Run(attachment);

string chartData = result.PageElement.Text;
Console.WriteLine("Extracted chart data:");
Console.WriteLine(chartData);

File.WriteAllText("chart_data.txt", chartData);
Console.WriteLine("\nSaved to chart_data.txt");

The Chart Recognition: instruction activates PaddleOCR VL's chart understanding pipeline. The model identifies chart type, axes, legends, labels, and data values, and returns a structured textual representation.


Step 3: Extract Charts from a PDF Report

Business reports often contain multiple charts across pages. Scan each page and collect chart data:

using System.Text;
using LMKit.Data;
using LMKit.Extraction.Ocr;
using LMKit.Model;

LMKit.Licensing.LicenseManager.SetLicenseKey("");

Console.InputEncoding = Encoding.UTF8;
Console.OutputEncoding = Encoding.UTF8;

// ──────────────────────────────────────
// 1. Load PaddleOCR VL model
// ──────────────────────────────────────
Console.WriteLine("Loading PaddleOCR VL model...");
using LM model = LM.LoadFromModelID("paddleocr-vl:0.9b",
    downloadingProgress: (_, len, read) =>
    {
        if (len.HasValue) Console.Write($"\r  Downloading: {(double)read / len.Value * 100:F1}%   ");
        return true;
    },
    loadingProgress: p => { Console.Write($"\r  Loading: {p * 100:F0}%   "); return true; });
Console.WriteLine("\n");

// ──────────────────────────────────────
// 2. Multi-page chart extraction
// ──────────────────────────────────────
var ocr = new VlmOcr(model, VlmOcrIntent.ChartRecognition)
{
    MaximumCompletionTokens = 4096
};

string pdfPath = "annual_report.pdf";
var attachment = new Attachment(pdfPath);

int pageCount = attachment.PageCount;
Console.WriteLine($"Scanning {pageCount} pages for charts...\n");

var allChartData = new StringBuilder();

for (int page = 0; page < pageCount; page++)
{
    Console.Write($"  Page {page + 1}/{pageCount}... ");

    VlmOcr.VlmOcrResult pageResult = ocr.Run(attachment, pageIndex: page);
    string pageContent = pageResult.PageElement.Text;

    if (!string.IsNullOrWhiteSpace(pageContent))
    {
        allChartData.AppendLine($"--- Chart data from page {page + 1} ---");
        allChartData.AppendLine(pageContent);
        allChartData.AppendLine();
        Console.WriteLine("chart(s) found");
    }
    else
    {
        Console.WriteLine("no chart detected");
    }
}

File.WriteAllText("report_charts.txt", allChartData.ToString());
Console.WriteLine($"\nAll chart data saved to report_charts.txt");

Step 4: Full Report Extraction (Text + Tables + Charts)

For comprehensive report digitization, run three passes with different instructions:

using System.Text;
using LMKit.Data;
using LMKit.Extraction.Ocr;
using LMKit.Model;

LMKit.Licensing.LicenseManager.SetLicenseKey("");

Console.InputEncoding = Encoding.UTF8;
Console.OutputEncoding = Encoding.UTF8;

// ──────────────────────────────────────
// 1. Load PaddleOCR VL model
// ──────────────────────────────────────
Console.WriteLine("Loading PaddleOCR VL model...");
using LM model = LM.LoadFromModelID("paddleocr-vl:0.9b",
    downloadingProgress: (_, len, read) =>
    {
        if (len.HasValue) Console.Write($"\r  Downloading: {(double)read / len.Value * 100:F1}%   ");
        return true;
    },
    loadingProgress: p => { Console.Write($"\r  Loading: {p * 100:F0}%   "); return true; });
Console.WriteLine("\n");

// ──────────────────────────────────────
// 2. Three-pass extraction from a report page
// ──────────────────────────────────────
var attachment = new Attachment("dashboard_slide.png");

// Pass 1: Full text
var textOcr = new VlmOcr(model, VlmOcrIntent.PlainText);
VlmOcr.VlmOcrResult textResult = textOcr.Run(attachment);
Console.WriteLine("=== Document Text ===");
Console.WriteLine(textResult.PageElement.Text);

// Pass 2: Table extraction
var tableOcr = new VlmOcr(model, VlmOcrIntent.TableRecognition);
VlmOcr.VlmOcrResult tableResult = tableOcr.Run(attachment);
Console.WriteLine("\n=== Tables ===");
Console.WriteLine(tableResult.PageElement.Text);

// Pass 3: Chart extraction
var chartOcr = new VlmOcr(model, VlmOcrIntent.ChartRecognition);
VlmOcr.VlmOcrResult chartResult = chartOcr.Run(attachment);
Console.WriteLine("\n=== Charts ===");
Console.WriteLine(chartResult.PageElement.Text);

This multi-pass approach extracts every data component from a complex report page: narrative text, tabular data, and visual chart data.


Industry Use Cases

Industry Document Type What You Extract
Finance Earnings slides, analyst reports, fund factsheets Revenue trends, profit margins, asset allocations
Marketing Campaign reports, market share analyses, survey results Channel performance, conversion rates, demographic breakdowns
Operations KPI dashboards, utilization charts, SLA reports Throughput metrics, uptime percentages, capacity trends
Healthcare Clinical trial result charts, epidemiological graphs Treatment outcomes, dose-response curves, incidence rates
Energy Production reports, consumption forecasts, grid load charts Output volumes, demand curves, peak/off-peak patterns
Government Census visualizations, economic indicators, budget charts Population distributions, GDP growth, expenditure breakdowns

Common Issues

Problem Cause Fix
Axis labels missing from output Small font or low-resolution image Use at least 150 DPI; zoom into the chart region if possible
Approximate values instead of exact Model reads from visual position, not source data Chart recognition provides best-effort readings from pixel positions; verify critical values against source data
Multiple charts on one page merged Dense slide with overlapping chart regions Crop each chart individually before processing
Legend colors described instead of named Chart uses colors without text labels Model will describe colors when no legend text is present; add text labels to chart source when possible

Next Steps