Table of Contents

Enforce Structured Output with Grammar-Constrained Decoding

Language models generate free-form text by default. When you need the output to be valid JSON, a strict boolean, or a schema-compliant data structure, free-form generation introduces parsing failures, malformed fields, and silent data corruption. LM-Kit.NET's Grammar class applies grammar-constrained decoding at the token level: the model can only emit tokens that produce structurally valid output. Every generated character is guaranteed to conform to the grammar you specify. This guide walks through predefined grammars, custom GBNF rules, JSON schema enforcement, and integrating grammar constraints into agent workflows.


Why Grammar-Constrained Decoding Matters

Two production problems that grammar constraints solve:

  1. Eliminating JSON parsing failures in data pipelines. When an LLM generates "almost valid" JSON (trailing commas, unquoted keys, missing brackets), downstream parsers break silently or throw exceptions. Grammar constraints make malformed output structurally impossible. Every response deserializes on the first attempt, every time.
  2. Enforcing schema compliance for API contracts. When your application expects a response with specific field names and types (strings, numbers, arrays), the model might hallucinate extra fields or skip required ones. A JSON schema grammar restricts output to exactly the fields you define, turning the LLM into a reliable structured data generator.

Prerequisites

Requirement Minimum
.NET SDK 8.0+
VRAM 4+ GB
Disk ~3 GB free for model download

Step 1: Create the Project

dotnet new console -n GrammarConstrainedOutput
cd GrammarConstrainedOutput
dotnet add package LM-Kit.NET

Step 2: Use Predefined Grammars for Common Formats

LM-Kit.NET ships with built-in grammars for the most common structured formats. These cover JSON objects, JSON arrays, boolean responses, and more:

using System.Text;
using System.Text.Json;
using LMKit.Model;
using LMKit.TextGeneration;
using LMKit.TextGeneration.Sampling;

LMKit.Licensing.LicenseManager.SetLicenseKey("");

Console.InputEncoding = Encoding.UTF8;
Console.OutputEncoding = Encoding.UTF8;

// ──────────────────────────────────────
// 1. Load model
// ──────────────────────────────────────
Console.WriteLine("Loading model...");
using LM model = LM.LoadFromModelID("qwen3:4b",
    loadingProgress: p => { Console.Write($"\rLoading: {p * 100:F0}%   "); return true; });
Console.WriteLine("\n");

// ──────────────────────────────────────
// 2. JSON object grammar
// ──────────────────────────────────────
var chat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 512,
    Grammar = new Grammar(Grammar.PredefinedGrammar.Json)
};

string jsonResult = chat.Submit(
    "Extract the following into a JSON object: " +
    "John Smith, age 34, works at Contoso Ltd as a Senior Engineer.");

Console.WriteLine("── JSON Object ──");
Console.WriteLine(jsonResult);

// Parse it immediately: grammar guarantees valid JSON
var doc = JsonDocument.Parse(jsonResult);
Console.WriteLine($"Parsed successfully: {doc.RootElement.GetProperty("name")}");

// ──────────────────────────────────────
// 3. Boolean grammar for yes/no decisions
// ──────────────────────────────────────
var boolChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 16,
    Grammar = new Grammar(Grammar.PredefinedGrammar.Boolean)
};

string verdict = boolChat.Submit("Is the following text written in English? Text: Bonjour le monde");
Console.WriteLine($"\n── Boolean Decision ──\nResult: {verdict}");

// ──────────────────────────────────────
// 4. JSON array grammar
// ──────────────────────────────────────
var arrayChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 512,
    Grammar = new Grammar(Grammar.PredefinedGrammar.JsonArray)
};

string arrayResult = arrayChat.Submit(
    "List 5 European capital cities with their countries as JSON array of objects.");

Console.WriteLine($"\n── JSON Array ──\n{arrayResult}");

Every response is guaranteed to be syntactically valid. The JsonDocument.Parse call will never throw on grammar-constrained output.


Step 3: Enforce a Custom JSON Schema

When you need specific fields with specific types, use CreateJsonGrammarFromJsonSchema to generate a grammar from a standard JSON Schema definition. The model can only produce output matching your exact schema:

// ──────────────────────────────────────
// Define a JSON schema for invoice line items
// ──────────────────────────────────────
string invoiceSchema = """
{
    "type": "object",
    "properties": {
        "invoice_number": { "type": "string" },
        "date": { "type": "string" },
        "vendor": { "type": "string" },
        "total_amount": { "type": "number" },
        "currency": { "type": "string" },
        "line_items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "description": { "type": "string" },
                    "quantity": { "type": "integer" },
                    "unit_price": { "type": "number" }
                },
                "required": ["description", "quantity", "unit_price"]
            }
        }
    },
    "required": ["invoice_number", "date", "vendor", "total_amount", "line_items"]
}
""";

var schemaGrammar = Grammar.CreateJsonGrammarFromJsonSchema(invoiceSchema);

var schemaChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 1024,
    Grammar = schemaGrammar
};

string invoiceText = """
    Invoice #INV-2024-0847 from Acme Supplies, dated March 15, 2025.
    Items:
    - 50x Widget A at $12.99 each
    - 20x Widget B at $24.50 each
    - 5x Premium Connector at $89.00 each
    Total: $1,539.50 USD
    """;

string extracted = schemaChat.Submit($"Extract the invoice data from this text:\n{invoiceText}");
Console.WriteLine("── Schema-Constrained Extraction ──");
Console.WriteLine(extracted);

// Deserialize into a strongly-typed C# object
var invoice = JsonSerializer.Deserialize<InvoiceData>(extracted,
    new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
Console.WriteLine($"\nParsed: {invoice!.Vendor}, {invoice.LineItems.Count} items, {invoice.Currency} {invoice.TotalAmount}");

Define the C# types for deserialization:

public record InvoiceData(
    string InvoiceNumber,
    string Date,
    string Vendor,
    decimal TotalAmount,
    string Currency,
    List<LineItem> LineItems
);

public record LineItem(
    string Description,
    int Quantity,
    decimal UnitPrice
);

The JSON schema grammar ensures every required field appears, every type is correct, and no hallucinated fields are added.


Step 4: Build Field-Level Extraction Grammars

For simpler extraction tasks, use the factory methods to create grammars from field definitions without writing a full JSON schema:

using LMKit.TextGeneration.Sampling;

// ──────────────────────────────────────
// Text-only fields (all strings)
// ──────────────────────────────────────
var textFieldGrammar = Grammar.CreateJsonGrammarFromTextFields(
    new[] { "company_name", "ceo", "headquarters", "industry" });

var textChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 256,
    Grammar = textFieldGrammar
};

string companyResult = textChat.Submit(
    "Extract company info: Microsoft Corporation is led by Satya Nadella, " +
    "headquartered in Redmond, Washington. They operate in the technology industry.");
Console.WriteLine("── Text Field Extraction ──");
Console.WriteLine(companyResult);

// ──────────────────────────────────────
// Typed fields (string, number, boolean)
// ──────────────────────────────────────
var typedGrammar = Grammar.CreateJsonGrammarFromFields(
    new[] { "product_name", "price", "in_stock", "rating" },
    new[] {
        Grammar.ElementType.String,
        Grammar.ElementType.Float,
        Grammar.ElementType.Boolean,
        Grammar.ElementType.Float
    });

var typedChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 256,
    Grammar = typedGrammar
};

string productResult = typedChat.Submit(
    "Extract: The UltraWidget Pro costs $149.99, is currently in stock, and has a 4.7 rating.");
Console.WriteLine($"\n── Typed Field Extraction ──\n{productResult}");

Step 5: Constrained String Lists

When you need the model to choose from a predefined set of values (categories, labels, status codes), use CreateGrammarFromStringList:

// ──────────────────────────────────────
// Force output to one of these values
// ──────────────────────────────────────
var categoryGrammar = Grammar.CreateGrammarFromStringList(
    new[] { "bug_report", "feature_request", "question", "documentation", "security_issue" });

var classifyChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 32,
    Grammar = categoryGrammar
};

string[] tickets = {
    "The login page crashes when I enter special characters in the password field.",
    "It would be great if we could export reports to PDF format.",
    "How do I configure two-factor authentication for my account?",
    "The README has outdated installation instructions for Linux.",
    "I found an SQL injection vulnerability in the search endpoint."
};

Console.WriteLine("── Ticket Classification ──");
foreach (string ticket in tickets)
{
    string category = classifyChat.Submit($"Classify this support ticket: {ticket}");
    Console.WriteLine($"  [{category}] {ticket[..Math.Min(60, ticket.Length)]}...");
}

The model is physically unable to output anything other than the exact strings you provided. No fuzzy matches, no variations, no hallucinated categories.


Step 6: Custom GBNF Grammars

For advanced use cases, write a custom grammar in GBNF (GGML BNF) notation. This gives you complete control over the output structure:

// ──────────────────────────────────────
// Custom GBNF: semantic version string
// ──────────────────────────────────────
string semverGbnf = """
    root   ::= major "." minor "." patch
    major  ::= [0-9]+
    minor  ::= [0-9]+
    patch  ::= [0-9]+
    """;

var semverGrammar = new Grammar(semverGbnf, "root");

var versionChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 32,
    Grammar = semverGrammar
};

string version = versionChat.Submit("What is the latest stable version of .NET?");
Console.WriteLine($"── Custom GBNF ──\nVersion: {version}");

// ──────────────────────────────────────
// Custom GBNF: email address format
// ──────────────────────────────────────
string emailGbnf = """
    root      ::= local "@" domain
    local     ::= [a-zA-Z0-9._-]+
    domain    ::= subdomain "." tld
    subdomain ::= [a-zA-Z0-9-]+
    tld       ::= [a-zA-Z] [a-zA-Z]+
    """;

var emailGrammar = new Grammar(emailGbnf, "root");

var emailChat = new SingleTurnConversation(model)
{
    MaximumCompletionTokens = 64,
    Grammar = emailGrammar
};

string email = emailChat.Submit("Generate a professional email address for John Smith at Contoso.");
Console.WriteLine($"Email: {email}");

GBNF grammars let you define any output format: CSV rows, XML fragments, custom DSLs, or domain-specific notations.


Step 7: Grammar Constraints in Agent Workflows

Combine grammar constraints with the Agent builder to get structured output from agentic reasoning:

using LMKit.Agents;
using LMKit.Agents.Tools.BuiltIn;

// ──────────────────────────────────────
// Agent that always returns structured JSON
// ──────────────────────────────────────
string analysisSchema = """
{
    "type": "object",
    "properties": {
        "summary": { "type": "string" },
        "sentiment": { "type": "string" },
        "key_topics": { "type": "array", "items": { "type": "string" } },
        "confidence_score": { "type": "number" },
        "recommended_action": { "type": "string" }
    },
    "required": ["summary", "sentiment", "key_topics", "confidence_score", "recommended_action"]
}
""";

var agent = Agent.CreateBuilder(model)
    .WithInstruction(
        "You are an analysis agent. Analyze the input and return structured results. " +
        "The sentiment field must be one of: positive, negative, neutral, mixed.")
    .WithTools(tools =>
    {
        tools.Register(BuiltInTools.DateTime);
    })
    .WithGrammar(Grammar.CreateJsonGrammarFromJsonSchema(analysisSchema))
    .WithMaxIterations(3)
    .Build();

var result = await agent.ExecuteAsync(
    "Analyze this customer feedback: 'The new update is amazing! " +
    "Loading times are 3x faster. However, the new UI is confusing and I can't find the settings menu.'");

Console.WriteLine("── Agent Structured Output ──");
Console.WriteLine(result.Content);

// Safe to deserialize directly
var analysis = JsonSerializer.Deserialize<JsonElement>(result.Content);
Console.WriteLine($"\nSentiment: {analysis.GetProperty("sentiment")}");
Console.WriteLine($"Confidence: {analysis.GetProperty("confidence_score")}");

Common Issues

Problem Cause Fix
Output is truncated mid-JSON MaximumCompletionTokens too low Increase the token limit to accommodate the full schema
Schema fields missing from output Required fields not listed in JSON schema Add all mandatory fields to the "required" array
GBNF grammar causes empty output Grammar rules are too restrictive or have syntax errors Test the grammar with simple prompts first; check for missing whitespace rules
Slow generation with large grammars Complex GBNF rules increase token evaluation cost Simplify the grammar or use CreateJsonGrammarFromJsonSchema for JSON use cases
Grammar.ElementType not found Missing using directive Add using LMKit.TextGeneration.Sampling;

Next Steps