Interface IPromptFilter

Namespace: LMKit.TextGeneration.Filters

Assembly: LM-Kit.NET.dll

A filter that intercepts the prompt pipeline before and after inference.

public interface IPromptFilter

Examples

Logging filter that records every prompt:

using LMKit.TextGeneration.Filters;
using System;
using System.Diagnostics;
using System.Threading.Tasks;

public sealed class LoggingPromptFilter : IPromptFilter
{
    public async Task OnPromptAsync(
        PromptFilterContext context,
        Func<PromptFilterContext, Task> next)
    {
        Console.WriteLine($"[Prompt] {context.Prompt}");
        var sw = Stopwatch.StartNew();

        await next(context);

        sw.Stop();
        Console.WriteLine($"[Done] {sw.ElapsedMilliseconds} ms, " +
            $"{context.Result?.GeneratedTokenCount} tokens");
    }
}

Content moderation filter that blocks unsafe prompts:

using LMKit.TextGeneration.Filters;
using System;
using System.Threading.Tasks;

public sealed class ModerationFilter : IPromptFilter
{
    public Task OnPromptAsync(
        PromptFilterContext context,
        Func<PromptFilterContext, Task> next)
    {
        if (ContainsBlockedContent(context.Prompt))
        {
            // Short-circuit: do not call next, do not run inference
            throw new InvalidOperationException("Prompt blocked by content policy.");
        }

        return next(context);
    }

    private static bool ContainsBlockedContent(string prompt) => false;
}

Remarks

Prompt filters execute in a middleware (onion) pattern around the inference call. Code before await next(context) runs before inference; code after runs when inference completes. Filters execute in the order they are added to PromptFilters.

Common use cases:

Content moderation: scan and reject unsafe prompts.
PII redaction: strip sensitive data before it reaches the model.
Prompt augmentation: append retrieved context (RAG) to the prompt.
Semantic caching: return a cached result and skip inference entirely.
Logging and telemetry: record every prompt and its latency.

Short-circuiting: Set Result to a non-null value and do not call next. The pipeline will skip inference and return the provided result.

Retry: Call next(context) more than once (e.g., after modifying the prompt following an error).

Methods

OnPromptAsync(PromptFilterContext, Func<PromptFilterContext, Task>): Called when a prompt is about to be submitted for inference.

Table of Contents

Interface IPromptFilter

Examples

Remarks

Methods