Table of Contents

Class BeforeTokenSamplingEventArgs

Namespace
LMKit.TextGeneration.Events
Assembly
LM-Kit.NET.dll

Provides details for the event that occurs before a token sampling operation.

public sealed class BeforeTokenSamplingEventArgs : EventArgs
Inheritance
BeforeTokenSamplingEventArgs
Inherited Members

Examples

The following example shows how to subscribe to the BeforeTokenSampling event, inspect and modify logits to bias token selection, and adjust repetition penalty settings before sampling occurs.

using LMKit.Model;
using LMKit.TextGeneration;
using LMKit.TextGeneration.Events;

LM model = LM.LoadFromModelID("gemma3:4b");
var chat = new MultiTurnConversation(model);

chat.BeforeTokenSampling += (sender, e) =>
{
    // Inspect the most likely token before sampling.
    int topTokenId = e.GetTokenCandidateByRank(0);
    Console.WriteLine($"Top candidate token ID: {topTokenId}");
    Console.WriteLine($"Current perplexity: {e.Perplexity:F2}");
    Console.WriteLine($"Tokens generated so far: {e.GeneratedTokens.Count}");

    // Apply a custom logit bias to boost a specific token.
    e.NextTokenLogitBias.AddTextChunkBias("yes", 2.0f);

    // Adjust repetition penalty for the upcoming sampling step.
    e.RepetitionPenalty.FrequencyPenalty = 0.5f;

    // Optionally stop generation early.
    if (e.GeneratedTokens.Count > 500)
    {
        e.Stop = true;
    }
};

var response = chat.Submit("Summarize the benefits of renewable energy.");

Remarks

The event is triggered before the application of optional repetition penalties and the subsequent execution of the sampling process.

Properties

GeneratedTokens

Retrieves a read-only collection of tokens that have been generated by the model up to this point.

Logits

An array of logit values representing the unnormalized log probabilities of all tokens.

Model

Gets the Model instance associated with this object.

NextTokenLogitBias

A LogitBias object designed to adjust the likelihood of particular tokens (or text chunks) selection for the next text completion.

Perplexity

Gets the perplexity of the model's predictions up to this point. Lower perplexity indicates better predictive performance.

RepetitionPenalty

Retrieves the current repetition penalty settings applied to prediction generation.
These settings can be modified to refine the behavior of subsequent sampling operations.

SamplingMode

Gets the current token sampling engine utilized for generating predictions.
It is possible adjust its parameter to fine-tune the next sampling operation.

Stop

A flag indicating whether the text completion process should be terminated prematurely.

Methods

GetTokenCandidateByRank(int)

Retrieves the identifier of a token based on its rank in terms of likelihood of occurrence, prior to the application of any sampling methods.

Share