Class TopNSigmaSampling
- Namespace
- LMKit.TextGeneration.Sampling
- Assembly
- LM-Kit.NET.dll
Implements Top-nσ sampling, a text generation strategy introduced in “Top-nσ: Not All Logits Are You Need”.
This method limits the next token selection to those whose pre-softmax logits fall within n * σ of the maximum logit. By filtering the candidate tokens directly based on this statistical threshold, Top-nσ sampling ensures a stable sampling space that is robust to temperature scaling. This makes it particularly effective in reasoning tasks even when higher temperatures are used.
A higher TopNSigma value (e.g., 5) expands the sampling space to include more tokens (potentially noisier), whereas a lower value (e.g., 1) restricts the focus to the most competitive tokens.
public class TopNSigmaSampling : TokenSampling- Inheritance
-
TopNSigmaSampling
- Inherited Members
Properties
- Seed
Specifies the seed used for random number generation.
If set, the seed ensures reproducibility of the sampling process by controlling the randomness in token generation.
When not set (null), the model's behavior is non-deterministic as it relies on a system-generated random seed.
Use an unsigned integer (uint) value to define the seed for reproducibility, or leave it null for standard random behavior.
- Temperature
Gets or sets the output randomness level.
Lowering the temperature leads to more deterministic and less diverse completions. Values should be in the range [0, 1], where 0 results in a deterministic output, and values closer to 1 introduce greater randomness.
The default value is 0.8, offering a balance between randomness and determinism.
- TopK
Gets or sets the number of candidate tokens to consider (TopK) before applying the Top-nσ filter.
This parameter limits the token selection process to the TopK tokens with the highest pre-softmax logits, thereby reducing computational overhead and focusing the sampling on the most relevant candidates.
Valid values are integers in the range [1, 1000].
- TopNSigma
Gets or sets the Top-nσ threshold used to filter candidate tokens based on their pre-softmax logits.
Tokens are retained only if their logits are within TopNSigma * σ of the maximum logit. A higher threshold (e.g., 5) includes a broader set of tokens (even those with higher noise), while a lower threshold (e.g., 1) focuses on tokens that are very close to the top candidate.
Acceptable values are in the range [0, 5], where setting this value to 0 disables the Top-nσ filtering. The default value is 1.
Methods
- Clone()
Creates a deep copy of the current TopNSigmaSampling instance.