Field EnableDynamicSampling

Namespace

LMKit.Global

Assembly

LM-Kit.NET.dll

Gets or sets a value indicating whether the dynamic sampling strategy is enabled.

When enabled, the dynamic sampling strategy is applied during inference time to optimize model performance and the quality of generated outputs by utilizing multiple token selection methods.

Key features of dynamic sampling include:

• Dynamic Constrained GenerationRestricts the token space at each decoding step based on real-time conditions, ensuring relevance and adherence to specific constraints.
• Perplexity-Based Token SelectionSelects tokens that minimize perplexity, enhancing the coherence and contextual consistency of the generated output.
• Context-Aware SamplingLeverages predefined contextual data to guide token choices, resulting in more fluent and contextually appropriate completions.
• Speculative SamplingIncorporates speculative sampling techniques based on real-time natural language processing (NLP) analysis during the decoding process.
• Adaptive Model CompatibilityEliminates the need for model fine-tuning to achieve high accuracy. The strategy adapts to the model's stylistic preferences during inference while maintaining low perplexity for future token selections.

Dynamic sampling acts as a real-time "voting" mechanism, blending constrained sampling with speculative sampling based on the current decoding state during inference.

This strategy is particularly effective at reducing inference times while improving accuracy and quality. It excels in tasks such as function calling, classification, and information extraction.

public static bool EnableDynamicSampling

Returns

bool

Default is true.