Property LookupDraftEqualization
LookupDraftEqualization
Chain-length rule for lookup-drafted passes that carry several slots. Defaults to Longest: every drafting slot verifies the whole depth, a decision that depends on the proposals alone, so a run's drafting is the same from one run to the next. The expected-value rule prices each pass against timings it measures, and a device shared with other engines makes those timings move; measured on a document workload at four slots the two rules cost the same and the priced rule declined a tenth of the passes for nothing. Read on every pass, so a change applies to the next pass without a rebuild. Reading answers with the inherited value while LookupDraftEqualizationOverride is null; assigning sets the override.
public SharedSlotPoolOptions.DraftEqualization LookupDraftEqualization { get; set; }Property Value
Remarks
On a model carrying recurrent state, every drafting slot in a pass must verify the same number of rows: the recurrent memory decodes unequal per-sequence lengths as separate micro-batches, one per sequence, which costs more than the drafts return. The rows that bring a short proposal up to the pass length are fillers: decoded, verified, rejected and rolled back, each costing a row of decode time for nothing. Attention-only models have no such constraint; on them only ExpectedValue changes anything, by capping the depth when rows are expensive.