Property ContextSize
ContextSize
Longest single input the pool embeds in one pass. Defaults to 4096. Values below 256 are raised to 256; the model's own trained window caps it at attach.
public int ContextSize { get; set; }Property Value
Remarks
The pool's window is ONE window for the whole pool, not a per-request share: pending inputs whose combined length fits it decode together, each as its own sequence. The pool opens at the window its inputs need (at least 1024 tokens) and grows, up to this value, the first time a longer input arrives, because the memory a decode reserves grows faster than linearly with the window: a pool serving short retrieval chunks never pays for a window it does not use. An input longer than this value, or than a window the device could not grow to, is split into window-sized chunks whose vectors are mean-pooled. Reading answers with the inherited value while ContextSizeOverride is null; assigning sets the override.