Property EmbeddingPool
EmbeddingPool
The embedding-pool shape that governs THIS model: window, per-decode input ceiling and allocation timing of the single pinned context its embedding and reranking requests decode on. Callers must read the shape from the model rather than from the global configuration, because this is where a per-model override applies.
public EmbeddingPoolOptions EmbeddingPool { get; }Property Value
Remarks
Every value not stated here is inherited LIVE from
EmbeddingPool: a model with no
override reports the process-wide shape and follows later changes to
it. Setting ContextSize or
MaxSequences pins that
value for this model alone; assigning null to the matching
*Override property hands it back to the process-wide shape.
A change applies to the next request: the model's resident pool is marked stale and rebuilt at the new shape once its in-flight requests finish. The device memory ceiling still governs, so a shape the device cannot hold is fitted down at attach.