Property EmbeddingPool
EmbeddingPool
Shape and allocation policy of the resident embedding pool: the single pinned context each embedding-capable model decodes its embedding and reranking requests on. Honoured under every InferenceScheduling strategy, because embedding inference is one-shot under all of them.
public static EmbeddingPoolOptions EmbeddingPool { get; }