Class ChunkQualityGateOptions
- Namespace
- LMKit.Retrieval.ChunkQuality
- Assembly
- LM-Kit.NET.dll
Configuration for a SemanticChunkQualityGate. The defaults implement the recommended Balanced behavior; most callers never need to change anything.
public sealed class ChunkQualityGateOptions- Inheritance
-
ChunkQualityGateOptions
- Inherited Members
Remarks
The gate copies the option values at construction time, so an options instance can be mutated and reused freely afterwards without affecting an existing gate. Threshold overrides are for advanced scenarios; the profile presets are calibrated jointly against a labeled corpus, and individual overrides shift the operating point of the whole gate.
Properties
- DropScoreThreshold
Optional override of the semantic-value score below which a chunk is dropped, in the range [0, 1]. Leave null (the default) to use the profile's calibrated value. A score below the threshold is necessary but not sufficient for a drop: the gate additionally requires positive noise evidence (ChunkQualityIssues), so lowering this value alone never makes clean text disappear.
- EnableDuplicateSuppression
Enables per-document duplicate suppression: a chunk whose normalized text was already seen in the same document (repeated headers, footers, boilerplate) is dropped as RepeatedBoilerplate, keeping only the first occurrence. Default: true.
- EnableNeighborMerge
Enables folding weak-but-clean chunks (bare headings, tiny fragments) into an adjacent chunk of the same page inside a DocumentChunkGateSession. Default: true.
- ExtractionFailedDropRatio
Fraction of a document's chunks that must be dropped as noise (binary noise, mojibake, encoding artifacts) for LooksExtractionFailed to report the document's text extraction as failed. Default: 0.6.
- FlagScoreThreshold
Optional override of the semantic-value score below which a chunk is flagged (kept out of the index but fully reported), in the range [0, 1]. Leave null (the default) to use the profile's calibrated value. Must be greater than or equal to the effective drop threshold.
- MaxMergedTextLength
Upper bound, in characters, of the embedding text produced by merging chunks. A merge that would exceed this length is not performed. Default: 4096.
- MinStandaloneChunkLength
Normalized chunks shorter than this many characters are considered too weak to stand alone and become merge candidates (never silent drops). Default: 24.