Property CharEntropyBits
- Namespace
- LMKit.Retrieval.ChunkQuality
- Assembly
- LM-Kit.NET.dll
CharEntropyBits
Shannon entropy, in bits per character, of the non-whitespace character distribution. Prose in alphabetic scripts sits around 3.8-5.2; binary noise drifts well above; repeated filler falls well below. Not meaningful for CJK scripts, whose natural entropy is intrinsically high; the gate accounts for that.
public double CharEntropyBits { get; }