Table of Contents

🔄 Understanding Reranking in LM-Kit.NET


📄 TL;DR:

Reranking refines or reorders an initial set of retrieval results by applying a secondary scoring model, typically based on embeddings, to improve relevance. In LM-Kit.NET, the Reranker class computes similarity scores and blends them to produce normalized RerankScore values, ensuring that the most pertinent documents or text partitions rise to the top.


📝 What is Reranking?

Definition: Reranking is the process of taking an initial list of candidate items, such as documents or text partitions, scored by a primary retrieval method, and applying a more focused or specialized model to recompute and reorder them based on refined similarity metrics.

  • Initial Retrieval: A coarse search (e.g., cosine similarity on embeddings) produces a ranked list.
  • Secondary Scoring: A reranking model re-evaluates each candidate’s relevance.
  • Final Order: Items are resorted by the new RerankScore, boosting precision at top ranks.

🔍 The Role of Reranking in Retrieval Workflows

  1. Enhanced Precision: Elevate truly relevant results by re-evaluating borderline cases.
  2. Blended Metrics: Combine raw similarity and reranker insights via a blend factor for balanced scoring.
  3. Noise Reduction: Demote false positives by penalizing low-quality matches.

⚙️ Key Class: Reranker

Located in LMKit.Embeddings, the Reranker encapsulates embedding-based rescoring logic:

public sealed class Reranker
{
    // Constructor: requires an embedding-capable LM instance
    public Reranker(LM model);

    // Controls whether scores normalize to [0,1]. Default: true
    public bool NormalizeScore { get; set; }

    // Underlying language model used for computing embeddings
    public LM Model { get; }

    // Synchronous scoring methods
    public float GetScore(string query, string document, CancellationToken token);
    public IEnumerable<float> GetScore(string query, IEnumerable<string> docs, CancellationToken token);
    public float GetScore(string query, TextPartition part, CancellationToken token);
    public IEnumerable<float> GetScore(string query, IEnumerable<TextPartition> parts, CancellationToken token);

    // Asynchronous counterparts: GetScoreAsync(...)

    // Reranking methods: update PartitionSimilarity.RerankScore
    public void Rerank(string query, PartitionSimilarity item, float blend, CancellationToken token);
    public void Rerank(string query, IEnumerable<PartitionSimilarity> items, float blend, CancellationToken token);

    // Async versions: RerankAsync(...)
}

🧩 Supporting Type: PartitionSimilarity

Found in LMKit.Retrieval, PartitionSimilarity holds both raw and reranked scores:

  • RawSimilarity: Initial score from embedding comparison.
  • RerankScore: Blended score after reranking is applied.
  • Embeddings: Vector(s) for the partition.
  • DataSourceIdentifier, SectionIdentifier, PartitionIndex, Metadata, Payload: Contextual properties.

🔗 Integration with RagEngine

The RagEngine in LMKit.Retrieval supports optional reranking via its Reranker property:

var engine = new RagEngine(lm, vectorStore);
engine.Reranker = new Reranker(lm);

// When calling FindMatchingPartitions, set the last boolean to true
var results = engine.FindMatchingPartitions(query, topK, threshold, includePayload: true, useReranker: true, token);

If Reranker is null or useReranker is false, only raw similarity scores are used.


📖 Common Terms

  • Reranking: The act of rescoring and reordering retrieval results using a secondary model.
  • Blend Factor: A float weight (0–1) indicating how much to mix raw and reranked scores.
  • RawSimilarity: The baseline similarity score from initial retrieval.
  • RerankScore: Final relevance score after applying the reranker.

🔗 Related Concepts

  • Semantic Search: Finding relevant items based on embedding similarity.
  • Embedding: High-dimensional vector representation of text or images.
  • Retrieval Augmented Generation (RAG): Combining retrieval with generation, where reranking refines context selection.

📝 Summary

In LM-Kit.NET, reranking refines retrieval outputs by applying the Reranker class to adjust and normalize similarity scores. By blending raw and secondary metrics, reranking ensures that the most relevant documents or partitions appear at the top, enhancing precision for downstream tasks such as semantic search, clustering, and RAG pipelines.