Namespace LMKit.Retrieval

RagEngine: Provides the core functionalities required for Retrieval Augmented Generation (RAG) within a data processing system.

RagEngine.TextPartitionSimilarity: Represents the result of a text retrieval operation, encapsulating the similarity measure between a text partition and a specific content.

TextChunking: Implements a recursive chunking strategy for partitioning text into manageable segments, known as "chunks," to support retrieval-augmented generation tasks.
This approach is particularly effective for processing extensive texts, systematically breaking them down into smaller segments that are easier to handle.
Unlike linear chunking methods that sequentially divide text, this recursive strategy dynamically adjusts the segmentation process based on the complexity and structure of the text.
This allows for more nuanced and efficient handling of text data, especially when dealing with nested or hierarchical information.