Class RagEngine

Namespace: LMKit.Retrieval

Assembly: LM-Kit.NET.dll

Provides core functionalities for Retrieval-Augmented Generation (RAG) within a data processing system.

public class RagEngine

Inheritance: object

RagEngine

Derived: DocumentRag

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Examples

// Basic RAG setup with text embeddings
LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
RagEngine ragEngine = new RagEngine(embeddingModel);

// Import some text
ragEngine.ImportText("Your document content here", "docs", "section1");

// Find matching partitions
var matches = ragEngine.FindMatchingPartitions("your query", topK: 5);

Remarks

The RagEngine class enables importing, indexing, and querying text and multimodal content using embedding models. It supports vector similarity search, optional reranking, and integration with external vector stores.

To use multimodal retrieval (text + images), supply both a text embedding model and a vision-enabled embedding model to the constructor. All models must share the same embedding space dimensions.

Constructors

RagEngine(LM, IVectorStore): Initializes a new instance of the RagEngine class with a single embedding model.

RagEngine(IEnumerable<LM>, IVectorStore): Initializes a new instance of the RagEngine class with multiple embedding models.

Properties

DataSources: Gets a read-only collection of DataSource objects representing the imported content repositories.

DefaultIChunking: Gets or sets the default text chunking configuration used for splitting text into manageable partitions.

DefaultImagePayloadPix: Gets or sets the maximum pixel count for images when generating vision embeddings.

Filter: Gets or sets the filtering criteria used when processing data sources and sections.

Reranker: Gets or sets the RagEngine.RagReranker used to optionally rerank retrieval results.

Methods

AddDataSource(DataSource): Registers a new DataSource with this engine for retrieval operations.

AddDataSource(string, CancellationToken): Loads and registers an existing DataSource from the configured vector store.

AddDataSourceAsync(string, CancellationToken): Asynchronously loads and registers an existing DataSource from the configured vector store.

AddDataSources(IEnumerable<DataSource>): Registers multiple DataSource instances with this engine for retrieval operations.

ClearDataSources(): Removes all registered DataSource objects from this engine.

FindMatchingPartitions(Attachment, int, float, bool, bool, CancellationToken): Searches for similar partitions across all registered data sources using attachment content.

FindMatchingPartitions(string, int, float, bool, bool, CancellationToken): Searches for similar partitions across all registered data sources using text similarity.

FindMatchingPartitionsAsync(Attachment, int, float, bool, bool, CancellationToken): Asynchronously searches for similar partitions across all registered data sources using attachment content.

FindMatchingPartitionsAsync(string, int, float, bool, bool, CancellationToken): Asynchronously searches for similar partitions across all registered data sources using text similarity.

GetDataSource(string): Retrieves a registered DataSource by its identifier.

Import(Attachment, IChunking, string, string, MetadataCollection, CancellationToken): Imports content from a single Attachment into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

Import(Attachment, IChunking, string, string, CancellationToken): Imports content from a single Attachment into a DataSource, creating a new section with a custom chunking strategy.

Import(Attachment, string, string, MetadataCollection, CancellationToken): Imports content from a single Attachment into a DataSource, creating a new section and attaching additional metadata.

Import(Attachment, string, string, CancellationToken): Imports content from a single Attachment into a DataSource, creating a new section.

Import(ImageBuffer, string, string, MetadataCollection, CancellationToken): Imports a single image into a DataSource, creating a new section and attaching additional metadata.

Import(ImageBuffer, string, string, CancellationToken): Imports a single image into a DataSource, creating a new section.

Import(IList<Attachment>, IChunking, string, IList<string>, IList<MetadataCollection>, CancellationToken): Imports content from multiple Attachment instances into a DataSource, creating a new section for each attachment with a custom chunking strategy.

Import(IList<Attachment>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Imports content from multiple Attachment instances into a DataSource, creating a new section for each attachment.

Import(IList<ImageBuffer>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Imports multiple images into a DataSource, creating a new section for each.

ImportAsync(Attachment, IChunking, string, string, MetadataCollection, CancellationToken): Asynchronously imports content from a single Attachment into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

ImportAsync(Attachment, IChunking, string, string, CancellationToken): Asynchronously imports content from a single Attachment into a DataSource, creating a new section with a custom chunking strategy.

ImportAsync(Attachment, string, string, MetadataCollection, CancellationToken): Asynchronously imports content from a single Attachment into a DataSource, creating a new section and attaching additional metadata.

ImportAsync(Attachment, string, string, CancellationToken): Asynchronously imports content from a single Attachment into a DataSource, creating a new section.

ImportAsync(ImageBuffer, string, string, MetadataCollection, CancellationToken): Asynchronously imports a single image into a DataSource, creating a new section and attaching additional metadata.

ImportAsync(ImageBuffer, string, string, CancellationToken): Asynchronously imports a single image into a DataSource, creating a new section.

ImportAsync(IList<Attachment>, IChunking, string, IList<string>, IList<MetadataCollection>, CancellationToken): Asynchronously imports content from multiple Attachment instances into a DataSource, creating a new section for each attachment with a custom chunking strategy.

ImportAsync(IList<Attachment>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Asynchronously imports content from multiple Attachment instances into a DataSource, creating a new section for each attachment.

ImportAsync(IList<ImageBuffer>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Asynchronously imports multiple images into a DataSource, creating a new section for each.

ImportText(IList<string>, IChunking, string, IList<string>, IList<MetadataCollection>, CancellationToken): Imports multiple text strings into a DataSource, creating a new section for each with a custom chunking strategy.

ImportText(IList<string>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Imports multiple text strings into a DataSource, creating a new section for each.

ImportText(string, IChunking, string, string, MetadataCollection, CancellationToken): Imports a text string into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

ImportText(string, IChunking, string, string, CancellationToken): Imports a text string into a DataSource, creating a new section with a custom chunking strategy.

ImportText(string, string, string, MetadataCollection, CancellationToken): Imports a text string into a DataSource, creating a new section and attaching additional metadata.

ImportText(string, string, string, CancellationToken): Imports a text string into a DataSource, creating a new section.

ImportTextAsync(IList<string>, IChunking, string, IList<string>, IList<MetadataCollection>, CancellationToken): Asynchronously imports multiple text strings into a DataSource, creating a new section for each with a custom chunking strategy.

ImportTextAsync(IList<string>, string, IList<string>, IList<MetadataCollection>, CancellationToken): Asynchronously imports multiple text strings into a DataSource, creating a new section for each.

ImportTextAsync(string, IChunking, string, string, MetadataCollection, CancellationToken): Asynchronously imports a text string into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

ImportTextAsync(string, IChunking, string, string, CancellationToken): Asynchronously imports a text string into a DataSource, creating a new section with a custom chunking strategy.

ImportTextAsync(string, string, string, MetadataCollection, CancellationToken): Asynchronously imports a text string into a DataSource, creating a new section and attaching additional metadata.

ImportTextAsync(string, string, string, CancellationToken): Asynchronously imports a text string into a DataSource, creating a new section.

ImportTextFromFile(string, Encoding, IChunking, string, string, MetadataCollection, CancellationToken): Imports text from a file into a DataSource by creating a new section, using a custom chunking strategy and attaching additional metadata.

ImportTextFromFile(string, Encoding, IChunking, string, string, CancellationToken): Imports text from a file into a DataSource by creating a new section, using a custom chunking strategy.

ImportTextFromFile(string, Encoding, string, string, MetadataCollection, CancellationToken): Imports text from a file into a DataSource by creating a new section, and attaches additional metadata.

ImportTextFromFile(string, Encoding, string, string, CancellationToken): Imports text from a file into a DataSource by creating a new section.

ImportTextFromFileAsync(string, Encoding, IChunking, string, string, MetadataCollection, CancellationToken): Asynchronously imports text from a file into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

ImportTextFromFileAsync(string, Encoding, IChunking, string, string, CancellationToken): Asynchronously imports text from a file into a DataSource, creating a new section with a custom chunking strategy.

ImportTextFromFileAsync(string, Encoding, string, string, MetadataCollection, CancellationToken): Asynchronously imports text from a file into a DataSource, creating a new section and attaching additional metadata.

ImportTextFromFileAsync(string, Encoding, string, string, CancellationToken): Asynchronously imports text from a file into a DataSource, creating a new section.

QueryPartitions(string, IEnumerable<PartitionSimilarity>, IConversation, CancellationToken): Generates a text response by querying specified partitions within a conversation context.

QueryPartitions(string, string, IEnumerable<PartitionSimilarity>, IConversation, CancellationToken): Generates a text response by querying specified partitions within a conversation context, using a custom prompt template.

QueryPartitionsAsync(string, IEnumerable<PartitionSimilarity>, IConversation, CancellationToken): Asynchronously generates a text response by querying specified partitions within a conversation context.

QueryPartitionsAsync(string, string, IEnumerable<PartitionSimilarity>, IConversation, CancellationToken): Asynchronously generates a text response by querying specified partitions within a conversation context, using a custom prompt template.

RemoveDataSource(DataSource): Removes a DataSource from this engine.

RemoveDataSource(string): Removes a registered DataSource by its identifier.