Method ImportDocument

Namespace: LMKit.Retrieval

Assembly: LM-Kit.NET.dll

ImportDocument(Attachment, string, string, DocumentMetadata, CancellationToken)

Imports a document into a DataSource, extracting text from each page and generating embeddings for retrieval.

public DataSource ImportDocument(Attachment attachment, string dataSourceIdentifier, string pageRange = null, DocumentRag.DocumentMetadata documentMetadata = null, CancellationToken cancellationToken = default)

Parameters

attachment Attachment: The document attachment to import. Must not be null.
dataSourceIdentifier string: The unique identifier for the target DataSource. If a matching data source exists, pages are added as new sections; otherwise, a new data source is created.
pageRange string: An optional page range specification (e.g., "1-5", "1,3,5-10") to import only specific pages. If null or empty, all pages are imported.
documentMetadata DocumentRag.DocumentMetadata: Optional metadata to associate with the document. If null, default metadata is created using the attachment's name. Use this to specify a custom document name, reference URL, or additional metadata fields for source attribution.
cancellationToken CancellationToken: A token to cancel the operation.

Returns

DataSource: The DataSource containing the imported document, or null if no text could be extracted from any page.

Examples

LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
DocumentRag docRag = new DocumentRag(embeddingModel);

// Import all pages with default metadata
var attachment = Attachment.FromFile("document.pdf");
var dataSource = docRag.ImportDocument(attachment, "documents");

// Import with custom metadata
var metadata = new DocumentMetadata(attachment, "https://example.com/doc.pdf")
{
    AdditionalMetadata = new MetadataCollection { { "category", "technical" } }
};
var dataSourceWithMeta = docRag.ImportDocument(attachment, "documents", documentMetadata: metadata);

// Import only pages 1-10
var partialSource = docRag.ImportDocument(attachment, "documents", pageRange: "1-10");

Remarks

This method processes the document page by page according to the configured ProcessingMode. Each page becomes a separate section in the data source, with metadata recording the page number and document name for source attribution.

The Progress event is raised throughout the import process to report status updates.

Pages that produce no extractable text (e.g., blank pages or images without OCR) are skipped.

Exceptions

ArgumentNullException: Thrown if attachment is null.
ArgumentException: Thrown if dataSourceIdentifier is null, empty, or whitespace.
OperationCanceledException: Thrown if the operation is canceled.