Method ImportDocumentAsync

Namespace: LMKit.Retrieval

Assembly: LM-Kit.NET.dll

ImportDocumentAsync(Attachment, string, string, DocumentMetadata, CancellationToken)

Asynchronously imports a document into a DataSource, extracting text from each page and generating embeddings for retrieval.

public Task<DataSource> ImportDocumentAsync(Attachment attachment, string dataSourceIdentifier, string pageRange = null, DocumentRag.DocumentMetadata documentMetadata = null, CancellationToken cancellationToken = default)

Parameters

attachment Attachment: The document attachment to import. Must not be null.
dataSourceIdentifier string: The unique identifier for the target DataSource. If a matching data source exists, pages are added as new sections; otherwise, a new data source is created.
pageRange string: An optional page range specification (e.g., "1-5", "1,3,5-10") to import only specific pages. If null or empty, all pages are imported.
documentMetadata DocumentRag.DocumentMetadata: Optional metadata to associate with the document. If null, default metadata is created using the attachment's name. Use this to specify a custom document name, reference URL, or additional metadata fields for source attribution.
cancellationToken CancellationToken: A token to cancel the operation.

Returns

Task<DataSource>: A task that resolves to the DataSource containing the imported document, or null if no text could be extracted from any page.

Examples

LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
DocumentRag docRag = new DocumentRag(embeddingModel);

// Import all pages with default metadata
var attachment = Attachment.FromFile("document.pdf");
var dataSource = await docRag.ImportDocumentAsync(attachment, "documents");

// Import with custom metadata including reference URL
var metadata = new DocumentMetadata(attachment, "https://intranet.example.com/docs/q4-report.pdf")
{
    DocumentName = "Q4 2024 Financial Report",
    AdditionalMetadata = new MetadataCollection
    {
        { "author", "Finance Team" },
        { "department", "Finance" }
    }
};
var dataSourceWithMeta = await docRag.ImportDocumentAsync(attachment, "financial-reports", documentMetadata: metadata);

// Import specific pages
var selectedSource = await docRag.ImportDocumentAsync(attachment, "documents", pageRange: "1,5,10-15");

Remarks

This method processes the document page by page according to the configured ProcessingMode. Each page becomes a separate section in the data source, with metadata recording the page number and document name for source attribution.

The Progress event is raised throughout the import process to report status updates.

Pages that produce no extractable text (e.g., blank pages or images without OCR) are skipped.

Exceptions

ArgumentNullException: Thrown if attachment is null.
ArgumentException: Thrown if dataSourceIdentifier is null, empty, or whitespace.
OperationCanceledException: Thrown if the operation is canceled.