Table of Contents

Method ImportTextAsync

Namespace
LMKit.Retrieval
Assembly
LM-Kit.NET.dll

ImportTextAsync(string, string, string, CancellationToken)

Asynchronously imports a text string into a DataSource, creating a new section.

public Task<DataSource> ImportTextAsync(string data, string dataSourceIdentifier, string sectionIdentifier, CancellationToken cancellationToken = default)

Parameters

data string

The text content to import.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the identifier exists, the text is added as a new section; otherwise, a new data source is created.

sectionIdentifier string

The identifier for the new section.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the DataSource containing the imported text.

Examples

LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
RagEngine ragEngine = new RagEngine(embeddingModel);

string textData = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
DataSource dataSource = await ragEngine.ImportTextAsync(
    data: textData,
    dataSourceIdentifier: "latinData",
    sectionIdentifier: "introduction");

Console.WriteLine($"Data imported into source: {dataSource.Identifier}");

Exceptions

ArgumentNullException

Thrown if data or dataSourceIdentifier is null or empty.

OperationCanceledException

Thrown if the operation is canceled.

ImportTextAsync(string, IChunking, string, string, CancellationToken)

Asynchronously imports a text string into a DataSource, creating a new section with a custom chunking strategy.

public Task<DataSource> ImportTextAsync(string data, IChunking chunker, string dataSourceIdentifier, string sectionIdentifier, CancellationToken cancellationToken = default)

Parameters

data string

The text content to import.

chunker IChunking

The chunking strategy for splitting the text. If null, DefaultIChunking is used.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the identifier exists, the text is added as a new section; otherwise, a new data source is created.

sectionIdentifier string

The identifier for the new section.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the DataSource containing the imported text.

Examples

LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
RagEngine ragEngine = new RagEngine(embeddingModel);

string textData = "Lorem ipsum dolor sit amet, consectetur adipiscing elit.";
DataSource dataSource = await ragEngine.ImportTextAsync(
    data: textData,
    chunker: new TextChunking { MaxChunkSize = 500 },
    dataSourceIdentifier: "latinData",
    sectionIdentifier: "introduction");

Console.WriteLine($"Data imported into source: {dataSource.Identifier}");

Exceptions

ArgumentNullException

Thrown if data or dataSourceIdentifier is null or empty.

OperationCanceledException

Thrown if the operation is canceled.

ImportTextAsync(string, string, string, MetadataCollection, CancellationToken)

Asynchronously imports a text string into a DataSource, creating a new section and attaching additional metadata.

public Task<DataSource> ImportTextAsync(string data, string dataSourceIdentifier, string sectionIdentifier, MetadataCollection additionalMetadata, CancellationToken cancellationToken = default)

Parameters

data string

The text content to import.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the identifier exists, the text is added as a new section; otherwise, a new data source is created.

sectionIdentifier string

The identifier for the new section.

additionalMetadata MetadataCollection

Metadata to associate with the imported section.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the DataSource containing the imported text.

Exceptions

ArgumentNullException

Thrown if data or dataSourceIdentifier is null or empty.

OperationCanceledException

Thrown if the operation is canceled.

ImportTextAsync(string, IChunking, string, string, MetadataCollection, CancellationToken)

Asynchronously imports a text string into a DataSource, creating a new section with a custom chunking strategy and attaching additional metadata.

public Task<DataSource> ImportTextAsync(string data, IChunking chunker, string dataSourceIdentifier, string sectionIdentifier, MetadataCollection additionalMetadata, CancellationToken cancellationToken = default)

Parameters

data string

The text content to import.

chunker IChunking

The chunking strategy for splitting the text. If null, DefaultIChunking is used.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the identifier exists, the text is added as a new section; otherwise, a new data source is created.

sectionIdentifier string

The identifier for the new section.

additionalMetadata MetadataCollection

Metadata to associate with the imported section.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the DataSource containing the imported text.

Exceptions

ArgumentNullException

Thrown if data or dataSourceIdentifier is null or empty.

OperationCanceledException

Thrown if the operation is canceled.

ImportTextAsync(IList<string>, string, IList<string>, IList<MetadataCollection>, CancellationToken)

Asynchronously imports multiple text strings into a DataSource, creating a new section for each.

public Task<DataSource> ImportTextAsync(IList<string> data, string dataSourceIdentifier, IList<string> sectionIdentifiers, IList<MetadataCollection> metadataCollections = null, CancellationToken cancellationToken = default)

Parameters

data IList<string>

A list of text strings to import, each representing a separate section.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the data source exists, new sections are added; otherwise, a new data source is created.

sectionIdentifiers IList<string>

A list of unique identifiers for the new sections. The count must match the number of text strings in data.

metadataCollections IList<MetadataCollection>

An optional list of metadata collections to associate with each imported section. If provided, the count must match the number of text strings.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the updated or newly created DataSource.

Exceptions

ArgumentNullException

Thrown if data, dataSourceIdentifier, or sectionIdentifiers is null or empty.

ArgumentOutOfRangeException

Thrown if the count of sectionIdentifiers does not match the count of data.

ImportTextAsync(IList<string>, IChunking, string, IList<string>, IList<MetadataCollection>, CancellationToken)

Asynchronously imports multiple text strings into a DataSource, creating a new section for each with a custom chunking strategy.

public Task<DataSource> ImportTextAsync(IList<string> data, IChunking chunker, string dataSourceIdentifier, IList<string> sectionIdentifiers, IList<MetadataCollection> metadataCollections = null, CancellationToken cancellationToken = default)

Parameters

data IList<string>

A list of text strings to import, each representing a separate section.

chunker IChunking

The chunking strategy for splitting each text. If null, DefaultIChunking is used.

dataSourceIdentifier string

The unique identifier for the target DataSource. If the data source exists, new sections are added; otherwise, a new data source is created.

sectionIdentifiers IList<string>

A list of unique identifiers for the new sections. The count must match the number of text strings in data.

metadataCollections IList<MetadataCollection>

An optional list of metadata collections to associate with each imported section. If provided, the count must match the number of text strings.

cancellationToken CancellationToken

A token to cancel the operation.

Returns

Task<DataSource>

A task that resolves to the updated or newly created DataSource.

Examples

LM embeddingModel = LM.LoadFromModelID("embeddinggemma-300m");
RagEngine ragEngine = new RagEngine(embeddingModel);

var pages = new List<string>
{
    "Page 1: Introduction to RAG systems.",
    "Page 2: Advanced RAG techniques."
};
var sectionIds = new List<string> { "chapter1", "chapter2" };

DataSource ds = await ragEngine.ImportTextAsync(
    data: pages,
    chunker: new TextChunking { MaxChunkSize = 300 },
    dataSourceIdentifier: "ragBook",
    sectionIdentifiers: sectionIds);

Console.WriteLine($"Imported {pages.Count} pages into DataSource: {ds.Identifier}");

Exceptions

ArgumentNullException

Thrown if data, dataSourceIdentifier, or sectionIdentifiers is null or empty.

ArgumentOutOfRangeException

Thrown if the count of sectionIdentifiers does not match the count of data.