Method ImportTextFromFile

Namespace

LMKit.Retrieval

Assembly

LM-Kit.NET.dll

ImportTextFromFile(string, Encoding, TextChunking, string, string, CancellationToken)

Imports text data from a file into a specified DataSource object, creating a new or updating an existing Section entry.

public DataSource ImportTextFromFile(string path, Encoding encoding, TextChunking textChunking, string dataSourceIdentifier, string sectionIdentifier = "default", CancellationToken cancellationToken = default)

Parameters

path string

The file path from which to read the text data.

encoding Encoding

The character encoding used in the file.

textChunking TextChunking

A TextChunking object specifying the text chunking strategy to be used.

dataSourceIdentifier string

The unique identifier for the DataSource. If this identifier matches an existing DataSource, the data is added to a new section within it. If no matching identifier is found, a new DataSource is created.

sectionIdentifier string

Optional. The identifier for the new Section. Defaults to 'default'.

cancellationToken CancellationToken

Optional. A CancellationToken for canceling the operation.

Returns

DataSource

The DataSource object into which the data has been imported.

Examples

using LMKit.Data;
using LMKit.Model;
using LMKit.Retrieval;
using System;
using System.IO;
using System.Text;

class Example
{
    static void Main()
    {
        // Suppose there's a text file 'sample.txt' in the same folder.
        string filePath = "sample.txt";

        LM embeddingModel = LM.LoadFromModelID("nomic-embed-text");
        RagEngine ragEngine = new RagEngine(embeddingModel);

        DataSource dataSource = ragEngine.ImportTextFromFile(
            path: filePath,
            encoding: Encoding.UTF8,
            textChunking: new TextChunking() { MaxChunkSize = 500 },
            dataSourceIdentifier: "myDataSource",
            sectionIdentifier: "intro"
        );

        Console.WriteLine($"Imported text into DataSource: {dataSource.Identifier}");
    }
}

Exceptions

ArgumentNullException

Thrown if 'path' or 'dataSourceIdentifier' is null or empty.

OperationCanceledException

Thrown if the operation is canceled.