Method ImportTextFromFile

Namespace: LMKit.Retrieval

Assembly: LM-Kit.NET.dll

ImportTextFromFile(string, Encoding, TextChunking, string, string, CancellationToken)

Imports text data from a file into a specified DataSource object, creating a new or updating an existing Section entry.

public DataSource ImportTextFromFile(string path, Encoding encoding, TextChunking textChunking, string dataSourceIdentifier, string sectionIdentifier = "default", CancellationToken cancellationToken = default)

Parameters

path string: The file path from which to read the text data.
encoding Encoding: The character encoding used in the file.
textChunking TextChunking: A TextChunking object specifying the text chunking strategy to be used by the engine.
dataSourceIdentifier string: The unique identifier for the DataSource. If this identifier matches an existing DataSource, the data is added to a new section within it. If no matching identifier is found, a new DataSource is created.
sectionIdentifier string: Optional. The identifier for the new Section within the DataSource where the data will be imported. The default value is 'default'.
cancellationToken CancellationToken: Optional. A CancellationToken that can be used to signal cancellation of the import operation.

Returns

DataSource: The DataSource object into which the data has been imported.

Remarks

The hierarchy of data structures is organized as follows: a DataSource contains a collection of Section objects, each of which can hold a collection of TextPartition instances.

Each import operation can target a new or existing data source, depending on whether the specified dataSourceIdentifier matches an existing DataSource element within the DataSources property.

If the import operation targets an existing data source that contains a section with the same identifier as dataSourceIdentifier, the imported text data will be appended to that Section. Otherwise, a new Section is created with the provided identifier.

Exceptions