Property MaxChunkSize

Namespace

LMKit.Retrieval

Assembly

LM-Kit.NET.dll

MaxChunkSize

Gets or sets the maximum number of tokens that each text chunk can contain.
This property determines the size of the chunks into which the text is divided.

public int MaxChunkSize { get; set; }

Property Value

int

The default value is 500.

Remarks

A "token" refers to the smallest unit of text, such as a word or punctuation mark, identifiable in the context of text processing.
Tokens are enumerated in the Vocabs property of the Vocabulary object, which provides a comprehensive index of recognizable text elements.