Class FileSystemVectorStore

Namespace

LMKit.Data.Storage

Assembly

LM-Kit.NET.dll

Provides a file system-based implementation of IVectorStore that persists vector collections as individual files on disk.

public class FileSystemVectorStore : IVectorStore, IDisposable

Inheritance

object

FileSystemVectorStore

Implements

IVectorStore

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Each collection is stored as a separate file with a ".ds" extension in the specified directory. The store maintains an in-memory cache of opened data sources for improved performance.

A collection file holds a DataSource, whose sections own partitions. One point of the vector-store model is one partition: its vector is the partition embeddings, and its metadata combines the position of the partition in the file with the metadata of the owning section. Point identifiers are therefore derived from that layout: a section holding a single partition is addressed by its own identifier, while a section holding several partitions addresses them as identifier#index. Identifiers returned by retrieval and search are always valid inputs to GetMetadataAsync(string, string, CancellationToken) and UpdateMetadataAsync(string, string, MetadataCollection, MetadataUpdateMode, CancellationToken).

Keys that describe the layout (section identifier, partition index, payload, partition type, split mode, first and last markers, section chaining) are owned by the file and are reported on every point; supplying them to an update leaves them untouched. Every other key is stored on the owning section and is reported back in the same prefixed form used by the other IVectorStore implementations. Matching accepts both the prefixed and the plain spelling of a user key.

Similarity search is an exact linear scan over the collection, so its cost grows with the number of stored partitions and no payload index is needed: the payloadIndexFields argument of CreateCollectionAsync(string, uint, IEnumerable<string>, CancellationToken) is accepted and has no effect. Deletion granularity is the section, so a delete removes every point of a section as soon as one of its points matches.

Constructors

FileSystemVectorStore(string)

Initializes a new instance of the FileSystemVectorStore class.

Properties

DirectoryPath

Gets the directory path where vector collections are stored.

Methods

Dispose()

Releases all resources used by the FileSystemVectorStore and disposes all cached data sources.

~FileSystemVectorStore()

Releases unmanaged resources.