Method GetRetainedSessionStateMemorySize

Namespace

LMKit.Model

Assembly

LM-Kit.NET.dll

GetRetainedSessionStateMemorySize(out int)

Gets the host memory held by the states of idle conversations that left device memory (see EnableSessionStateOffload), across every loaded model. These states let a returning conversation resume without reprocessing its history; they are bounded by one process-wide budget and released by ReleaseRetainedSessionState(out long).

public static long GetRetainedSessionStateMemorySize(out int conversations)

Parameters

conversations int

Receives the number of conversations whose state is retained, in memory or on disk.

Returns

long

The host memory, in bytes, the retained states occupy.