Property CosineDecayRestart

Namespace

LMKit.Finetuning

Assembly

LM-Kit.NET.dll

CosineDecayRestart

Gets or sets the multiplier that increases the number of steps for cosine decay upon each restart, applicable only when using the Adam optimizer with a cyclical learning rate policy.

public float CosineDecayRestart { get; set; }

Property Value

float

The default value is 1.1f.

Remarks

This parameter controls how the number of steps between resets of the learning rate increases with each cycle, enabling progressive extension of learning rate cycles.