Property CosineDecayMin

Namespace

LMKit.Finetuning

Assembly

LM-Kit.NET.dll

CosineDecayMin

Gets or sets the minimum value for the cosine decay when using the Adam optimizer. This defines the lower limit of the decay rate, influencing the learning rate adjustment over time.

public float CosineDecayMin { get; set; }

Property Value

float

The default value is 0.1.

Remarks

Cosine decay is used to adjust the learning rate in a non-linear fashion throughout training cycles, mimicking a cosine wave. This setting ensures that the decay does not reduce the learning rate below a specified threshold, essential for maintaining training stability towards the end of training cycles.