Enum LearningRateSchedule

Shape of the learning-rate curve applied across a training run.

public enum LearningRateSchedule

Fields

Constant = 0: A fixed learning rate for the whole run. Simple and predictable.
Cosine = 1: A cosine decay from the initial learning rate down to MinLearningRate. The default, and usually the strongest choice for multi-epoch fine-tunes.
Linear = 2: A linear decay from the initial learning rate down to MinLearningRate.
CosineWithRestarts = 3: Cosine decay that restarts once per epoch: each epoch begins back at the initial learning rate and decays toward MinLearningRate. Helps escape plateaus on longer multi-epoch runs.
Polynomial = 4: A polynomial (power 2) decay from the initial learning rate down to MinLearningRate: faster early decay than linear, gentler near the end.