Enum LoraTargetModules

Namespace

LMKit.Finetuning

Assembly

LM-Kit.NET.dll

Selects which weight matrices of the base model receive trainable LoRA adapters. Combine flags with a bitwise OR, or use Attention or All for the common presets.

[Flags]
public enum LoraTargetModules

Fields

None = 0

No modules. Not a valid training target on its own.

AttentionQuery = 1

Attention query projection.

AttentionKey = 2

Attention key projection.

AttentionValue = 4

Attention value projection.

AttentionOutput = 8

Attention output projection.

FeedForwardGate = 16

Feed-forward gate projection.

FeedForwardUp = 32

Feed-forward up projection.

FeedForwardDown = 64

Feed-forward down projection.

Output = 128

Final output (unembedding) projection.

ExpertsGate = 256

Mixture-of-Experts gate projection (MoE models only).

ExpertsUp = 512

Mixture-of-Experts up projection (MoE models only).

ExpertsDown = 1024

Mixture-of-Experts down projection (MoE models only).

ExpertsGateUp = 2048

Fused Mixture-of-Experts gate+up projection (MoE models only).

Attention = AttentionQuery | AttentionKey | AttentionValue | AttentionOutput

The four attention projections (query, key, value, output). This is the default and the recommended starting point.

AttentionAndFeedForward = Attention | FeedForwardGate | FeedForwardUp | FeedForwardDown

Attention plus the dense feed-forward projections. A good balance of capacity and cost for tasks that teach new content, not just style.

All = -1

Every supported module, including MoE experts.

Remarks

Attention-only adaptation (the default) is the memory-light choice and covers most stylistic and instruction-following fine-tunes. Adding the feed-forward projections increases capacity for knowledge-heavy tasks at the cost of a larger adapter and more memory. Expert projections apply only to Mixture-of-Experts models and match no tensors on dense models.