Noisy Embedding & Token-Weighted SFT (NEFTune / Prompt-Loss Weighting)

Modifies the SFT forward/loss pass to prevent memorization and improve conversational generation. NEFTune injects uniform random noise $\mathcal{U}(-1,1)$ scaled by $\frac{\alpha}{\sqrt{Ld}}$ into token embeddings during SFT; Prompt-Loss Weighting adds a small weight $\lambda \in [0, 0.05]$ on prompt tokens or upweights final-answer / pivotal tokens relative to boilerplate formatting tokens.

NEFTune significantly increases AlpacaEval win rates when SFT datasets are small and prone to overfitting.