PPG (Phasic Policy Gradient) / Decoupled Policy-Value RLHF

Separates policy updates (actor phase) and value-function distillation/training (auxiliary phase) into distinct alternating phases so value estimation can train for more epochs without corrupting shared representations or destabilizing policy gradients.

Prevents representation interference when the actor and critic share a backbone.