DeepSeek-GRM & SPCT (Self-Principled Critique Tuning)

Pointwise Generative Reward Model trained via Self-Principled Critique Tuning (SPCT): the model first dynamically generates evaluation principles (rubric criteria and weights tailored to the specific prompt $x$), then generates a detailed critique of candidate responses against those principles, and outputs pointwise scores. Trained via a two-stage pipeline: (1) Reject-sampled SFT (RFT) with hint-guided vs. non-hinted principle trajectories, followed by (2) Rule-based Online RL (GRPO) rewarding accurate winner identification. Supports Meta-RM guided voting over $K$ parallel principle-critique samples at inference time.