Self-Instruct, Magpie & Persona-Driven Synthetic SFT (Alpaca / Magpie / PersonaHub)

Generates synthetic instruction-response pairs directly from a teacher model. Self-Instruct bootstraps from a 175-task seed pool via few-shot prompting; Magpie feeds only the pre-query chat template token (e.g., <|start_header_id|>user<|end_header_id|>) into an aligned LLM to sample high-density user queries from its own prior; PersonaHub conditions generation on 1 billion diverse synthetic personas to prevent mode collapse.

Without persona conditioning or Magpie sampling, naive synthetic prompt generation rapidly collapses onto repetitive topics and phrasings.