Microsoft announces Phi-4 in research preview

Microsoft has today officially unveiled Phi-4, the latest iteration in its Phi series of generative AI models.

Phi-4 comes with 14 billion parameters, and is positioned as a small yet powerful model that is said to ‘excel’ in specialized tasks, particularly mathematical reasoning.

“We present phi-4, a 14-billion parameter language model developed with a training recipe that is centrally focused on data quality.

Unlike most language models, where pre-training is based primarily on organic data sources such as web content or code, phi-4 strategically incorporates synthetic data throughout the training process.

While previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 substantially surpasses its teacher model on STEM-focused QA capabilities, giving evidence that our data-generation and post-training techniques go beyond distillation.

Despite minimal changes to the phi-3 architecture, phi-4 achieves strong performance relative to its size– especially on reasoning-focused benchmarks– due to improved data, training curriculum, and innovations in the post-training scheme,” the company noted in a technical report.