Qwen 2.5 Math PRM 72B

Overview

Qwen2.5-Math-PRM-72B is a process reward model (PRM) developed by the Qwen team at Alibaba Group, designed specifically to evaluate and supervise intermediate reasoning steps in mathematical problem solving by large language models (LLMs). Detailed in the publication “The Lessons of Developing Process Reward Models in Mathematical Reasoning,” released on January 14, 2025, the model is part of the Qwen2.5-Math model family, which includes smaller variants and alternative reward modeling approaches. Qwen2.5-Math-PRM-72B targets the identification and mitigation of errors throughout the stepwise reasoning process, distinguishing it from outcome-focused models that assess only the final answer.

Model Architecture and Training

Qwen2.5-Math-PRM-72B is initialized from the supervised fine-tuned Qwen2.5-Math-72B-Instruct model. Unlike traditional language models with a next-token prediction head, this PRM replaces the language modeling head with a scalar-value head composed of two linear layers. This architectural adjustment allows the model to assign a correctness score to each intermediate step within a mathematical solution. The model comprises approximately 72.8 billion parameters, making it the largest member of its PRM family.

A novel aspect of the training regimen is the introduction of the “consensus filtering” mechanism. The training data, initially synthesized through Monte Carlo (MC) estimation, is subsequently filtered using both LLM-as-a-judge and MC-estimated process label agreement. This two-phase approach first expands the data set using MC estimation with approximately 500,000 gold-standard queries, generating diverse solution paths, segmenting them into steps, and estimating step correctness as an empirical probability of leading to a correct answer. In the filtering phase, the Qwen-2_5-72b is deployed to review and cross-validate the error assignment at each reasoning step, retaining only those samples with clear consensus between the MC and LLM evaluator. This process ultimately refines the training data size from three million to 1.5 million high-quality samples.

The model is trained using cross-entropy loss applied at the end of each solution step for binary correctness classification. Training with “hard labels”—explicitly marking MC estimation values of 0 as negatives—was found to be optimal.

Process Reward Supervision and Error Identification

A key distinction of Qwen2.5-Math-PRM-72B is its focus on process-level supervision, as opposed to outcome reward models (ORMs) that only score the final outcome. The model evaluates each intermediate step, enabling more granular feedback and the identification of errors that might otherwise go undetected if only the final answer is considered. This functionality is particularly relevant in educational and automated tutoring contexts, where the path to a solution is as important as the answer.

Qwen2.5-Math-PRM-72B employs a consensus filtering process that improves the quality of its supervision signals. By reconciling MC estimation and LLM-as-a-judge voting, the model addresses label noise and ambiguity in identifying flawed reasoning within solutions. The effectiveness of process supervision for isolating errors has been demonstrated across a variety of mathematical reasoning tasks.

Benchmark Evaluation and Performance

Qwen2.5-Math-PRM-72B was rigorously evaluated on two principal benchmarks. The first, Best-of-N (BoN) at the response level, selects the highest-scoring solution from a sample set, using metrics such as GSM8K, MATH, Minerva Math, GaoKao 2023 En, OlympiadBench, College Math, and MMLU STEM. The aggregate solution score is computed as the product of stepwise scores. The second, PROCESSBENCH, directly assesses the ability to identify the specific step where erroneous reasoning first occurs or to verify full correctness.

In these evaluations, Qwen2.5-Math-PRM-72B achieved a Best-of-8 average accuracy of 69.3% using the Qwen-2_5-Math-7b policy model, and 70.7% with the 72B policy variant. These figures are competitive with, and in some cases outperform, other open-source PRMs and even proprietary models. On PROCESSBENCH, the model attained a mean F1 score of 78.3%, demonstrating strong proficiency in step-level error detection. Its performance extended to nuanced scenarios, such as identifying mistakes in answers that were ultimately correct but rooted in flawed reasoning, with an average accuracy of 58.1% across several challenging mathematical benchmarks.

The accompanying performance chart illustrates these results in relation to peer models, highlighting consistent gains in both overall accuracy and process-level evaluation.

Comparison with Related Models

Within the Qwen2.5-Math model family, Qwen2.5-Math-PRM-72B stands as the largest process reward model, generally offering improved performance over its smaller sibling, Qwen2.5-Math-PRM-7b. In contrast to the alternative Qwen-2_5-Math-72b outcome reward model, which produces a single global score per solution, the process-oriented PRM offers greater sensitivity to intermediate reasoning errors. While both model types show proficiency in error identification, the PRM’s stepwise evaluation confers an advantage in process verification and instructional feedback contexts. However, the outcome reward model maintains competitive accuracy at the solution level, at times matching or even surpassing some existing open-source PRMs.

Limitations and Considerations

Despite its strengths, there remains a considerable gap between the current PRM results and theoretical upper bounds, such as pass@8. The model’s reliance on BoN strategies introduces evaluation bias, particularly in cases where correct answers may emerge from fundamentally flawed processes. Furthermore, greater integration of high-quality human annotation and more robust policy search strategies are necessary to further close this gap and to fully realize the benefits of process-level supervision. The efficient optimization of process reward models continues to be an area of active investigation.

Licensing and Availability

The license for Qwen2.5-Math-PRM-72B is not explicitly specified in the available documentation. For information on weights, datasets, and further technical details, reference should be made to model documentation and release notes on Hugging Face and arXiv.

Helpful Links

• Qwen2.5-Math-PRM-72B Hugging Face page
• Qwen2.5-Math-PRM-7B Hugging Face page
• Qwen2.5-Math-72B-Instruct Hugging Face page
• arXiv Paper PDF: The Lessons of Developing Process Reward Models in Mathematical Reasoning
• PROCESSBENCH benchmark summary
• Qwen2 Speed Benchmark
• Hugging Face documentation for model cards and text classification