Qwen 2.5 Math PRM 7B

Overview

Qwen2.5-Math-PRM-7B is a process reward model (PRM) developed by the Qwen team at Alibaba Group, released in January 2025. Designed specifically for mathematical reasoning within large language models, Qwen2.5-Math-PRM-7B provides fine-grained feedback on intermediate steps, aiming to strengthen the reliability and interpretability of complex reasoning processes. Unlike traditional outcome-based models, which evaluate only the correctness of final answers, this model supervises each step of mathematical problem solving, allowing developers and researchers to more rigorously identify and mitigate sources of error in automated reasoning. The development, technical framework, and benchmarking of Qwen2.5-Math-PRM-7B are documented in the original release paper and subsequent benchmark evaluations.

Technical Foundations and Architecture

Qwen2.5-Math-PRM-7B builds on the foundation established by the Qwen2.5-Math-7B-Instruct model, with key modifications tailored for process supervision tasks. Instead of a conventional language modeling head, which predicts subsequent tokens, Qwen2.5-Math-PRM-7B replaces this with a scalar value head composed of two linear layers. This structure produces a continuous reward score for each reasoning step, quantifying its correctness and coherence within the overall problem solution as described in the Qwen2.5-Math-PRM introduction. The model contains 7.63 billion parameters and is distributed in Safetensors format for openness and reproducibility.

Qwen2.5-Math-PRM-7B's design centers around the concept of PRM, which systematically evaluates the reasoning process. This is in contrast to outcome reward models (ORMs), which assess only the final answer. By focusing on step-level quality, PRMs offer a more fine-grained view of LLM performance, though at the cost of greater requirements for curated training data, as outlined in the original paper.

For reward computation, reasoning steps within mathematical solutions are separated using double line breaks, with a special <extra_0> token inserted after each step. The model scores the probability of this token being interpreted as correct, assigning a reward between 0 and 1.

Data Construction and Training Methodology

The development of Qwen2.5-Math-PRM-7B addressed the substantial challenge of generating high-quality, step-level supervision data for mathematical reasoning. Data collection and labeling followed a two-stage process, each enhancing the granularity and consistency of resulting annotations.

Initially, a large dataset of about 500,000 queries with verified correct answers was compiled. For each query, 6 to 8 diverse reasoning chains were created by combining outputs from Qwen2-Math-Instruct and Qwen2.5-Math-Instruct models across different parameter scales, as detailed in the dataset construction process. These responses were broken down into individual steps. Step correctness was empirically estimated by independently completing each partial solution multiple times and observing whether it could lead to the correct final answer—an approach rooted in Monte Carlo (MC) estimation. In this regime, a step is labeled as correct if any continuation from that step produces the correct solution, aligning with the MC estimation threshold described in the research report.

Subsequently, the dataset underwent consensus filtering. Here, step-level annotations generated by a large language model acting as a judge (specifically, Qwen2.5-Instruct-72B) were compared with MC-estimated labels. Only those instances with agreement between model-generated and empirical labels were retained, ensuring high consistency and reducing annotation noise. The final training employed binary (hard) classification labels and cross-entropy loss at each step, allowing the model to focus on determinate error identification.

Evaluation Benchmarks and Model Performance

Qwen2.5-Math-PRM-7B underwent extensive benchmarking across standardized mathematical problem-solving tasks. It was primarily evaluated using Best-of-N (BoN) strategies—where the top N reasoning chains are assessed for accuracy—and the ProcessBench benchmark, which targets step-level error identification.

In BoN evaluations, using Qwen2.5-Math-7B-Instruct as the policy model, Qwen2.5-Math-PRM-7B achieved an average accuracy of 67.6% across key mathematical datasets, including GSM8K, MATH, Minerva Math, OlympiadBench, College Math, and MMLU STEM. This surpassed the maj@8 baseline (66.2%) across all seven tasks, with an average improvement of 1.4% documented in the evaluation results.

On the ProcessBench task—focusing on step-level error detection—the model achieved an average F1 score of 73.5%. It demonstrated stronger process-based error identification relative to other published open-source PRMs and outperformed the GPT-4o-0806 baseline on this metric. However, a performance gap remains between Qwen2.5-Math-PRM-7B and both the o1-mini PRM and theoretical upper bounds such as pass@8.

Furthermore, Qwen2.5-Math-PRM-7B achieved 53.9% accuracy in identifying flawed reasoning steps even when answers were ultimately correct, surpassing most comparable open-source PRMs—this indicates its capabilities in process-oriented assessment over outcome-oriented metrics.

Applications and Usage Practices

Qwen2.5-Math-PRM-7B is primarily designed for applications that benefit from rigorous, step-wise evaluation of reasoning in mathematical domains. Its granularity enables it to surface and flag errors at intermediate steps, facilitating model debugging, educational feedback, and reliability analysis in AI systems intended for mathematical and scientific problem solving, as discussed in the primary release documentation.

To use the model effectively, solutions should be formatted such that each step is separated by a double newline. The process reward for each step is then calculated by appending the <extra_0> token and extracting its positive classification probability. The recommended minimum version of the transformers library for running Qwen2.5-Math-PRM-7B is 4.40.0.

Limitations and Considerations

Despite its advancements in process supervision, several limitations are documented. There remains a measurable gap between Qwen2.5-Math-PRM-7B’s performance and the BoN upper bound (pass@8), indicating room for optimization. Data efficiency, particularly in leveraging high-quality human-annotated reasoning steps, is yet to be fully realized despite the integration of LLM-as-a-judge with MC estimation. Developers further caution that BoN-based evaluations may introduce bias in favor of models which generate correct answers through suboptimal or flawed intermediate reasoning, potentially weakening the alignment with true process-based assessment objectives. Optimal scoring approaches also vary depending on whether hard MC labels or LLM judgments are used.

Comparison and Model Family

Qwen2.5-Math-PRM-7B is part of a broader suite of models within the Qwen2.5-Math family. Its close relative, Qwen2.5-Math-PRM-72B, is a larger PRM exhibiting generally stronger performance, particularly on more challenging benchmarks. The family also includes the outcome-based Qwen2.5-Math-RM-72B, as well as base and instruction-tuned variants such as Qwen2.5-Math-7B and Qwen2.5-Math-7B-Instruct, which are used as initializers and as policy models for evaluation.

Further Reading and External Resources

• The Lessons of Developing Process Reward Models in Mathematical Reasoning (arXiv) — Full technical report detailing Qwen2.5-Math-PRM-7B’s data construction, architecture, and benchmarking.
• ProcessBench: Process-based Evaluation of Open-source PRMs — Benchmarking protocol and comparative analysis of process reward models in mathematical reasoning.
• Qwen2 Speed Benchmark Documentation — Details on throughput and memory requirements for Qwen2.5 models.