DeepCoder 14B Preview

Overview

DeepCoder-14B-Preview is an open-source code reasoning large language model (LLM) developed by the Agentica team in collaboration with Together AI. With 14 billion parameters, DeepCoder-14B-Preview is fine-tuned from the Deepseek-R1-Distilled-Qwen-14B foundation model, leveraging distributed reinforcement learning techniques to advance automated code reasoning and problem-solving performance. The model's development focuses on scalable long-context processing, robust benchmarking, and fostering accessible research in reinforcement learning (RL) for LLMs.

Model Architecture and Training Methods

DeepCoder-14B-Preview is built upon a 14B parameter transformer architecture, fine-tuned from Deepseek-R1-Distilled-Qwen-14B. A distinguishing aspect of its development is the use of distributed RL fine-tuning, enabling the model to generalize to longer contexts than those present in training. The training process employed iterative context lengthening: the context window increased from 16,000 to 32,000 tokens during training, with demonstrated inference generalization up to 64,000 tokens.

The model's optimization leverages the GRPO+ algorithm—an enhanced version of the Generalized Reversed Policy Optimization (GRPO) approach, incorporating strategies inspired by DAPO for greater training stability. Key features of GRPO+ include the removal of entropy and KL divergence losses to prevent instability and accelerate training, overlong sequence filtering to enable robust reasoning in long contexts, and modifications to PPO's surrogate loss to support exploration and entropy stabilization. Offline difficulty filtering ensures the training set includes problems within an appropriate difficulty range without introducing runtime overhead.

System-level efficiency was addressed with verl-pipeline, an optimized extension to the open-source verl RLHF library. This extension pipelines training, reward calculation, and sampling, with optimizations such as one-off pipelining designed to maximize computational throughput during RL training.

Dataset and Evaluation Methodology

Training DeepCoder-14B-Preview required a curated dataset spanning approximately 24,000 high-quality, verifiable coding problems. The training set integrates 7,500 items from TACO Verified, 16,000 from PrimeIntellect's SYNTHETIC-1, and 600 from LiveCodeBench, with rigorous verification and deduplication protocols to ensure dataset integrity. Each coding problem includes a minimum of five unit tests and is fully programmatically verified against an external, official solution to guard against reward hacking and contamination.

During RL fine-tuning, DeepCoder employs a sandboxed reward environment for code execution. Two primary sandboxes are used: the Together Code Interpreter, which supports scalable and efficient batch evaluations, and a local sandbox based on the official LiveCodeBench infrastructure for controlled, guard-railed Python execution. The reward function itself is a sparse Outcome Reward Model (ORM): models are rewarded only if all sampled unit tests pass, with partial rewards strictly avoided. For problems with numerous tests, the 15 most challenging test cases—determined by input length—are selected, with each execution bounded by timeout.

Performance and Benchmark Results

DeepCoder-14B-Preview's performance has been systematically evaluated on multiple public coding and reasoning benchmarks. On LiveCodeBench (LCB) v5, it achieves a Pass@1 accuracy of 60.6% during the main evaluation window (August 2024 to February 2025), closely matching the performance of contemporary models such as o3-mini (Low), which records 60.9%. At 64K context evaluation, DeepCoder further demonstrates its generalization capabilities: performance increases at longer contexts, with reported scores of 45.6% at 16K, 57.9% at 32K, and 60.6% at 64K. On the HumanEval+ test suite, it registers a Pass@1 score of 92.6%.

In Codeforces rating evaluations using the Qwen CodeElo benchmark, DeepCoder-14B-Preview attains a rating of 1936, placing it in the 95.3 percentile relative to Codeforces users, and comparable to peer models such as o3-mini (Low) and O1-2024-12-17 (Low). Although trained primarily for code, DeepCoder-14B-Preview generalizes well to mathematical reasoning, scoring 73.8% on AIME 2024—a 4.1% improvement over its base.

Model Variants, Applications, and Limitations

DeepCoder-14B-Preview is developed alongside smaller and related models, including DeepCoder-1.5B-Preview and DeepScaleR-1.5B-Preview, which target math reasoning and iterative context lengthening research. While DeepCoder-14B-Preview is primarily designed for automated code generation, problem-solving, and code understanding, its performance on math tasks suggests strong reasoning capabilities beyond programming.

A key limitation lies in the model's performance at shorter context lengths: at 16K, its Pass@1 scores are lower than its base model due to longer average response lengths and truncation penalties. Training time requirements are substantial, attributed to the computational demands of long-context RL at scale. The sparse binary reward function, while effective at deterring reward hacking, may not always generalize to edge cases, especially when using a subset of all available test cases. Context inference length scaling and dataset construction protocols were explicitly implemented to mitigate test contamination and support confident evaluation.

Licensing and Open Research Commitment

DeepCoder-14B-Preview is released under the MIT License, supporting free use, modification, and community contribution. Comprehensive training logs, evaluation datasets, and code are made available to facilitate transparency, reproducibility, and further research in reinforcement learning for large language models. Supplementary benchmarking scripts and open infrastructure are accessible via the Agentica RLLM GitHub repository.

Helpful Resources

• Agentica Official Website
• DeepCoder-14B-Preview on Hugging Face
• RLLM GitHub Repository
• Wandb Training Logs
• DeepCoder-Preview-Dataset on Hugging Face
• LiveCodeBench Benchmark
• Qwen CodeElo Benchmark
• Together Code Interpreter Documentation
• Verl RLHF Library
• Evaluation Logs