README.md

---
license: mit
library_name: transformers
pipeline_tag: text-generation
---

# R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised and Reinforcement Learning

The model was presented in the paper [R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised and Reinforcement Learning](https://huggingface.co/papers/2505.21668).

Our code is based on [Llama-factory](https://github.com/hiyouga/LLaMA-Factory)/[VeRL](https://github.com/volcengine/verl)/[Search-R1](https://github.com/PeterGriffinJin/Search-R1?tab=readme-ov-file) for the SFT and RL training and [SymBench](https://github.com/yongchao98/CodeSteer-v1.0/tree/main)/[BIG-Bench-Hard](https://github.com/yongchao98/R1-Code-Interpreter/tree/main)/[reasoning-gym](https://github.com/open-thought/reasoning-gym) for datasets/benchmarks of reasoning/planning tasks.

## 📝 Introduction
R1-Code-Interpreter is the first framework to train LLMs for step-by-step code reasoning using multi-turn supervised fine-tuning and reinforcement learning. By curating 144 diverse reasoning and planning tasks, we enable Qwen-2.5 models (3B/7B/14B) to autonomously decide when and how to invoke code. Our best model, R1-CI-14B, outperforms GPT-4o (text-only) and approaches GPT-4o with Code Interpreter, showing emergent self-checking behavior via code generation.

[Github repository](https://github.com/yongchao98/R1-Code-Interpreter)

Project page: https://huggingface.co/yongchao98