Hugging Face's logo

MathLLMs
/
FigCodifier

Image-Text-to-Text
Transformers
Safetensors
English
internvl_chat
feature-extraction
mathematics
reasoning
multi-modal-qa
math-qa
figure-qa
geometry-qa
math-word-problem
textbook-qa
vqa
geometry-diagram
synthetic-scene
chart
plot
scientific-figure
table
function-plot
abstract-scene
puzzle-test
document-image
science
conversational
custom_code
Model card Files Community
1
FigCodifier / README.md
nielsr's picture
nielsr HF Staff
Update model card with paper link and Github README content
83a3255 verified
preview code
|
raw
history blame
11.3 kB
metadata
base_model:
  - OpenGVLab/InternVL2-8B
language:
  - en
library_name: transformers
license: apache-2.0
metrics:
  - accuracy
pipeline_tag: image-text-to-text

MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning

Repo: https://github.com/mathllm/MathCoder

Paper: https://huggingface.co/papers/2505.10557

Introduction

We introduce MathCoder-VL, a series of open-source large multimodal models (LMMs) specifically tailored for general math problem-solving. We also introduce FigCodifier-8B, an image-to-code model.

Base Model Ours
Mini-InternVL-Chat-2B-V1-5 MathCoder-VL-2B
InternVL2-8B MathCoder-VL-8B

Usage

For training and inference code, please refer to InternVL.

Prompt for TikZ Code Generation 

<image>
Please generate the corresponding TikZ code that accurately represents the visual elements in the image. TikZ is a powerful tool for creating vector graphics within LaTeX documents. Your generated code should be precise, well-structured, and should recreate the image as faithfully as possible.

Prompt for Python Code Generation 

Please provide the Python code needed to reproduce this image.
<image>

Motivation

Construction of FigCodifier

Construction of MathCoder-VL

Performance

Citation

Please cite the paper if you use our data, model or code.

@inproceedings{
wang2025mathcodervl,
title={MathCoder-{VL}: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning},
author={Ke Wang and Junting Pan and Linda Wei and Aojun Zhou and Weikang Shi and Zimu Lu and Han Xiao and Yunqiao Yang and Houxing Ren and Mingjie Zhan and Hongsheng Li},
booktitle={The 63rd Annual Meeting of the Association for Computational Linguistics},
year={2025},
url={https://openreview.net/forum?id=nuvtX1imAb}
}

@inproceedings{
lu2025mathcoder2,
title={MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code},
author={Zimu Lu and Aojun Zhou and Ke Wang and Houxing Ren and Weikang Shi and Junting Pan and Mingjie Zhan and Hongsheng Li},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=1Iuw1jcIrf}
}

@inproceedings{
wang2024mathcoder,
title={MathCoder: Seamless Code Integration in {LLM}s for Enhanced Mathematical Reasoning},
author={Ke Wang and Houxing Ren and Aojun Zhou and Zimu Lu and Sichun Luo and Weikang Shi and Renrui Zhang and Linqi Song and Mingjie Zhan and Hongsheng Li},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=z8TW0ttBPp}
}

MathCoder

This repo is for "MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning"

🔥🔥🔥 We release "MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning"

💥 News 💥

Datasets and Models

Our models are available at Hugging Face now.

🤗 MathCodeInstruct Dataset

Base Model: Llama-2 Base Model: Code Llama
MathCoder-L-7B MathCoder-CL-7B
MathCoder-L-13B MathCoder-CL-34B

Training Data

The models are trained on the MathCodeInstruct Dataset.


Introduction

The recently released GPT-4 Code Interpreter has demonstrated remarkable proficiency in solving challenging math problems, primarily attributed to its ability to seamlessly reason with natural language, generate code, execute code, and continue reasoning based on the execution output. In this paper, we present a method to fine-tune open-source language models, enabling them to use code for modeling and deriving math equations and, consequently, enhancing their mathematical reasoning abilities.

We propose a method of generating novel and high-quality datasets with math problems and their code-based solutions, referred to as MathCodeInstruct. Each solution interleaves natural language, code, and execution results.

We also introduce a customized supervised fine-tuning and inference approach. This approach yields the MathCoder models, a family of models capable of generating code-based solutions for solving challenging math problems.

Impressively, the MathCoder models achieve state-of-the-art scores among open-source LLMs on the MATH (45.2%) and GSM8K (83.9%) datasets, substantially outperforming other open-source alternatives. Notably, the MathCoder model not only surpasses ChatGPT-3.5 and PaLM-2 on GSM8K and MATH but also outperforms GPT-4 on the competition-level MATH dataset. The proposed dataset and models will be released upon acceptance.

Usage

Model deployment

We use the Text Generation Inference (TGI) to deploy our MathCoders for response generation. TGI is a toolkit for deploying and serving Large Language Models (LLMs). TGI enables high-performance text generation for the most popular open-source LLMs, including Llama, Falcon, StarCoder, BLOOM, GPT-NeoX, and T5. Your can follow the guide here. After successfully installing TGI, you can easily deploy the models using deploy.sh.

model_path="local model path"

max_input_tokens=1536
max_total_tokens=2048

set -x
hostname -I # print the host ip

text-generation-launcher --port 8000 \
--max-batch-prefill-tokens ${max_input_tokens} \
--max-input-length ${max_input_tokens} \
--max-total-tokens ${max_total_tokens} \
--model-id ${model_path}

Inference

We provide a script for inference. Just replace the ip and port in the following command correctly with the API forwarded by TGI like:

python inference.py --pnum=4 --outdir=outs/debug --ip=10.119.18.159 --port=8001 --type=test --dataset=GSM8K

We also open-source all of the model outputs from our MathCoders under the outs/ folder.

Evaluation

To evaluate the predicted answer, run the following command:

python evaluate.py outs/MathCoder-L-7b/MATH/MATH_test_result-20230917-2026.jsonl

Citation

Please cite the paper if you use our data, model or code.

@inproceedings{
wang2025mathcodervl,
title={MathCoder-{VL}: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning},
author={Ke Wang and Junting Pan and Linda Wei and Aojun Zhou and Weikang Shi and Zimu Lu and Han Xiao and Yunqiao Yang and Houxing Ren and Mingjie Zhan and Hongsheng Li},
booktitle={The 63rd Annual Meeting of the Association for Computational Linguistics},
year={2025},
url={https://openreview.net/forum?id=nuvtX1imAb}
}

@inproceedings{
lu2025mathcoder2,
title={MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code},
author={Zimu Lu and Aojun Zhou and Ke Wang and Houxing Ren and Weikang Shi and Junting Pan and Mingjie Zhan and Hongsheng Li},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=1Iuw1jcIrf}
}

@inproceedings{
wang2024mathcoder,
title={MathCoder: Seamless Code Integration in {LLM}s for Enhanced Mathematical Reasoning},
author={Ke Wang and Houxing Ren and Aojun Zhou and Zimu Lu and Sichun Luo and Weikang Shi and Renrui Zhang and Linqi Song and Mingjie Zhan and Hongsheng Li},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=z8TW0ttBPp}
}

@inproceedings{
zhou2024solving,
title={Solving Challenging Math Word Problems Using {GPT}-4 Code Interpreter with Code-based Self-Verification},
author={Aojun Zhou and Ke Wang and Zimu Lu and Weikang Shi and Sichun Luo and Zipeng Qin and Shaoqing Lu and Anya Jia and Linqi Song and Mingjie Zhan and Hongsheng Li},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=c8McWs4Av0}
}