apple/DiffuCoder-7B-Base

DiffuCoder-7B-Base

The DiffuCoder-7B-Base model is our foundational masked diffusion LLM for code generation.

• Training recipe: Using DiffuLLaMA's adaptation approach, trained on a large corpus of code: with Stage 1 65B tokens and Stage 2 65B tokens.

• Benchmarks: Strong baseline performance on HumanEval, MBPP and BigCodeBench.

More details and usage examples:

• Paper: DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation

• GitHub: https://github.com/apple/ml-diffucoder

Acknowledgement

To power this HuggingFace model release, we reuse Dream's modeling architecture and generation utils.