Update README.md
Browse files
README.md
CHANGED
|
@@ -18,16 +18,17 @@ base_model:
|
|
| 18 |
|
| 19 |
[ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
|
| 20 |
|
| 21 |
-
**CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for
|
| 22 |
|
| 23 |
## Evaluations
|
| 24 |
|
| 25 |
-
ProX models are evaluated
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
| | ArC-c | ARC-e | CSQA | HellaS | MMLU | OBQA | PiQA | SIQA | WinoG | SciQ | AVG |
|
| 28 |
-
|-----------------------|-------|-------|-------|-----------|-------|-------|-------|-------|-------|-------|------|
|
| 29 |
-
| raw | 26.1 | 44.3 | 29.7 | 39.1 | 27.3 | 29.2 | 66.9 | 39.0 | 52.0 | 67.4 | 42.1 |
|
| 30 |
-
| ours | 26.4 | 51.9 | 30.9 | 42.4 | 29.4 | 31.6 | 67.9 | 40.0 | 52.2 | 73.5 | 44.6 |
|
| 31 |
|
| 32 |
### Citation
|
| 33 |
```
|
|
|
|
| 18 |
|
| 19 |
[ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
|
| 20 |
|
| 21 |
+
**CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for **10**B tokens.
|
| 22 |
|
| 23 |
## Evaluations
|
| 24 |
|
| 25 |
+
ProX models are evaluated on 9 common math reasoning benchmarks.
|
| 26 |
+
|
| 27 |
+
| Model | asdiv | gsm8k | mathqa | mawps | minerva_math | mmlu_stem | sat_math | svamp | tabmwp | average |
|
| 28 |
+
|----------------------- |---------- |---------- |---------- |---------- |-------------- |----------- |---------- |---------- |---------- |---------- |
|
| 29 |
+
| CodeLlama-7B | 50.7 | 11.8 | 14.3 | 62.6 | 5.0 | 20.4 | 21.9 | 44.2 | 30.6 | 29.1 |
|
| 30 |
+
| CodeLlama-7B-ProXMath | **67.9** | **35.6** | **38.9** | **82.7** | **17.6** | **42.6** | **62.5** | **55.8** | **41.3** | **49.4** |
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
### Citation
|
| 34 |
```
|