Update README.md
Browse files
README.md
CHANGED
@@ -169,18 +169,6 @@ Hello! As an AI, I don't have consciousness in the way humans do, but I am fully
|
|
169 |
|
170 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
171 |
|
172 |
-
Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
173 |
-
|
174 |
-
## baseline
|
175 |
-
```Shell
|
176 |
-
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8
|
177 |
-
```
|
178 |
-
|
179 |
-
## int8 dynamic activation and int4 weight quantization (8da4w)
|
180 |
-
```Shell
|
181 |
-
lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --tasks hellaswag --device cuda:0 --batch_size 8
|
182 |
-
```
|
183 |
-
|
184 |
| Benchmark | | |
|
185 |
|----------------------------------|----------------|---------------------------|
|
186 |
| | Phi-4-mini-ins | Phi-4-mini-instruct-8da4w |
|
@@ -203,6 +191,21 @@ lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --t
|
|
203 |
| Mathqa (0-shot) | 42.31 | 36.95 |
|
204 |
| **Overall** | 55.35 | 48.45 |
|
205 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
206 |
|
207 |
# Exporting to ExecuTorch
|
208 |
|
|
|
169 |
|
170 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
171 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
172 |
| Benchmark | | |
|
173 |
|----------------------------------|----------------|---------------------------|
|
174 |
| | Phi-4-mini-ins | Phi-4-mini-instruct-8da4w |
|
|
|
191 |
| Mathqa (0-shot) | 42.31 | 36.95 |
|
192 |
| **Overall** | 55.35 | 48.45 |
|
193 |
|
194 |
+
<details>
|
195 |
+
<summary> Reproduce Model Quality Results </summary>
|
196 |
+
|
197 |
+
Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
198 |
+
|
199 |
+
## baseline
|
200 |
+
```Shell
|
201 |
+
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8
|
202 |
+
```
|
203 |
+
|
204 |
+
## int8 dynamic activation and int4 weight quantization (8da4w)
|
205 |
+
```Shell
|
206 |
+
lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --tasks hellaswag --device cuda:0 --batch_size 8
|
207 |
+
```
|
208 |
+
</details>
|
209 |
|
210 |
# Exporting to ExecuTorch
|
211 |
|