| license: apache-2.0 | |
| --- | |
| # GreenBit Yi | |
| This is GreenBitAI's pretrained **2-bit** Yi 6B model with extreme compression yet still strong performance. | |
| Please refer to our [Github page](https://github.com/GreenBitAI/low_bit_llama) for the code to run the model and more information. | |
| ## Model Description | |
| - **Developed by:** [GreenBitAI](https://github.com/GreenBitAI) | |
| - **Model type:** Causal (Llama 2/Yi 6B) | |
| - **Language(s) (NLP):** English | |
| - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), [Llama 2 license agreement](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) | |
| ## Zero-Shot Evaluation | |
| | Task | Metric | FP16 | Yi-6B w4a16g32 | Yi-6B w2a16g8 | Yi-6B w2a16g32| | |
| |-----------------|--------|-------|-------------|----------------------|-----------------------| | |
| | Openbookqa | acc | 0.314 | 0.324 | 0.26 | 0.228 | | |
| | | ac_norm| 0.408 | 0.42 | 0.394 | 0.352 | | |
| | arc_challenge | acc | 0.462 | 0.4573 | 0.4082 | 0.337 | | |
| | | ac_norm| 0.504 | 0.483 | 0.4249 | 0.3523 | | |
| | hellawswag | acc | 0.553 | 0.5447 | 0.5083 | 0.4326 | | |
| | | ac_norm| 0.749 | 0.7327 | 0.6909 | 0.58 | | |
| | piqa | acc | 0.777 | 0.7709 | 0.7535 | 0.7051 | | |
| | | ac_norm| 0.787 | 0.7894 | 0.7655 | 0.7143 | | |
| | arc_easy | acc | 0.777 | 0.7697 | 0.7373 | 0.6523 | | |
| | | ac_norm| 0.774 | 0.7659 | 0.7314 | 0.6115 | | |
| | Winogrande | acc | 0.707 | 0.7095 | 0.6803 | 0.6219 | | |
| | boolq | acc | 0.755 | 0.7648 | 0.7507 | 0.732 | | |
| | truthfulqa_mc | mc1 | 0.29 | 0.2729 | 0.2753 | 0.219 | | |
| | | mc2 | 0.419 | 0.4033 | 0.4156 | 0.3479 | | |
| | anli_r1 | acc | 0.423 | 0.416 | 0.383 | 0.38 | | |
| | anli_r2 | acc | 0.409 | 0.409 | 0.387 | 0.374 | | |
| | anli_r3 | acc | 0.411 | 0.393 | 0.38 | 0.3475 | | |
| | wic | acc | 0.529 | 0.545 | 0.515 | 0.5 | | |
| | rte | acc | 0.685 | 0.7039 | 0.7436 | 0.6787 | | |
| | record | f1 | 0.904 | 0.9011 | 0.8906 | 0.8521 | | |
| | | em | 0.8962| 0.8927 | 0.8819 | 0.8429 | | |
| | Average | | 0.596 | 0.5937 | 0.5703 | 0.517 | | |
| | wikitext2 (2048)| ppl | 5.841 | 6.01 | 6.57 | 8.06 | | |
| | ptb (2048) | ppl | 18.93 | 19.76 | 25.9 | 35.3 | | |
| | Model Size | GiB | 12.12 | 3.76 | 3.09 | 2.5 | | |