README.md

metadata

license: apache-2.0
tags:
  - code
  - mistral

Exllama v2 Quantization of Mistral-7B-codealpaca-lora

Using turboderp's ExLlamaV2 v0.0.6 for quantization.

Conversion done using evol-codealpaca-v1.parquet as calibration dataset.

Original model: https://huggingface.co/Nondzu/Mistral-7B-codealpaca-lora

6.0 bits per weight

8.0 bits per weight

4.0 bits per weight

3.5 bits per weight