This is a int8_float16 quantized version of WizardLM/WizardCoder-Python-7B-V1.0, quantized using ctranslate2 (see inference instructions there).
The license/caveats/intended usage is the same as the original model.
The quality of its output may have
been negatively affected by the quantization process.
The command run to quantize the model was:
ct2-transformers-converter --model ./models-hf/WizardLM/WizardCoder-Python-7B-V1.0 --quantization int8_float16 --output_dir ./models-ct/WizardLM/WizardCoder-Python-7B-V1.0-ct2-int8_float16
The quantization was run on a 'high-mem', CPU only (8 core, 51GB) colab instance and took approximately 10 minutes.
- Downloads last month
- 0
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support