|
|
--- |
|
|
license: cc-by-sa-4.0 |
|
|
metrics: |
|
|
- accuracy |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- code |
|
|
--- |
|
|
# Exllama v2 defog/llama-3-sqlcoder-8b |
|
|
|
|
|
Using <a href="https://github.com/turboderp/exllamav2/releases/tag/v0.0.21">turboderp's ExLlamaV2 v0.0.21</a> for quantization. |
|
|
|
|
|
<b>The "main" branch only contains the measurement.json, download one of the other branches for the model</b> |
|
|
|
|
|
Each branch contains an individual bits per weight, with the main one containing only the meaurement.json for further conversions. |
|
|
|
|
|
Original model: <a href="https://huggingface.co/defog/llama-3-sqlcoder-8b">defog/llama-3-sqlcoder-8b</a><br> |
|
|
Calibration dataset is based on: <a href="https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction">m-a-p/CodeFeedback-Filtered-Instruction</a> |
|
|
|
|
|
## Available sizes |
|
|
|
|
|
| Branch | Bits | lm_head bits | Description | |
|
|
| ----- | ---- | ------- | ------------ | |
|
|
| [8_0](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/8_0) | 8.0 | 8.0 | Maximum quality that ExLlamaV2 can produce, near unquantized performance. | |
|
|
| [6_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/6_5) | 6.5 | 8.0 | Very similar to 8.0, good tradeoff of size vs performance, **recommended**. | |
|
|
| [5_5](https://huggingface.co/Apel-sin/llama-3-sqlcoder-8B-exl2/tree/5_5) | 5.5 | 8.0 | Slightly lower quality vs 6.5, but usable on 8GB cards. | |
|
|
|
|
|
A capable language model for text to SQL generation for Postgres, Redshift and Snowflake that is on-par with the most capable generalist frontier models. |
|
|
|
|
|
 |
|
|
|
|
|
## Model Description |
|
|
|
|
|
Developed by: Defog, Inc |
|
|
Model type: [Text to SQL] |
|
|
License: [CC-by-SA-4.0] |
|
|
Finetuned from model: [Meta-Llama-3-8B-Instruct] |
|
|
|
|
|
## Demo Page |
|
|
[https://defog.ai/sqlcoder-demo/](https://defog.ai/sqlcoder-demo/) |
|
|
|
|
|
## Ideal prompt and inference parameters |
|
|
Set temperature to 0, and do not do sampling. |
|
|
|
|
|
### Prompt |
|
|
``` |
|
|
<|begin_of_text|><|start_header_id|>user<|end_header_id|> |
|
|
|
|
|
Generate a SQL query to answer this question: `{user_question}` |
|
|
{instructions} |
|
|
|
|
|
DDL statements: |
|
|
{create_table_statements}<|eot_id|><|start_header_id|>assistant<|end_header_id|> |
|
|
|
|
|
The following SQL query best answers the question `{user_question}`: |
|
|
```sql |
|
|
|
|
|
``` |
|
|
|
|
|
## Evaluation |
|
|
This model was evaluated on SQL-Eval, a PostgreSQL based evaluation framework developed by Defog for testing and alignment of model capabilities. |
|
|
|
|
|
You can read more about the methodology behind SQLEval [here](https://defog.ai/blog/open-sourcing-sqleval/). |
|
|
|
|
|
## Contact |
|
|
Contact us on X at [@defogdata](https://twitter.com/defogdata), or on email at [email protected] |
