Hugging Face's logo

cycloneboy
/
CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct

Text Generation
Transformers
Safetensors
qwen2
text-to-sql
reinforcement-learning
conversational
text-generation-inference
Model card Files Community
1

Improve model card: Update paper link, add GitHub link and usage example

#1
by nielsr HF Staff - opened
base: refs/heads/main
←
from: refs/pr/1
Discussion Files changed
+79
-20
Files changed (1) hide show
  1. README.md +79 -20
README.md CHANGED
@@ -1,19 +1,19 @@
1
  ---
2
- pipeline_tag: text-generation
3
  library_name: transformers
4
  license: cc-by-nc-4.0
 
5
  tags:
6
  - text-to-sql
7
  - reinforcement-learning
8
  ---
9
 
10
-
11
  # SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
12
 
13
  ### Important Links
14
 
15
- 📖[Arxiv Paper](https://arxiv.org/abs/2507.22478) |
16
- 🤗[HuggingFace](https://huggingface.co/collections/cycloneboy/slm-sql-688b02f99f958d7a417658dc) |
 
17
  🤖[ModelScope](https://modelscope.cn/collections/SLM-SQL-624bb6a60e9643) |
18
 
19
  ## News
@@ -42,6 +42,65 @@ tags:
42
 
43
  <img src="https://raw.githubusercontent.com/CycloneBoy/slm_sql/main/data/image/slmsql_framework.png" height="500" alt="slmsql_framework">
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ### Main Results
46
 
47
  <img src="https://raw.githubusercontent.com/CycloneBoy/slm_sql/main/data/image/slmsql_bird_result.png" height="500" alt="slm_sql_result">
@@ -59,25 +118,25 @@ Performance Comparison of different Text-to-SQL methods on BIRD dev and test dat
59
 
60
  | **Model** | Base Model | Train Method | Modelscope | HuggingFace |
61
  |------------------------------------------|------------------------------|--------------|---------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
62
- | SLM-SQL-Base-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-0.5B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-0.5B) |
63
- | SLM-SQL-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-0.5B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-0.5B) |
64
- | CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct) |
65
- | SLM-SQL-Base-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1.5B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1.5B) |
66
- | SLM-SQL-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-1.5B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-1.5B) |
67
- | CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct) |
68
- | SLM-SQL-Base-0.6B | Qwen3-0.6B | SFT | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-0.6B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-0.6B) |
69
- | SLM-SQL-0.6B | Qwen3-0.6B | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-0.6B) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-0.6B) |
70
- | SLM-SQL-Base-1.3B | deepseek-coder-1.3b-instruct | SFT | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1.3B ) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1.3B ) |
71
- | SLM-SQL-1.3B | deepseek-coder-1.3b-instruct | SFT + GRPO | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-1.3B ) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-1.3B ) |
72
- | SLM-SQL-Base-1B | Llama-3.2-1B-Instruct | SFT | [🤖 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1B ) | [🤗 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1B ) |
73
 
74
  ## Dataset
75
 
76
- | **Dataset** | Modelscope | HuggingFace |
77
- |----------------------------|------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
78
- | SynsQL-Think-916k | [🤖 Modelscope](https://modelscope.cn/datasets/cycloneboy/SynsQL-Think-916k) | [🤗 HuggingFace](https://huggingface.co/datasets/cycloneboy/SynsQL-Think-916k) |
79
- | SynsQL-Merge-Think-310k | [🤖 Modelscope](https://modelscope.cn/datasets/cycloneboy/SynsQL-Merge-Think-310k) | [🤗 HuggingFace](https://huggingface.co/datasets/cycloneboy/SynsQL-Merge-Think-310k) |
80
- | bird train and dev dataset | [🤖 Modelscope](https://modelscope.cn/datasets/cycloneboy/bird_train) | [🤗 HuggingFace](https://huggingface.co/datasets/cycloneboy/bird_train) |
81
 
82
  ## TODO
83
 
 
1
  ---
 
2
  library_name: transformers
3
  license: cc-by-nc-4.0
4
+ pipeline_tag: text-generation
5
  tags:
6
  - text-to-sql
7
  - reinforcement-learning
8
  ---
9
 
 
10
  # SLM-SQL: An Exploration of Small Language Models for Text-to-SQL
11
 
12
  ### Important Links
13
 
14
+ 📖[Paper](https://huggingface.co/papers/2507.22478) |
15
+ \ud83d\udcbb[GitHub Repository](https://github.com/CycloneBoy/slm_sql) |
16
+ 🤗[HuggingFace Collection](https://huggingface.co/collections/cycloneboy/slm-sql-688b02f99f958d7a417658dc) |
17
  🤖[ModelScope](https://modelscope.cn/collections/SLM-SQL-624bb6a60e9643) |
18
 
19
  ## News
 
42
 
43
  <img src="https://raw.githubusercontent.com/CycloneBoy/slm_sql/main/data/image/slmsql_framework.png" height="500" alt="slmsql_framework">
44
 
45
+ ## How to use
46
+ You can use the model with the `transformers` library for Text-to-SQL tasks. Make sure you have `transformers` and `torch` installed.
47
+
48
+ ```python
49
+ import torch
50
+ from transformers import AutoTokenizer, AutoModelForCausalLM
51
+
52
+ model_name = "cycloneboy/SLM-SQL-0.5B" # Or any other SLM-SQL model from the collection
53
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
54
+ model = AutoModelForCausalLM.from_pretrained(
55
+ model_name,
56
+ torch_dtype=torch.bfloat16,
57
+ device_map="auto"
58
+ )
59
+
60
+ # Example for Text-to-SQL
61
+ db_schema = """
62
+ CREATE TABLE Employee (
63
+ employee_id INTEGER PRIMARY KEY,
64
+ name TEXT,
65
+ department TEXT,
66
+ salary INTEGER
67
+ );
68
+ CREATE TABLE Department (
69
+ department_id INTEGER PRIMARY KEY,
70
+ name TEXT,
71
+ location TEXT
72
+ );
73
+ """
74
+ question = "What are the names of employees in the 'Sales' department earning more than 50000?"
75
+ prompt = f"Given the database schema:
76
+ {db_schema}
77
+
78
+ Translate the following question to SQL: {question}"
79
+
80
+ messages = [
81
+ {"role": "system", "content": "You are a helpful assistant that translates natural language questions into SQL queries."},
82
+ {"role": "user", "content": prompt}
83
+ ]
84
+
85
+ input_ids = tokenizer.apply_chat_template(
86
+ messages,
87
+ add_generation_prompt=True,
88
+ return_tensors="pt"
89
+ ).to(model.device)
90
+
91
+ outputs = model.generate(
92
+ input_ids,
93
+ max_new_tokens=256,
94
+ do_sample=True,
95
+ temperature=0.7,
96
+ top_k=50,
97
+ top_p=0.95
98
+ )
99
+ response = tokenizer.decode(outputs[0][input_ids.shape[-1]:], skip_special_tokens=True)
100
+ print(response)
101
+ # Expected output similar to: SELECT name FROM Employee WHERE department = 'Sales' AND salary > 50000
102
+ ```
103
+
104
  ### Main Results
105
 
106
  <img src="https://raw.githubusercontent.com/CycloneBoy/slm_sql/main/data/image/slmsql_bird_result.png" height="500" alt="slm_sql_result">
 
118
 
119
  | **Model** | Base Model | Train Method | Modelscope | HuggingFace |
120
  |------------------------------------------|------------------------------|--------------|---------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
121
+ | SLM-SQL-Base-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-0.5B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-0.5B) |
122
+ | SLM-SQL-0.5B | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-0.5B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-0.5B) |
123
+ | CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct | Qwen2.5-Coder-0.5B-Instruct | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct) |
124
+ | SLM-SQL-Base-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1.5B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1.5B) |
125
+ | SLM-SQL-1.5B | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-1.5B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-1.5B) |
126
+ | CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct | Qwen2.5-Coder-1.5B-Instruct | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/CscSQL-Merge-Qwen2.5-Coder-1.5B-Instruct) |
127
+ | SLM-SQL-Base-0.6B | Qwen3-0.6B | SFT | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-0.6B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-0.6B) |
128
+ | SLM-SQL-0.6B | Qwen3-0.6B | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-0.6B) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-0.6B) |
129
+ | SLM-SQL-Base-1.3B | deepseek-coder-1.3b-instruct | SFT | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1.3B ) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1.3B ) |
130
+ | SLM-SQL-1.3B | deepseek-coder-1.3b-instruct | SFT + GRPO | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-1.3B ) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-1.3B ) |
131
+ | SLM-SQL-Base-1B | Llama-3.2-1B-Instruct | SFT | [\ud83e\udd16 Modelscope](https://modelscope.cn/models/cycloneboy/SLM-SQL-Base-1B ) | [\ud83e\udd17 HuggingFace](https://huggingface.co/cycloneboy/SLM-SQL-Base-1B ) |
132
 
133
  ## Dataset
134
 
135
+ | **Dataset** | Modelscope | HuggingFace |
136
+ |---|---|---|
137
+ | SynsQL-Think-916k | [\ud83e\udd16 Modelscope](https://modelscope.cn/datasets/cycloneboy/SynsQL-Think-916k) | [\ud83e\udd17 HuggingFace](https://huggingface.co/datasets/cycloneboy/SynsQL-Think-916k) |
138
+ | SynsQL-Merge-Think-310k | [\ud83e\udd16 Modelscope](https://modelscope.cn/datasets/cycloneboy/SynsQL-Merge-Think-310k) | [\ud83e\udd17 HuggingFace](https://huggingface.co/datasets/cycloneboy/SynsQL-Merge-Think-310k) |
139
+ | bird train and dev dataset | [\ud83e\udd16 Modelscope](https://modelscope.cn/datasets/cycloneboy/bird_train) | [\ud83e\udd17 HuggingFace](https://huggingface.co/datasets/cycloneboy/bird_train) |
140
 
141
  ## TODO
142