Files changed (1) hide show
  1. README.md +210 -198
README.md CHANGED
@@ -1,198 +1,210 @@
1
- ---
2
- language:
3
- - en
4
- license: apache-2.0
5
- library_name: transformers
6
- tags:
7
- - role-play
8
- - fine-tuned
9
- - qwen2.5
10
- base_model:
11
- - Qwen/Qwen2.5-14B-Instruct
12
- pipeline_tag: text-generation
13
- model-index:
14
- - name: oxy-1-small
15
- results:
16
- - task:
17
- type: text-generation
18
- name: Text Generation
19
- dataset:
20
- name: IFEval (0-Shot)
21
- type: HuggingFaceH4/ifeval
22
- args:
23
- num_few_shot: 0
24
- metrics:
25
- - type: inst_level_strict_acc and prompt_level_strict_acc
26
- value: 62.45
27
- name: strict accuracy
28
- source:
29
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
30
- name: Open LLM Leaderboard
31
- - task:
32
- type: text-generation
33
- name: Text Generation
34
- dataset:
35
- name: BBH (3-Shot)
36
- type: BBH
37
- args:
38
- num_few_shot: 3
39
- metrics:
40
- - type: acc_norm
41
- value: 41.18
42
- name: normalized accuracy
43
- source:
44
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
45
- name: Open LLM Leaderboard
46
- - task:
47
- type: text-generation
48
- name: Text Generation
49
- dataset:
50
- name: MATH Lvl 5 (4-Shot)
51
- type: hendrycks/competition_math
52
- args:
53
- num_few_shot: 4
54
- metrics:
55
- - type: exact_match
56
- value: 18.28
57
- name: exact match
58
- source:
59
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
60
- name: Open LLM Leaderboard
61
- - task:
62
- type: text-generation
63
- name: Text Generation
64
- dataset:
65
- name: GPQA (0-shot)
66
- type: Idavidrein/gpqa
67
- args:
68
- num_few_shot: 0
69
- metrics:
70
- - type: acc_norm
71
- value: 16.22
72
- name: acc_norm
73
- source:
74
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
75
- name: Open LLM Leaderboard
76
- - task:
77
- type: text-generation
78
- name: Text Generation
79
- dataset:
80
- name: MuSR (0-shot)
81
- type: TAUR-Lab/MuSR
82
- args:
83
- num_few_shot: 0
84
- metrics:
85
- - type: acc_norm
86
- value: 16.28
87
- name: acc_norm
88
- source:
89
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
90
- name: Open LLM Leaderboard
91
- - task:
92
- type: text-generation
93
- name: Text Generation
94
- dataset:
95
- name: MMLU-PRO (5-shot)
96
- type: TIGER-Lab/MMLU-Pro
97
- config: main
98
- split: test
99
- args:
100
- num_few_shot: 5
101
- metrics:
102
- - type: acc
103
- value: 44.45
104
- name: accuracy
105
- source:
106
- url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
107
- name: Open LLM Leaderboard
108
- ---
109
-
110
- ![Oxy 1 Small](https://cdn-uploads.huggingface.co/production/uploads/64fb80c8bb362cbf2ff96c7e/tTIVIblPUbTYnlvHQQjXB.png)
111
-
112
- ## Introduction
113
-
114
- **Oxy 1 Small** is a fine-tuned version of the [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen/Qwen2.5-14B-Instruct) language model, specialized for **role-play** scenarios. Despite its small size, it delivers impressive performance in generating engaging dialogues and interactive storytelling.
115
-
116
- Developed by **Oxygen (oxyapi)**, with contributions from **TornadoSoftwares**, Oxy 1 Small aims to provide an accessible and efficient language model for creative and immersive role-play experiences.
117
-
118
- ## Model Details
119
-
120
- - **Model Name**: Oxy 1 Small
121
- - **Model ID**: [oxyapi/oxy-1-small](https://huggingface.co/oxyapi/oxy-1-small)
122
- - **Base Model**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
123
- - **Model Type**: Chat Completions
124
- - **Prompt Format**: ChatML
125
- - **License**: Apache-2.0
126
- - **Language**: English
127
- - **Tokenizer**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
128
- - **Max Input Tokens**: 32,768
129
- - **Max Output Tokens**: 8,192
130
-
131
- ### Features
132
-
133
- - **Fine-tuned for Role-Play**: Specially trained to generate dynamic and contextually rich role-play dialogues.
134
- - **Efficient**: Compact model size allows for faster inference and reduced computational resources.
135
- - **Parameter Support**:
136
- - `temperature`
137
- - `top_p`
138
- - `top_k`
139
- - `frequency_penalty`
140
- - `presence_penalty`
141
- - `max_tokens`
142
-
143
- ### Metadata
144
-
145
- - **Owned by**: Oxygen (oxyapi)
146
- - **Contributors**: TornadoSoftwares
147
- - **Description**: A Qwen/Qwen2.5-14B-Instruct fine-tune for role-play trained on custom datasets
148
-
149
- ## Usage
150
-
151
- To utilize Oxy 1 Small for text generation in role-play scenarios, you can load the model using the Hugging Face Transformers library:
152
-
153
- ```python
154
- from transformers import AutoModelForCausalLM, AutoTokenizer
155
-
156
- tokenizer = AutoTokenizer.from_pretrained("oxyapi/oxy-1-small")
157
- model = AutoModelForCausalLM.from_pretrained("oxyapi/oxy-1-small")
158
-
159
- prompt = "You are a wise old wizard in a mystical land. A traveler approaches you seeking advice."
160
- inputs = tokenizer(prompt, return_tensors="pt")
161
- outputs = model.generate(**inputs, max_length=500)
162
- response = tokenizer.decode(outputs[0], skip_special_tokens=True)
163
- print(response)
164
- ```
165
-
166
- ## Performance
167
-
168
- Performance benchmarks for Oxy 1 Small are not available at this time. Future updates may include detailed evaluations on relevant datasets.
169
-
170
- ## License
171
-
172
- This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
173
-
174
- ## Citation
175
-
176
- If you find Oxy 1 Small useful in your research or applications, please cite it as:
177
-
178
- ```
179
- @misc{oxy1small2024,
180
- title={Oxy 1 Small: A Fine-Tuned Qwen2.5-14B-Instruct Model for Role-Play},
181
- author={Oxygen (oxyapi)},
182
- year={2024},
183
- howpublished={\url{https://huggingface.co/oxyapi/oxy-1-small}},
184
- }
185
- ```
186
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
187
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_oxyapi__oxy-1-small)
188
-
189
- | Metric |Value|
190
- |-------------------|----:|
191
- |Avg. |33.14|
192
- |IFEval (0-Shot) |62.45|
193
- |BBH (3-Shot) |41.18|
194
- |MATH Lvl 5 (4-Shot)|18.28|
195
- |GPQA (0-shot) |16.22|
196
- |MuSR (0-shot) |16.28|
197
- |MMLU-PRO (5-shot) |44.45|
198
-
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zho
4
+ - eng
5
+ - fra
6
+ - spa
7
+ - por
8
+ - deu
9
+ - ita
10
+ - rus
11
+ - jpn
12
+ - kor
13
+ - vie
14
+ - tha
15
+ - ara
16
+ license: apache-2.0
17
+ library_name: transformers
18
+ tags:
19
+ - role-play
20
+ - fine-tuned
21
+ - qwen2.5
22
+ base_model:
23
+ - Qwen/Qwen2.5-14B-Instruct
24
+ pipeline_tag: text-generation
25
+ model-index:
26
+ - name: oxy-1-small
27
+ results:
28
+ - task:
29
+ type: text-generation
30
+ name: Text Generation
31
+ dataset:
32
+ name: IFEval (0-Shot)
33
+ type: HuggingFaceH4/ifeval
34
+ args:
35
+ num_few_shot: 0
36
+ metrics:
37
+ - type: inst_level_strict_acc and prompt_level_strict_acc
38
+ value: 62.45
39
+ name: strict accuracy
40
+ source:
41
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
42
+ name: Open LLM Leaderboard
43
+ - task:
44
+ type: text-generation
45
+ name: Text Generation
46
+ dataset:
47
+ name: BBH (3-Shot)
48
+ type: BBH
49
+ args:
50
+ num_few_shot: 3
51
+ metrics:
52
+ - type: acc_norm
53
+ value: 41.18
54
+ name: normalized accuracy
55
+ source:
56
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
57
+ name: Open LLM Leaderboard
58
+ - task:
59
+ type: text-generation
60
+ name: Text Generation
61
+ dataset:
62
+ name: MATH Lvl 5 (4-Shot)
63
+ type: hendrycks/competition_math
64
+ args:
65
+ num_few_shot: 4
66
+ metrics:
67
+ - type: exact_match
68
+ value: 18.28
69
+ name: exact match
70
+ source:
71
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
72
+ name: Open LLM Leaderboard
73
+ - task:
74
+ type: text-generation
75
+ name: Text Generation
76
+ dataset:
77
+ name: GPQA (0-shot)
78
+ type: Idavidrein/gpqa
79
+ args:
80
+ num_few_shot: 0
81
+ metrics:
82
+ - type: acc_norm
83
+ value: 16.22
84
+ name: acc_norm
85
+ source:
86
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
87
+ name: Open LLM Leaderboard
88
+ - task:
89
+ type: text-generation
90
+ name: Text Generation
91
+ dataset:
92
+ name: MuSR (0-shot)
93
+ type: TAUR-Lab/MuSR
94
+ args:
95
+ num_few_shot: 0
96
+ metrics:
97
+ - type: acc_norm
98
+ value: 16.28
99
+ name: acc_norm
100
+ source:
101
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
102
+ name: Open LLM Leaderboard
103
+ - task:
104
+ type: text-generation
105
+ name: Text Generation
106
+ dataset:
107
+ name: MMLU-PRO (5-shot)
108
+ type: TIGER-Lab/MMLU-Pro
109
+ config: main
110
+ split: test
111
+ args:
112
+ num_few_shot: 5
113
+ metrics:
114
+ - type: acc
115
+ value: 44.45
116
+ name: accuracy
117
+ source:
118
+ url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=oxyapi/oxy-1-small
119
+ name: Open LLM Leaderboard
120
+ ---
121
+
122
+ ![Oxy 1 Small](https://cdn-uploads.huggingface.co/production/uploads/64fb80c8bb362cbf2ff96c7e/tTIVIblPUbTYnlvHQQjXB.png)
123
+
124
+ ## Introduction
125
+
126
+ **Oxy 1 Small** is a fine-tuned version of the [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen/Qwen2.5-14B-Instruct) language model, specialized for **role-play** scenarios. Despite its small size, it delivers impressive performance in generating engaging dialogues and interactive storytelling.
127
+
128
+ Developed by **Oxygen (oxyapi)**, with contributions from **TornadoSoftwares**, Oxy 1 Small aims to provide an accessible and efficient language model for creative and immersive role-play experiences.
129
+
130
+ ## Model Details
131
+
132
+ - **Model Name**: Oxy 1 Small
133
+ - **Model ID**: [oxyapi/oxy-1-small](https://huggingface.co/oxyapi/oxy-1-small)
134
+ - **Base Model**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
135
+ - **Model Type**: Chat Completions
136
+ - **Prompt Format**: ChatML
137
+ - **License**: Apache-2.0
138
+ - **Language**: English
139
+ - **Tokenizer**: [Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
140
+ - **Max Input Tokens**: 32,768
141
+ - **Max Output Tokens**: 8,192
142
+
143
+ ### Features
144
+
145
+ - **Fine-tuned for Role-Play**: Specially trained to generate dynamic and contextually rich role-play dialogues.
146
+ - **Efficient**: Compact model size allows for faster inference and reduced computational resources.
147
+ - **Parameter Support**:
148
+ - `temperature`
149
+ - `top_p`
150
+ - `top_k`
151
+ - `frequency_penalty`
152
+ - `presence_penalty`
153
+ - `max_tokens`
154
+
155
+ ### Metadata
156
+
157
+ - **Owned by**: Oxygen (oxyapi)
158
+ - **Contributors**: TornadoSoftwares
159
+ - **Description**: A Qwen/Qwen2.5-14B-Instruct fine-tune for role-play trained on custom datasets
160
+
161
+ ## Usage
162
+
163
+ To utilize Oxy 1 Small for text generation in role-play scenarios, you can load the model using the Hugging Face Transformers library:
164
+
165
+ ```python
166
+ from transformers import AutoModelForCausalLM, AutoTokenizer
167
+
168
+ tokenizer = AutoTokenizer.from_pretrained("oxyapi/oxy-1-small")
169
+ model = AutoModelForCausalLM.from_pretrained("oxyapi/oxy-1-small")
170
+
171
+ prompt = "You are a wise old wizard in a mystical land. A traveler approaches you seeking advice."
172
+ inputs = tokenizer(prompt, return_tensors="pt")
173
+ outputs = model.generate(**inputs, max_length=500)
174
+ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
175
+ print(response)
176
+ ```
177
+
178
+ ## Performance
179
+
180
+ Performance benchmarks for Oxy 1 Small are not available at this time. Future updates may include detailed evaluations on relevant datasets.
181
+
182
+ ## License
183
+
184
+ This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
185
+
186
+ ## Citation
187
+
188
+ If you find Oxy 1 Small useful in your research or applications, please cite it as:
189
+
190
+ ```
191
+ @misc{oxy1small2024,
192
+ title={Oxy 1 Small: A Fine-Tuned Qwen2.5-14B-Instruct Model for Role-Play},
193
+ author={Oxygen (oxyapi)},
194
+ year={2024},
195
+ howpublished={\url{https://huggingface.co/oxyapi/oxy-1-small}},
196
+ }
197
+ ```
198
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
199
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_oxyapi__oxy-1-small)
200
+
201
+ | Metric |Value|
202
+ |-------------------|----:|
203
+ |Avg. |33.14|
204
+ |IFEval (0-Shot) |62.45|
205
+ |BBH (3-Shot) |41.18|
206
+ |MATH Lvl 5 (4-Shot)|18.28|
207
+ |GPQA (0-shot) |16.22|
208
+ |MuSR (0-shot) |16.28|
209
+ |MMLU-PRO (5-shot) |44.45|
210
+