openai/gpt-oss-120b
Text Generation
•
120B
•
Updated
•
429k
•
•
3.21k
Fatihhan Gulec
fatihhangulec
·
AI & ML interests
None yet
Recent Activity
liked
a model
1 day ago
openai/gpt-oss-120b
liked
a model
1 day ago
openai/gpt-oss-20b
liked
a model
19 days ago
Ultralytics/YOLOv8
Organizations
liked
2
models
1 day ago
liked
a
model
19 days ago
liked
2
models
about 1 month ago
upvoted
2
articles
about 1 month ago
Article
Asynchronous Robot Inference: Decoupling Action Prediction and Execution
By
and 7 others
•
•
38
Article
Reachy Mini - The Open-Source Robot for Today's and Tomorrow's AI Builders
By
and 1 other
•
•
641
liked
3
models
3 months ago
reacted to
wolfram's
post with 🤗
3 months ago
Post
7446
Finally finished my extensive **Qwen 3 evaluations** across a range of formats and quantisations, focusing on **MMLU-Pro** (Computer Science).
A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
**Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
A few take-aways stood out - especially for those interested in local deployment and performance trade-offs:
1️⃣ **Qwen3-235B-A22B** (via Fireworks API) tops the table at **83.66%** with ~55 tok/s.
2️⃣ But the **30B-A3B Unsloth** quant delivered **82.20%** while running locally at ~45 tok/s and with zero API spend.
3️⃣ The same Unsloth build is ~5x faster than Qwen's **Qwen3-32B**, which scores **82.20%** as well yet crawls at <10 tok/s.
4️⃣ On Apple silicon, the **30B MLX** port hits **79.51%** while sustaining ~64 tok/s - arguably today's best speed/quality trade-off for Mac setups.
5️⃣ The **0.6B** micro-model races above 180 tok/s but tops out at **37.56%** - that's why it's not even on the graph (50 % performance cut-off).
All local runs were done with LM Studio on an M4 MacBook Pro, using Qwen's official recommended settings.
**Conclusion:** Quantised 30B models now get you ~98 % of frontier-class accuracy - at a fraction of the latency, cost, and energy. For most local RAG or agent workloads, they're not just good enough - they're the new default.
Well done, Qwen - you really whipped the llama's ass! And to OpenAI: for your upcoming open model, please make it MoE, with toggleable reasoning, and release it in many sizes. *This* is the future!
- 4 replies
liked
a
model
3 months ago
is it possible to inference on gtx 1600ti (laptop) with cuda or without cuda
1
#42 opened 3 months ago
by
fatihhangulec
liked
6
models
3 months ago
deepseek-ai/DeepSeek-Prover-V2-671B
Text Generation
•
685B
•
Updated
•
1.71k
•
•
807
JetBrains/Mellum-4b-base
Text Generation
•
4B
•
Updated
•
26.5k
•
414
microsoft/bitnet-b1.58-2B-4T
Text Generation
•
0.8B
•
Updated
•
4.63k
•
1.15k
XiaomiMiMo/MiMo-7B-RL
Text Generation
•
8B
•
Updated
•
27.3k
•
265
Qwen/Qwen3-235B-A22B
Text Generation
•
235B
•
Updated
•
153k
•
•
1.03k
nari-labs/Dia-1.6B
Text-to-Speech
•
Updated
•
99k
•
•
2.69k
liked
a
dataset
4 months ago