Instructions to use AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT") model = AutoModelForCausalLM.from_pretrained("AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT
- SGLang
How to use AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT with Docker Model Runner:
docker model run hf.co/AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT
ToolSure SFT (Qwen2.5 1.5B Instruct)
Small models break function calls: malformed JSON, invented function names, missing or mistyped arguments, and calls when no tool fits. Retrieval cannot fix that, because the failure is in how the model writes the call. ToolSure trains Qwen/Qwen2.5-1.5B-Instruct to call tools reliably. This model is one of six in the ToolSure collection: five training methods on the same data, plus constrained decoding, all on one table.
Method
LoRA supervised finetuning in bf16 on correct tool calls, plus examples where the right answer is plain text because no offered tool fits.
Training data: AmareshHebbar/ToolSure_Calls_Qwen2.5_1.5B. LoRA adapter merged into full bf16 safetensors; the adapter itself is in adapter/.
Why finetune instead of prompting
The table includes the base model prompted normally, the base model with a strict system prompt, and the base model with grammar constrained decoding. Constrained decoding guarantees valid JSON and real function names, but it cannot choose the right function, fill the right values, or decide when not to call. That gap is what training closes.
Results
All rows use greedy decoding and the same checker. BFCL columns are AST accuracy on the Berkeley Function Calling Leaderboard non live categories, scored with a reimplementation of the BFCL AST checker (function name, required parameters, no unexpected parameters, values against the possible answers; parallel calls matched in any order). Irrelevance is the share of prompts where the model correctly made no call. xLAM held out is exact call accuracy on 500 xLAM examples never used in training. JSON parse rate counts outputs that attempted a call. Latency is batch size 1 on vLLM; cost assumes $1.0/hour GPU at the measured batched throughput.
| Model | BFCL Simple | BFCL Multiple | BFCL Parallel | BFCL Parallel multiple | BFCL AST mean | Irrelevance | xLAM held out | JSON parse rate | Hallucinated function rate | MMLU | Latency p50 (s) | Cost per 1k queries (USD) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5 1.5B Instruct (prompted) | 91.5 | 86.5 | 80.0 | 79.0 | 84.2 | 64.6 | 84.2 | 99.2 | 0.7 | 0.21 | 0.0054 | |
| Qwen2.5 1.5B Instruct (strict system prompt) | 91.5 | 88.0 | 80.0 | 76.5 | 84.0 | 29.2 | 83.8 | 99.2 | 0.9 | 0.21 | 0.0051 | |
| Qwen2.5 1.5B Instruct (constrained decoding) | 92.0 | 87.0 | 80.5 | 79.0 | 84.6 | 63.7 | 83.2 | 99.6 | 0.0 | 1.21 | 0.0546 | |
| Qwen2.5-7B-Instruct (prompted, reference) | 94.5 | 93.5 | 89.0 | 85.5 | 90.6 | 67.5 | 86.8 | 99.8 | 0.5 | 0.73 | 0.0155 | |
| ToolSure SFT | 91.8 | 90.5 | 82.0 | 79.5 | 85.9 | 29.6 | 86.2 | 99.3 | 1.0 | 0.20 | 0.0044 | |
| ToolSure ORPO | 91.5 | 84.5 | 84.5 | 79.0 | 84.9 | 60.4 | 84.0 | 99.2 | 0.5 | 0.21 | 0.0045 | |
| ToolSure GRPO | 92.2 | 85.0 | 82.0 | 82.5 | 85.4 | 43.8 | 84.0 | 99.3 | 0.8 | 0.21 | 0.0044 | |
| ToolSure CONSTRAINED | 91.8 | 86.0 | 83.5 | 79.5 | 85.2 | 60.4 | 82.4 | 99.9 | 0.0 | 1.19 | 0.0621 |
Training hyperparameters
| init | base |
| mode | lora |
| lr | 0.0001 |
| epochs | 1 |
| micro_batch | 4 |
| grad_accum | 8 |
| warmup_ratio | 0.03 |
| max_len | 3584 |
| logits_chunk | 1024 |
| max_examples | 0 |
| lora_r | 64 |
| lora_alpha | 128 |
| lora_dropout | 0.0 |
Usage
import json
from vllm import LLM, SamplingParams
llm = LLM("AmareshHebbar/ToolSure_Qwen2.5_1.5B_SFT")
tok = llm.get_tokenizer()
tools = [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
messages = [{"role": "user", "content": "What is the weather in Paris?"}]
prompt = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
print(llm.generate([prompt], SamplingParams(temperature=0, max_tokens=512))[0].outputs[0].text)
Tool calls come back in the standard Qwen2.5 format, one <tool_call>{...}</tool_call> block per call, so any server that parses Qwen2.5 tool calls (for example vLLM with --tool-call-parser hermes) works unchanged.
Files
The repository root holds bf16 safetensors. adapter/ holds the LoRA adapter where one was trained, training/ holds logs and curves, and eval/ holds raw results and sample generations.
- Downloads last month
- 161

