Instructions to use ManniX-ITA/gemma-4-26B-A4B-agentic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ManniX-ITA/gemma-4-26B-A4B-agentic with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="ManniX-ITA/gemma-4-26B-A4B-agentic") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ManniX-ITA/gemma-4-26B-A4B-agentic") model = AutoModelForMultimodalLM.from_pretrained("ManniX-ITA/gemma-4-26B-A4B-agentic", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ManniX-ITA/gemma-4-26B-A4B-agentic with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ManniX-ITA/gemma-4-26B-A4B-agentic" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/gemma-4-26B-A4B-agentic", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/ManniX-ITA/gemma-4-26B-A4B-agentic
- SGLang
How to use ManniX-ITA/gemma-4-26B-A4B-agentic with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ManniX-ITA/gemma-4-26B-A4B-agentic" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/gemma-4-26B-A4B-agentic", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ManniX-ITA/gemma-4-26B-A4B-agentic" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ManniX-ITA/gemma-4-26B-A4B-agentic", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use ManniX-ITA/gemma-4-26B-A4B-agentic with Docker Model Runner:
docker model run hf.co/ManniX-ITA/gemma-4-26B-A4B-agentic
Gemma-4-26B-A4B agentic
google/gemma-4-26B-A4B-it, trained to stop looping on long agentic tasks. Same architecture, same tokenizer and chat template, same vision tower (bit-identical), same voice: a BF16 recovery SFT on the model's own regenerated answers, filtered so that no trained turn loops.
GGUF (F16, Q8_0, Q6_K, Q4_K_M, imatrix; built with omnimergekit): ManniX-ITA/gemma-4-26B-A4B-agentic-GGUF.
Ollama: mannix/gemma4-26b-agentic (Q4_K_M, Q6_K, Q8_0; plain and vision- tags, plus mtp- tags for xOllama with the Gemma-4 assistant drafter and a thinking budget; the head-to-head sampler is preset).
The problem it fixes
Stock Gemma-4-26B-A4B-it is a strong agent until a task gets long. Then it can fall into a loop — the same paragraph of reasoning, or the same tool call, again and again until the context or the wall clock runs out. This model was trained to keep working instead.
Measured head-to-head against the base model on a long agentic repair task, 20 runs each with identical settings: 12 of 20 runs finished within the 30-minute budget against 3 of 20 for the base model, with reasoning loops cut about 6× less often. Details below.
Head-to-head with the base model
| base gemma-4-26B-A4B-it | this model | |
|---|---|---|
| runs finished within the 30-minute limit (20 each) | 3 / 20 (15 %) | 12 / 20 (60 %) |
| 95 % interval (Wilson) | 5 %–36 % | 39 %–78 % |
| passed within 10 / 20 / 30 min | 0 % / 15 % / 15 % | 25 % / 50 % / 60 % |
| reasoning-loop cuts per run (median) | 8.0 | 1.0 |
| reasoning-loop cuts per model-hour | 18.1 | 2.9 |
Pass/fail difference: Fisher exact two-sided p = 0.0079.
What the pass rate means. It is not an absolute success rate: it is the share of runs that finished the repair
within a fixed 30-minute budget. The test is built to measure what the fine-tune changes in a definite amount of
time — how much of the budget goes to loops, how quickly the model gets from a broken file to a working game. A timeout
means "not done in 30 minutes", not "cannot do it". With more time both models would pass more often and the gap would
likely narrow, but not by much: in extended manual testing of the base model and of its pruned 98-expert variants
(the v7-coder and v9-agentic recipes), a run that does not converge quickly becomes steadily more likely to loop, so
the base model either needs one to two hours or more to pass, or ends in massive loops. Anti-repetition sampling hides
the loops but costs the model much of its effectiveness and its ability to converge. The fine-tune targets exactly
this — less looping, faster convergence — and the data below shows it achieved. With only presence_penalty 0.025
(no other anti-repetition setting) it is virtually loop-free and completes the task in an acceptable time even on
runs where it does not converge quickly. The
pass curve below shows how the rate builds up minute by minute.
Setup. One task, run 20 times per model: repair a broken HTML game (manic_miner, a syntax error plus logic
bugs). The runs are orchestrated and scored by automaticum, a test harness
for coding agents whose oracle runs the game itself. The agent is the
Cerebriline SDK 4.100.277 (VS Code host profile, LSP diagnostics after every edit
and the check_file / ask_lsp tools on, browser off) until the game's own oracle reports that it loads, reaches play
and passes 23 logic checks. 30 minutes wall clock per run, one run at a time per model. Both models as BF16 GGUF on the
same llama.cpp build, each on its own RTX PRO 6000 Blackwell (96 GB), f16 KV cache, Gemma-4 assistant drafter for
speculative decoding, thinking on (effort high), temperature 0.8 / top_p 0.95 / top_k 64 / min_p 0 /
presence_penalty 0.025. The two configurations differ only in the model file: harness, task, sampler, server build,
chat template and guards are identical (checked from each run's recorded provenance). The harness guards were on for
both: a reasoning block that repeats itself is cut and the request retried, a repeated identical tool call draws
warnings, then a stop. Arms, server launchers, analysis scripts, the full report and per-run data:
omk/models/gemma-4-26b-a4b/sft11/eval/.
Outcome
| base | this model | |
|---|---|---|
| passed | 3 | 12 |
| failed: game never starts (syntax / runtime error left in the file) | 17 | 6 |
| failed: game starts, some logic checks still failing | 0 | 2 (17/23, 21/23 checks) |
| runs ended by the harness for repeating a tool call | 1 | 1 |
| runs that claimed success but failed | 0 | 0 |
| time to the first passing game check, median (passed runs) | 16.9 min | 11.5 min |
Share of all runs passed by minute t:
| 5 | 10 | 15 | 20 | 25 | 30 | |
|---|---|---|---|---|---|---|
| base | 0 % | 0 % | 0 % | 15 % | 15 % | 15 % |
| this model | 5 % | 25 % | 40 % | 50 % | 55 % | 60 % |
Loops and guards
Medians per run, all 20 + 20 runs; p = Mann–Whitney two-sided.
| base | this model | p | |
|---|---|---|---|
| reasoning blocks cut for repeating themselves | 8.0 | 1.0 | < 0.001 |
| guard interventions, all kinds | 9.5 | 4.0 | 0.001 |
| model requests cut or failed | 9.0 | 1.0 | < 0.001 |
| reasoning blocks with a phrase repeated ≥ 10× | 2.0 | 1.0 | 0.03 |
| worst repeat count in any reasoning block | 12 | 10 | 0.05 |
| repeated-tool-call warnings | 0.5 | 1.5 | 0.12 |
| runs with no reasoning-loop cut | 0 / 20 | 9 / 20 |
Per model-hour (sum over all runs ÷ total model time): reasoning-loop cuts 18.1 → 2.9, all guard interventions 24.8 → 17.7, repeated-tool-call warnings 6.1 → 9.9.
Time, length and effort
| base | this model | p | |
|---|---|---|---|
| wall time per run (min, 30 = timeout) | 30.2 | 21 | 0.05 |
| longest single model turn (s) | 213 | 108 | 0.001 |
| median model turn (s) | 22 | 9.1 | 0.006 |
| reasoning per run (k characters) | 226 | 230 | 0.95 |
| longest reasoning block (k characters) | 38 | 30 | 0.005 |
| generated tokens per run (k) | 121 | 128 | 0.56 |
| tokens written into tool-call arguments (k) | 12.1 | 22.6 | 0.17 |
| generated tokens per tool call | 3,866 | 2,100 | < 0.001 |
| peak context (k tokens) | 84 | 110 | 0.26 |
Same restricted to failed runs (all or nearly all of the 30 minutes used): reasoning 232k vs 416k characters, tool calls 24 vs 69, reasoning-loop cuts 9.0 vs 1.0.
Behaviour
| base | this model | p | |
|---|---|---|---|
| tool calls per run | 28 | 50 | < 0.001 |
| tool calls per model-minute | 0.9 | 3.3 | < 0.001 |
| edits applied | 5 | 10 | < 0.001 |
| edits refused by the tool | 3 | 7 | 0.004 |
| edits not followed by a check | 4 | 6 | 0.08 |
game runs (run_game) |
2 | 5 | < 0.001 |
read_files calls |
8 | 13 | 0.006 |
| LSP: edits checked automatically | 8 | 18 | < 0.001 |
LSP: check_file calls |
1 | 0 | 0.58 |
LSP: ask_lsp calls |
0 | 0 | 0.35 |
| runs that restored the original file and started over | 7 / 20 | 12 / 20 |
What changes. Total reasoning per run is about the same (226k vs 230k characters); how it is spent is not. The base model spends 3,866 generated tokens per tool call against 2,100, its longest turn runs 213 s against 108 s, and its reasoning repeats itself until the guard cuts it (8.0 cuts per run against 1.0). It makes 0.9 tool calls per model-minute against 3.3 and runs the game 2 times per run against 5, so it sees far less often what is still broken. All 17 of its failed runs ended with the game not even loading; 2 of this model's 8 failures got the game running and passing 17/23, 21/23 logic checks.
Known weaknesses
- Repeating a refused tool call. When an edit is refused, this model is more likely than the base model to send the same call again: more repeated-tool-call warnings in failed runs (median: base 0.0, this model 6.0; p = 0.010). Runs ended by the harness for repeating a call: base 1, this model 1.
- Degenerate tool-call bursts. In 2 of 20 runs this model emitted the same refused
editorcall 310 and 320 times in a single response, until the output-token cap. Both began with an edit whosestart_lineargument was dropped, so the tool refused it (the base model hit that refusal in 7 runs, this model in 7); the base model never answered it with a burst. Worth capping tool calls per response when running it in an agent loop. - Serving. This model's server generated 168 tok/s against 176 for the base model's (4 % slower on the same build and GPU model), which if anything works against it under a wall-clock limit. Speculative-draft acceptance was the same for both (90 % vs 89 %), so it is not the drafter; this model also worked at longer contexts (median peak 110k vs 84k tokens), which slows decoding.
- Scope. One task, 20 runs per model, one harness. The intervals above are wide; treat the size of the gap as approximate, its direction as solid.
Earlier gates
Before the head-to-head, the same task ran 10 times per checkpoint (automaticum, Cerebriline SDK 4.100.266) at temperature 1.0 (top_p 0.95, top_k 64, min_p 0), 30 minutes per run:
| base | step 400 | step 800 | final (this model) | |
|---|---|---|---|---|
| runs finished within 30 min | 1 / 10 | 4 / 10 | 5 / 10 | 5 / 10 |
| loops per model-hour (guard cuts + loop blocks) | 9.2 | 9.7 | 3.1 | 2.0 |
On SDK 4.100.276 with the recommended sampler, 7 of 8 finished runs passed. In a VS Code agent session on the same server it fixed the game in 3 of 3 sessions in 6–10 minutes, with no loops seen.
How it was trained
Built with omk-model-optimizer in its BF16 recovery mode (guide): the model's own bf16 weights are trained directly (no quantizer, no LoRA, no fp32 master copy), and the export writes the source checkpoint back with only the trained tensors replaced.
Data — mostly on-policy. Prompts, tool declarations and recorded tool outputs come from public SFT sets (9 NVIDIA Nemotron sets and interstellarninja/hermes_reasoning_tool_use). Every trained assistant turn was regenerated by the untouched Gemma-4-26B-A4B-it itself, turn by turn, in its own chat template with thinking on, and kept only when a validator passed it: no reasoning loop, no foreign tokens, a tool call equivalent to the recorded one (so the recorded tool output still answers it), the right final answer where a reference exists. One source is regenerated by the bigger sibling Gemma-4-31B-it (Kimi-K2.5 agent tasks from lambda/hermes-agent-reasoning-traces). The model learns to finish tasks in its own style rather than imitating another model's.
| corpus | |
|---|---|
| sources | agentic, opencode, hermes, ifchat v1/v2, kimi31b (Gemma-4-31B), swe, multilingual, science, math, competitive programming |
| size | 16,296 training conversations → 4,217 packed windows of 16,384 tokens (11.6 M scored tokens), one epoch |
| held out | 522 conversations (per-source CE every 200 steps) + 27 agentic windows scored against Gemma-4-31B |
| published | ManniX-ITA/gemma-4-26B-A4B-agentic-sft (conversations + the exact training windows) · ManniX-ITA/hermes-agent-kimi-traces-gemma4-31b |
Training.
| trained | attention, per-layer dense MLP, router and all routed experts: 295 tensors, 24.5 B parameters (embedding, norms, vision tower frozen) |
| loss | next-token cross-entropy on the regenerated assistant tokens only (reasoning, prose, tool calls, stop token); prompts, user turns and tool outputs masked; each packed conversation attends only to itself |
| schedule | 2,108 steps × batch 2 × 16,384 tokens, peak LR 4.74e-6 (10-step warmup, cosine), momentum-free Adafactor with stochastic-rounding bf16 updates |
| hardware | one RTX PRO 6000 Blackwell (96 GB), 74 GiB peak, 25.5 h |
| result | held-out CE on the corpus 0.281 → 0.263; agreement with Gemma-4-31B on held-out agentic windows unchanged (top-1 0.930 → 0.929) |
Full recipe, every script as run, corpus provenance (upstream revisions, sha256 of every file) and reproduce commands:
omk/models/gemma-4-26b-a4b/
in omk-model-optimizer. The three training modes (QAD to GGUF, ternary recovery, BF16 recovery) are documented in
omk/docs/.
Use
Exactly like google/gemma-4-26B-A4B-it (same chat_template.jinja, processor and generation config; thinking via
enable_thinking).
Recommended sampler for agentic work (the setting used for the head-to-head): temperature 0.8, top_p 0.95, top_k 64, min_p 0, presence_penalty 0.025, no repetition penalty.
from transformers import AutoProcessor, AutoModelForMultimodalLM
model_id = "ManniX-ITA/gemma-4-26B-A4B-agentic"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
vLLM: vllm serve ManniX-ITA/gemma-4-26B-A4B-agentic --dtype bfloat16 --enforce-eager --max-num-batched-tokens 8192 --reasoning-parser gemma4 --default-chat-template-kwargs '{"enable_thinking": true}' — the settings the training
corpus was generated with on vLLM 0.20.2 (--enforce-eager avoids a Gemma-4 MoE routing fault under CUDA-graph
capture seen on that version). Tool calls were parsed there with a hardened Gemma-4 parser plugin, published in the
recipe as sft11/corpus/sft2/gemma4_safe_tool_parser.py
(--enable-auto-tool-choice --tool-call-parser gemma4_safe --tool-parser-plugin gemma4_safe_tool_parser.py).
Limitations
- Evaluated on agentic coding behaviour (the head-to-head above) and held-out CE; no general benchmark suite has been run on this checkpoint yet. 8 of 20 head-to-head runs still did not finish within 30 minutes: reasoning loops are much rarer, not gone, and repeated refused tool calls are a known failure mode (see Known weaknesses).
- Trained on English-dominant agentic, coding and instruction-following data, with a small multilingual share; vision was not trained (the tower is the base model's, byte for byte).
License
Apache-2.0, as the base model. Training data licenses: see the corpus dataset card.
- Downloads last month
- 53