Instructions to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX # Run inference directly in the terminal: llama cli -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX # Run inference directly in the terminal: llama cli -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX # Run inference directly in the terminal: ./build/bin/llama cli -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Use Docker
docker model run hf.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
- Llama new
- LM Studio
- Jan
- vLLM
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
- Ollama
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with Ollama:
ollama run hf.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
- Unsloth Desktop
- Pi
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with Docker Model Runner:
docker model run hf.co/kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
- Lemonade
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Run and chat with the model
lemonade run user.Qwen3.8-27B-ROCmFPX-Q8_0-GGUF-Q8_0_ROCMFPX
List all available models
lemonade list
- Hermes Agent
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF:Q8_0_ROCMFPX" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
โ ๏ธ STOCK
llama.cppWILL NOT LOAD THIS MODEL
Q8_0_ROCMFPXis a ROCmFPX quant type โ it exists only incharlie12345/ROCmFPX, not upstream llama.cpp. Ignore the auto-generated "Use this model" commands above.๐ฆ 25.92 GiB, 8.28 bpw ยท โ tools 7/7 in BOTH thinking and non-thinking ๐ 25.07 tok/s with MTP on Ryzen AI MAX+ 395. โ ๏ธ The 4-bit build is still faster (38.32). โ ๏ธ Run it WITH the bundled MTP draft head โ without it you get ~7.9 tok/s, a third of the speed.
Qwen3.8-27B โ ROCmFPX 8-bit (Q8_0_ROCMFPX) GGUF
An 8-bit ROCmFPX quantization of Qwen3.8-27B for AMD gfx1151 (Ryzen AI MAX+ 395 / Strix Halo), built because 128 GB of unified memory makes 8-bit genuinely affordable on this hardware. Quantized from the 51.3 GiB BF16 GGUF โ not requantized from a lower-bit build.
| File | Qwen3.8-27B-Q8_0_ROCMFPX.gguf |
| Size | 25.9232 GiB (27,834,808,672 bytes) |
| BPW | 8.28 |
| ftype | Q8_0_ROCMFPX (111) |
| sha256 | 960978d5b230485c35c2456988082fa5d1e27a05ac74150604127b0e95b904bf |
โ ๏ธ If the MTP draft command crashes on your build
The separate-model
draft-mtppath has known bugs in the legacycharlie12345/ROCmFPXline โ reported on Windows 11 / gfx1151 / HIP SDK 7.2 (five stacked bugs, ending in an h-row width mismatch in the draft's embedding buffer).Fix: build the official repo instead โ no patches needed.
git clone https://github.com/ROCmFPX/ROCmFPX.gitThe MTP path was reworked there (unified
n_embd_outrow widths,t_h_nextnreset inllm_graph_result::reset(),ctx_otherwired centrally), which covers the whole chain. Reported and verified on that configuration: 24โ31 tok/s, coherent output, tool calling working. Legacy-line patch: PR #109.Linux builds on the legacy line are not known to be affected.
Benchmarks โ run this WITH the MTP draft head
Ryzen AI MAX+ 395 (gfx1151, 128 GB unified, ROCm 7.2.4), median of 3, warm-up discarded,
idle box, shipped serving flags: --spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf --spec-draft-n-max 4 -ngl 999 -fa on -fit off.
| build | size | decode WITH MTP | range | draft acceptance |
|---|---|---|---|---|
this (Q8_0_ROCMFPX) |
25.92 GiB | 25.07 tok/s | [25.07 โ 25.51] | 0.911 |
Q8_0_ROCMFPX_AGENT |
26.28 GiB | 26.62 tok/s | [26.61 โ 27.15] | 0.953 |
Q4_0_ROCMFP4_STRIX (4-bit) |
14 GiB | 38.32 tok/s | [37.91 โ 38.61] | 1.000 |
โ ๏ธ Without the draft head this model runs at ~7.9 tok/s โ MTP is worth 3.2ร here. The
draft head (mtp-Qwen3.8-27B-Q4_0.gguf) is included in this repo; use it.
Choosing: the 4-bit build is ~1.53ร faster and 12 GiB smaller. Take 8-bit for fidelity headroom, not throughput. And if you are running MTP, prefer the AGENT variant โ it accepts more draft tokens (0.953 vs 0.911) and is 6.2% faster despite being marginally larger.
llama-server -m Qwen3.8-27B-Q8_0_ROCMFPX.gguf \
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
--spec-draft-ngl 99 --spec-draft-n-max 4 \
-ngl 999 -fa on -fit off --jinja --ctx-size 32768
Verified
| check | result |
|---|---|
| 17 ร 23 | โ
391 |
| capital of Japan | โ
Tokyo |
| days in 2024 | โ
366 |
| tool calling โ thinking | โ 7/7 (multi-arg, nested-object, enum, declines, multi-turn, streaming, parallel) |
| tool calling โ non-thinking | โ 7/7 |
Per-tensor types (audited, 851 tensors)
output.weight Q8_0 ยท token_embd.weight Q8_0 ยท bulk TYPE_103 (ROCmFPX 8-bit layout).
๐ฉน Prompt caching with the MTP draft head โ fixed
Reported by a user of this repo: with --spec-type draft-mtp loaded, llama-server disabled
prefix caching entirely. Every agentic turn reprocessed the whole prompt. Reproduced here on an
8045-token stable prefix:
| config | prompt_n | cache_n | prompt_ms |
|---|---|---|---|
| no draft head | 519 | 7526 reused | 1 908 |
| draft head (the defect) | 8045 | 0 | 27 948 |
| draft head + this patch | 4 | 5101 | 100 |
279ร less prompt processing per turn, with MTP still drafting.
Root cause
The saved speculative state is the MTP boundary โ the target model's pre-norm hidden row at the cached prompt's exact end position. Any partial-prefix reuse would leave it describing a position that no longer exists, so the server demanded an exact full-prefix match and otherwise reprocessed cold, erasing its own context checkpoints on the way.
The fix
patches/mtp-prompt-cache-fix.patch (4 files, applies to 2809dc5) captures the speculative
boundary inside the context checkpoint (common_prompt_checkpoint::data_spec).
create_checkpoint runs between decode batches โ exactly where that boundary is valid โ so exact
state is saved and restored together with the KV, never rebuilt.
โ Two approaches were tried first and rejected: rebuilding the boundary from a zero-fill changed the model's output (deterministically, 3/3), and truncating the KV back to the reuse point is impossible here โ the bounded rollback window is 4 tokens against the 333 a real turn needs. Exact state restore is the only shape that preserves output.
Verification
Independently gated 10/10: same prompt cold vs warm, temperature 0, byte-identical every run,
with the cache genuinely engaged (cache_n=5101, not a vacuous pass). The output hash also matches
the unpatched build, so behaviour is unchanged. Fails closed โ an unreachable rollback logs
reason=spec-checkpoint-missing and cold-reprocesses rather than guessing.
Related upstream
This is the same family as open llama.cpp issues #20225, #19794 and #24055 โ checkpoints being invalidated on hybrid/recurrent models. This patch is not upstreamed; it is offered here as-is.
What was NOT measured
- No perplexity run, and no quality A/B vs BF16 or the 4-bit build. We show 8-bit is slower; we have not demonstrated it is better. If you need proof that 8 bits buys accuracy here, that measurement does not yet exist.
- An earlier revision of this card quoted 7.92 vs 13.84 tok/s. Those were measured without the MTP draft head and understated both builds; the table above supersedes them.
- No long-context testing (model supports 131,072).
- No coding/reasoning benchmark.
Base model licence inherited. Credit for the model goes to Qwen.
- Downloads last month
- 170
4-bit
8-bit
Model tree for kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-GGUF
Base model
Qwen/Qwen3.8-27B