Instructions to use end000/Clover-35B-A3B-Preview-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use end000/Clover-35B-A3B-Preview-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS
# Run inference directly in the terminal:
llama cli -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS
# Run inference directly in the terminal:
llama cli -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS
# Run inference directly in the terminal:
./build/bin/llama cli -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Use Docker

docker model run hf.co/end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Llama new
LM Studio
Jan
Ollama
How to use end000/Clover-35B-A3B-Preview-GGUF with Ollama:
```
ollama run hf.co/end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS
```
Unsloth Desktop

How to use end000/Clover-35B-A3B-Preview-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use end000/Clover-35B-A3B-Preview-GGUF with Docker Model Runner:
```
docker model run hf.co/end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS
```

Lemonade

How to use end000/Clover-35B-A3B-Preview-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Run and chat with the model

lemonade run user.Clover-35B-A3B-Preview-GGUF-IQ4_XS

List all available models

lemonade list

Hermes Agent

How to use end000/Clover-35B-A3B-Preview-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use end000/Clover-35B-A3B-Preview-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "end000/Clover-35B-A3B-Preview-GGUF:IQ4_XS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Clover-35B-A3B-Preview - IQ4_XS GGUF

An IQ4_XS quantization of DiffuSpace's Clover-35B-A3B-Preview, made directly from the original BF16 weights through a BF16 GGUF intermediate. No previously quantized weights were used. All credit for the model belongs to DiffuSpace and its upstream authors.

File	Quantization	Size
`Clover-35B-A3B-Preview-IQ4_XS.gguf`	IQ4_XS, default llama.cpp mixed tensor policy	18,942,280,960 bytes (18.94 GB)

The original Clover architecture and its additional bidirectional GDN tensors are preserved. This is the Clover diffusion model, not its original Qwen base model. GGUF weights are not restricted to Apple Silicon; execution requires a backend that implements Clover.

Required runtime

Use Clover support PR #31306, branch dev-clover from Maphist0/llama.cpp-pub, commit 4838ffb0c8ade92f34d9157deea38d92758ab824. This quantization does not imply compatibility with unpatched llama.cpp releases.

The separate local CUDA build targets compute capability 8.9, with CUDA Graphs and Flash Attention enabled. The binary is not included in this repository.

Quantization provenance

Source revision: 4b0402b65e523d0183a14c431955438714cb16ff.

python convert_hf_to_gguf.py ./Clover-35B-A3B-Preview-BF16 --outfile ./Clover-35B-A3B-Preview-BF16.gguf --outtype bf16
llama-quantize ./Clover-35B-A3B-Preview-BF16.gguf ./Clover-35B-A3B-Preview-IQ4_XS.gguf IQ4_XS 16

No importance matrix was used. SHA-256 and reproducibility metadata are included in SHA256SUMS and quantization.json.

Intended RTX 4070 12 GB / 64 GB RAM configuration

The following command uses a 128K total context, keeps expert weights from the first 32 layers in RAM, and places the remaining 8 expert layers and other supported layers on the GPU. The 8 GPU expert layers contain 3,264 MiB of quantized weights. Additional context and compute buffers also require VRAM. Loading with --load-mode none avoids memory-mapped CPU expert weights. Batch and microbatch are both 4,096.

llama-server -m ./Clover-35B-A3B-Preview-IQ4_XS.gguf --alias clover-35b-a3b -c 131072 -ngl all --n-cpu-moe 32 -fit off --load-mode none -fa on -ctk q4_0 -ctv q4_0 -b 4096 -ub 4096 -t 8 -tb 16 -np 1 --cache-ram 0 --ctx-checkpoints 0 --jinja --spec-type none --host 127.0.0.1 --port 8081

This repository does not provide a controlled quality or throughput evaluation. Memory fit and performance depend on the workload and other GPU users. The published artifact was checked for architecture, quantization metadata, tensor preservation and SHA-256 integrity.

The default command uses causal decoding with --spec-type none. Clover's draft-self implementation is experimental and is not a guaranteed speedup, particularly with CPU-resident MoE experts. At the pinned commit above, llama-server also needs a target-context initialization fix before draft-self can initialize: params_base.speculative.draft.ctx_tgt = ctx_tgt must be set independently of the separate draft/MTP-model branch. The default command does not require that fix.

License and attribution

Apache-2.0, as in the original model. The original LICENSE and ATTRIBUTIONS.md are included. Clover is derived from Qwen3.6-35B-A3B; see the original model card and attributions for provenance.

Downloads last month: 335

GGUF

Model size

35B params

Architecture

clover_moe

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for end000/Clover-35B-A3B-Preview-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

Diffuspace/Clover-35B-A3B-Preview

Quantized

(20)

this model