Instructions to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Use Docker

docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Ollama
How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Ollama:
```
ollama run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
```
Unsloth Desktop

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Docker Model Runner:
```
docker model run hf.co/tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M
```

Lemonade

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.VeriLoop-E2-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

VeriLoop logo

VeriLoop E2 — GGUF

Official llama.cpp distribution of VeriLoop E2
BF16 reference · Q8_0 high fidelity · Q6_K overall sweet spot · Q5_K_M memory-quality sweet spot · IQ1_M minimum-footprint sweet spot

27B post-trained model for code, mathematics, and physics · 262K native context · Apache License 2.0
Developed by Tsinghua SIGS Robot Lab · Libo Wang

Parent Model · Technical Report · Evaluation Evidence · Riemann ζ Artifact

Model variants

Use case	File	Main size	BF16-paired retention
Default / overall balance	`VeriLoop-E2-Q6_K.gguf`	20.566 GiB	PPL parity within uncertainty; KLD 0.004409; Same top-p 98.204%
Memory-quality balance	`VeriLoop-E2-Q5_K_M.gguf`	18.965 GiB	PPL +0.4450%; KLD 0.006919; Same top-p 97.251%
Minimum footprint	`VeriLoop-E2-IQ1_M.gguf`	16.790 GiB	PPL +0.3191%; KLD 0.014357; Same top-p 95.870%
Lower-KLD low-footprint alternative	`VeriLoop-E2-IQ2_S.gguf`	16.799 GiB	PPL +0.3457%; KLD 0.014023; Same top-p 95.516%
Low-footprint alternative	`VeriLoop-E2-Q3_K_M.gguf`	16.826 GiB	PPL +0.4090%; KLD 0.014349; Same top-p 95.919%
Balanced compact	`VeriLoop-E2-Q4_K_M.gguf`	18.301 GiB	PPL +0.4821%; KLD 0.009700; Same top-p 96.786%
High fidelity	`VeriLoop-E2-Q8_0.gguf`	26.632 GiB	PPL +0.0643%; KLD 0.002176; Same top-p 98.815%
Reference	`VeriLoop-E2-BF16.gguf`	50.113 GiB	Canonical BF16 reference

Recommended deployment points

Q6_K — overall quality / efficiency sweet spot. It is 58.96% smaller than BF16 while remaining statistically consistent with BF16 PPL parity under the frozen paired protocol.

Q5_K_M — memory-quality sweet spot. It reduces the main-file footprint to 18.965 GiB while preserving wider KLD and Same-top margins than the sub-17 GiB variants.

IQ1_M — minimum-footprint sweet spot. It is 16.790078 GiB, 66.4955% smaller than BF16, and passes the frozen hard gate, engineering-reserve gate, stock llama.cpp runtime validation, and real MTP engagement validation. IQ2_S remains the lower-KLD low-footprint alternative.

Naming note: VeriLoop-E2-IQ1_M.gguf is a mixed-precision artifact, not a uniform 1-bit model. Its measured tensor policy is 353 F32 + 1 IQ1_M + 2 IQ2_S + 64 Q4_K + 429 Q5_K + 2 Q6_K = 851 tensors, with 5.36 effective BPW. VeriLoop-E2-IQ2_S.gguf is likewise mixed precision rather than uniform 2-bit quantization.

Precision ladder

Tier	Role	Main size	Reduction vs BF16	Effective density
BF16	Canonical reference	53.808 GB / 50.113 GiB	—	16-bit-class
Q8_0	High fidelity	28.596 GB / 26.632 GiB	46.86%	8.50 BPW
Q6_K	Overall sweet spot	22.083 GB / 20.566 GiB	58.96%	6.57 BPW
Q5_K_M	Memory-quality sweet spot	20.364 GB / 18.965 GiB	62.16%	6.05 BPW
Q4_K_M	Balanced compact	19.651 GB / 18.301 GiB	63.48%	5.84 BPW
Q3_K_M	Low-footprint alternative	18.067 GB / 16.826 GiB	66.42%	5.37 BPW
IQ2_S	Lower-KLD low-footprint alternative	18.037 GB / 16.799 GiB	66.48%	5.36 BPW
IQ1_M	Minimum-footprint sweet spot	18.028 GB / 16.790 GiB	66.50%	5.36 BPW

Quantization-retention benchmark

All measured tiers use the same frozen BF16 logits and the same paired protocol.

Tier	Mean PPL	PPL ratio vs BF16	Relative PPL change	Mean KLD	Same top-p	log-PPL correlation
BF16	4.840423 ± 0.119931	1.000000	—	0 reference	100% reference	100% reference
Q8_0	4.843536 ± 0.120062	1.000643 ± 0.000754	+0.0643%	0.002176 ± 0.000668	98.815 ± 0.120%	99.95%
Q6_K	4.838514 ± 0.119694	0.999605 ± 0.001222	−0.0395% observed; parity within uncertainty	0.004409 ± 0.000953	98.204 ± 0.147%	99.88%
Q5_K_M	4.861965 ± 0.120630	1.004450 ± 0.001361	+0.4450%	0.006919 ± 0.000945	97.251 ± 0.181%	99.85%
Q4_K_M	4.863760 ± 0.120720	1.004821 ± 0.001722	+0.4821%	0.009700 ± 0.001110	96.786 ± 0.195%	99.76%
Q3_K_M	4.860222 ± 0.120606	1.004090 ± 0.002181	+0.4090%	0.014349 ± 0.001985	95.919 ± 0.219%	99.62%
IQ2_S	4.857159 ± 0.120482	1.003457 ± 0.002100	+0.3457%	0.014023 ± 0.001408	95.516 ± 0.229%	99.64%
IQ1_M	4.855870 ± 0.120480	1.003191 ± 0.002175	+0.3191%	0.014357 ± 0.001317	95.870 ± 0.220%	99.62%

These figures measure quantization retention against the BF16 reference. They are not downstream benchmark-score loss percentages. The nine parent-model benchmarks were not independently rerun for every quantization tier.

IQ1_M validation record

IQ1_M changes exactly one tensor relative to the IQ2_S precision policy: blk.1.ffn_down.weight moves from IQ2_S to IQ1_M. All other tensor assignments remain unchanged.

Precision	Tensor assignment	Count
F32	Non-quantized tensors retained by GGUF conversion	353
Q6_K	`output.weight`, `token_embd.weight`	2
Q5_K	Remaining quantized internal tensors	429
Q4_K	all `*.ffn_up.weight`	64
IQ2_S	`blk.0.ffn_down.weight`, `blk.3.ffn_down.weight`	2
IQ1_M	`blk.1.ffn_down.weight`	1
Total		851

Fidelity and runtime validation

Validation	Requirement	Measured	Result
PPL ratio hard / reserve	≤ 1.0150 / ≤ 1.0135	1.003191	PASS / PASS
Mean KLD hard / reserve	≤ 0.0150 / ≤ 0.0145	0.014357	PASS / PASS
Same top-p hard / reserve	≥ 95.0% / ≥ 95.5%	95.870%	PASS / PASS
Exact tensor structure	851 tensors; declared policy	851 tensors; 0 mismatch	PASS
Stock llama.cpp main runtime	Successful real generation	HTTP 200, non-empty generation	PASS
MTP runtime	Successful real generation	HTTP 200, non-empty generation	PASS
Real MTP engagement	generated > 0; accepted > 0	76 accepted / 104 generated (73.0769%)	PASS
Main vs MTP deterministic audit	Advisory	Identical output SHA256	IDENTICAL

IQ1_M artifact identity

Property	Value
Filename	`VeriLoop-E2-IQ1_M.gguf`
Exact bytes	18,028,208,896
Binary size	16.790078 GiB
SHA256	`e4d395806994cdbcfecf7711e22456af7a663b3ea37c89ef15d5bab4571cf68b`
Effective density	5.36 BPW
Tensor count	851
Quantizer-reported size	17,182.55 MiB
Quantizer time	167.77645 s
llama.cpp validation revision	`42916d83f4a225e56709f873aa8050ac11f5b6a4`

Low-footprint comparison

Metric	Q3_K_M	IQ2_S	IQ1_M
Main size	16.825745 GiB	16.798508 GiB	16.790078 GiB
PPL ratio	1.004090	1.003457	1.003191
Mean KLD	0.014349	0.014023	0.014357
Same top-p	95.919%	95.516%	95.870%
RMS Δp	3.515%	3.679%	3.691%

IQ1_M is 8.633 MiB smaller than IQ2_S and 36.523 MiB smaller than Q3_K_M. IQ2_S retains the lowest Mean KLD of the three; IQ1_M has the smallest footprint, the lowest PPL ratio, and higher Same top-p than IQ2_S. The point-estimate differences remain small relative to the reported uncertainty scale.

Frozen BF16-paired protocol

Item	Value
Reference model	Canonical VeriLoop E2 BF16 GGUF
Corpus	WikiText-2 raw test
Corpus SHA256	`173c87a53759e0201f33e0ccf978e510c2042d7f2cb78229d9a50d79b9e7dd08`
Context	2,048
Chunks	8
Seed	42
GPU layers	40
Flash Attention	Off
KV cache	F16/F16
Batch / micro-batch	512 / 512
Evaluator	`llama-perplexity`
Reference logits	`--kl-divergence-base`
Quantized comparison	`--kl-divergence`
BF16 logits reused across tiers	Yes
llama.cpp revision	`42916d83f4a225e56709f873aa8050ac11f5b6a4`

Parent-model benchmark record

The scores below describe the parent VeriLoop E2 release and are not relabeled as quantization-specific reruns.

Benchmark	VeriLoop E2 parent score	Public evidence
SWE-bench Pro	76.2	Evidence
Terminal-Bench 2.1	88.8	Evidence
DeepSWE v1.1	64.6	Evidence
Terminal-Bench 3.0	29.7	Evidence
Terminal-Bench 4.0	37.9	Evidence
SWE-Marathon v1.1	45.0	Parent release
AIME 2026	98.3	Evidence
GPQA Diamond	93.9	Evidence
Apex 2025	89.6	Evidence

Quick start with llama.cpp

# Overall sweet spot
hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-Q6_K.gguf --local-dir .

# Minimum-footprint sweet spot
hf download tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF VeriLoop-E2-IQ1_M.gguf --local-dir .

# Run
llama-server \
  -m ./VeriLoop-E2-IQ1_M.gguf \
  -ngl 99 \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

Memory guidance

Variant	Main file	Positioning
BF16	50.113 GiB	Reference
Q8_0	26.632 GiB	High fidelity
Q6_K	20.566 GiB	Overall sweet spot
Q5_K_M	18.965 GiB	Memory-quality sweet spot
Q4_K_M	18.301 GiB	Balanced compact
Q3_K_M	16.826 GiB	Low-footprint alternative
IQ2_S	16.799 GiB	Lower-KLD low-footprint alternative
IQ1_M	16.790 GiB	Minimum-footprint sweet spot

A 16.79 GiB model file does not imply full offload on a 16 GiB GPU. Runtime memory also includes KV cache, compute buffers, allocator overhead, and optional speculative-decoding weights.

A standalone Q8_0 llama-bench record exists (pp512 3102.722776 tok/s; tg128 41.999677 tok/s), but there is no frozen paired BF16 throughput campaign. No universal speedup percentage is claimed for the quantization ladder.

Measurement boundaries

PPL, KLD, Same top-p, and token-probability statistics are quantization-retention measurements, not universal downstream capability-loss percentages.
IQ1_M and IQ2_S are mixed-precision artifacts; the tier name does not equal the model-wide effective bit width.
Native 262K context comes from the parent configuration; practical context depends on runtime memory.
MTP acceptance is prompt- and workload-dependent.
The model can still produce incorrect code, mathematics, scientific reasoning, or commands.

Source model and evidence

Resource	Link
Parent model	VeriLoop E2
GGUF repository	VeriLoop E2 GGUF
Quantization quality record	Quantization Quality
Artifact manifest	Release Manifest
Technical report	OpenReview
Evaluation evidence	VeriLoop E2 Evaluation Evidence
Riemann ζ artifact	Public artifact
llama.cpp	ggml-org/llama.cpp

License

The VeriLoop E2 model weights and this GGUF distribution are released under the Apache License 2.0, consistent with the parent model release.

Citation

@misc{wang2026veriloope2,
  title        = {VeriLoop E2: A 27B Post-Trained Model for Code, Mathematics, and Scientific Reasoning},
  author       = {Wang, Libo},
  year         = {2026},
  note         = {Tsinghua Shenzhen International Graduate School (SIGS)},
  howpublished = {Open model release},
  url          = {https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2}
}

For quantization-specific comparisons, identify the exact GGUF filename and corresponding manifest identity.

Downloads last month: 65

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

1-bit

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Model tree for tsinghua-sigs-robot-lab/VeriLoop-E2-GGUF

Base model

Qwen/Qwen3.8-27B

Finetuned

tsinghua-sigs-robot-lab/VeriLoop-E2

Quantized

(2)

this model