DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF

Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.

Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).


Model Summary

  • Foundation Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B (28 Layers, 28:4 GQA, SwiGLU FFN, Test-Time Compute Reasoning)
  • Quantization Precision: ~2.70 bpw (2.59 GiB)
  • Methodology: Sub-2.7-bit frontier test-time compute reasoning model compressed to 2.6 GiB. Runs at over 100 tokens/sec on consumer GPUs while maintaining coherent mathematical derivation chains.
  • Continuous Holdout Perplexity (131k tokens): 3.7013
  • GSM8K Native CoT Proofs: 24/25 (96.0%)
  • Average Thinking Trace: 394.8 tokens (</think> Closure Rate: 96.0%)
  • Olympiad Competition Math: 5/10 (50.0%)
  • Python Code AST Execution: 0/10 (0.0%)
  • Inference Decode Throughput: 157.5 t/s on NVIDIA GeForce RTX 4080 Super (32GB VRAM)

Theoretical Methodology

Extended chain-of-thought models rely on dynamic test-time compute, generating hundreds of intermediate deliberation tokens inside <think>...</think> scratchpads. Discretization noise from standard PTQ accumulates multiplicatively across intermediate reasoning steps, leading to proof derailment and infinite repetitive cycles.

DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: Ωi=1d(∥H~i,:∥22−H~ii2)\Omega_i = \frac{1}{d} \left( \| \tilde{H}_{i,:} \|_2^2 - \tilde{H}_{ii}^2 \right) and projecting parameter updates strictly into the contractive Lyapunov half-space: Re⁡(λ(S))≤−δ\operatorname{Re}(\lambda(S)) \le -\delta G-TAP dampens back-reaction noise and preserves reasoning attractor basins across extended thinking sequences.


Usage Instructions

Native Prompt Template

DeepSeek-R1 models require the native prompt format to activate internal chain-of-thought:

<|begin of sentence|><|User|>{prompt}<|Assistant|><think>

Running with llama-cli

llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF -p "<|begin of sentence|><|User|>Solve for all real roots: x^4 - 4x^3 + 6x^2 - 4x + 1 = 0.<|Assistant|><think>\n" -n 2048 -c 8192 --temp 0.6 --top-p 0.95

Serving with llama-server

llama-server -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF --port 8080 -c 8192 -ngl 99 -fa on

DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed
Downloads last month
56
GGUF
Model size
8B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

2-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF

Quantized
(208)
this model