Instructions to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M # Run inference directly in the terminal: llama cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M # Run inference directly in the terminal: llama cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M # Run inference directly in the terminal: ./build/bin/llama cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
Use Docker
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
- Ollama
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with Ollama:
ollama run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
- Unsloth Desktop
- Docker Model Runner
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with Docker Model Runner:
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
- Lemonade
How to use DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF:IQ2_M
Run and chat with the model
lemonade run user.DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF-IQ2_M
List all available models
lemonade list
- Atomic Chat
DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF
Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.
Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).
Model Summary
- Foundation Model:
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B(28 Layers, 28:4 GQA, SwiGLU FFN, Test-Time Compute Reasoning) - Quantization Precision:
~2.70 bpw(2.59 GiB) - Methodology: Sub-2.7-bit frontier test-time compute reasoning model compressed to 2.6 GiB. Runs at over 100 tokens/sec on consumer GPUs while maintaining coherent mathematical derivation chains.
- Continuous Holdout Perplexity (131k tokens):
3.7013 - GSM8K Native CoT Proofs:
24/25 (96.0%) - Average Thinking Trace:
394.8 tokens(</think>Closure Rate:96.0%) - Olympiad Competition Math:
5/10 (50.0%) - Python Code AST Execution:
0/10 (0.0%) - Inference Decode Throughput:
157.5 t/son NVIDIA GeForce RTX 4080 Super (32GB VRAM)
Theoretical Methodology
Extended chain-of-thought models rely on dynamic test-time compute, generating hundreds of intermediate deliberation tokens inside <think>...</think> scratchpads. Discretization noise from standard PTQ accumulates multiplicatively across intermediate reasoning steps, leading to proof derailment and infinite repetitive cycles.
DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: and projecting parameter updates strictly into the contractive Lyapunov half-space: G-TAP dampens back-reaction noise and preserves reasoning attractor basins across extended thinking sequences.
Usage Instructions
Native Prompt Template
DeepSeek-R1 models require the native prompt format to activate internal chain-of-thought:
<|begin of sentence|><|User|>{prompt}<|Assistant|><think>
Running with llama-cli
llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF -p "<|begin of sentence|><|User|>Solve for all real roots: x^4 - 4x^3 + 6x^2 - 4x + 1 = 0.<|Assistant|><think>\n" -n 2048 -c 8192 --temp 0.6 --top-p 0.95
Serving with llama-server
llama-server -hf DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF --port 8080 -c 8192 -ngl 99 -fa on
DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed
- Downloads last month
- 56
2-bit
Model tree for DuoNeural/DeepSeek-R1-Distill-Qwen-7B-GTAP-v3-IQ2_M-GGUF
Base model
deepseek-ai/DeepSeek-R1-Distill-Qwen-7B