Neutrino-Instruct

A 7B-parameter instruction-tuned language model for conversational AI

Model card · Quickstart · Architecture · Limitations · License

---

Model Card

Field	Value
Developer	Fardeen NB
Model type	Autoregressive transformer, decoder-only
Parameters	7B
Base model	neuralcrew/neutrino
Fine-tuning	Instruction tuning for multi-turn dialogue
Language(s)	English
Format	GGUF (quantized for llama.cpp, Ollama, llama-cpp-python)
License	Apache 2.0
Version	2.0

---

Overview

Neutrino-Instruct is a 7B-parameter language model fine-tuned from the Neutrino base model for conversational and instruction-following tasks. It is distributed in GGUF format for efficient local inference on consumer hardware via llama.cpp, Ollama, and llama-cpp-python.

The model is designed to hold coherent, contextual dialogue across multiple turns and to follow natural-language instructions reliably at chat scale, while remaining light enough to run on a single consumer GPU or CPU-only machine.

---

Intended Use

Primary use cases

• Conversational assistants and chatbots
• Instruction-following agents (task completion, Q&A, summarization)
• Local/offline research prototypes where data cannot leave the device
• Educational and hobbyist LLM experimentation

Out of scope

• Medical, legal, or financial advice, or any use where model error could cause real-world harm
• Autonomous decision-making without human review
• Generation of content intended to deceive, impersonate, or manipulate
• High-stakes classification (e.g., hiring, credit, law enforcement)

Neutrino-Instruct is a general-purpose research and hobbyist model. It has not been evaluated for production or safety-critical deployment, and Fardeen NB makes no warranty as to its factual accuracy, safety, or fitness for any particular purpose.

---

Quickstart

llama.cpp

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Single prompt
./main -m ./neutrino-instruct.gguf -p "Hello, who are you?"

# Interactive chat
./main -m ./neutrino-instruct.gguf -i -p "Let's chat."

# Control output length
./main -m ./neutrino-instruct.gguf -n 256 -p "Write a poem about stars."

# Adjust temperature
./main -m ./neutrino-instruct.gguf --temp 0.7 -p "Explain quantum computing simply."

# GPU offload (if built with CUDA/Metal)
./main -m ./neutrino-instruct.gguf --gpu-layers 50 -p "Summarize this article."

Ollama

ollama run fardeen0424/neutrino

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(model_path="./neutrino-instruct.gguf")

response = llm("Who are you?")
print(response["choices"][0]["text"])

# Streaming
for token in llm("Tell me a story about Neutrino:", stream=True):
    print(token["choices"][0]["text"], end="", flush=True)

---

Training Data

Neutrino-Instruct's base model was pretrained on a mixture of web, encyclopedic, code, and long-document text, and instruction-tuned on conversational data. Component sources include:

Dataset	Type	Role
finepdfs	Long-form documents	Pretraining
finewiki	Encyclopedic text	Pretraining
fineweb-edu-100b-shuffle	Educational web text	Pretraining
wikipedia	Encyclopedic text	Pretraining
github-code	Source code	Pretraining
TinyStories	Short narrative text	Fine-tuning / eval
awesome-chatgpt-prompts	Instruction/persona prompts	Instruction tuning

Full dataset links are available in the metadata panel on the right side of this page.

Training procedure

Field	Value
Context length	32,768 tokens
Precision	bfloat16

---

Architecture

Neutrino-Instruct is built on a standard decoder-only Transformer stack. The core computations are as follows.

Scaled dot-product self-attention, applied per head:

$$\text{Attention}(Q,K,V)=\text{softmax}\text{\hspace{0.17em}⁣}\left(\frac{Q{K}^{\mathrm{\top }}}{\sqrt{d_k}}\right)V\backslash text\{Attention\}(Q,\; K,\; V)\; =\; \backslash text\{softmax\}\backslash !\backslash left(\backslash frac\{QK^\backslash top\}\{\backslash sqrt\{d\_k\}\}\backslash right)V$$

with multi-head attention combining $h$ heads via a learned output projection:

$$\text{MultiHead}(Q,K,V)=\text{Concat}({\text{head}}_1,\dots ,{\text{head}}_h)W^O,{\text{head}}_i=\text{Attention}(Q{W}_i^Q,K{W}_i^K,V{W}_i^V)\backslash text\{MultiHead\}(Q,K,V)\; =\; \backslash text\{Concat\}(\backslash text\{head\}\_1,\; \backslash dots,\; \backslash text\{head\}\_h)\backslash ,W^O,\; \backslash qquad\; \backslash text\{head\}\_i\; =\; \backslash text\{Attention\}(QW\_i^Q,\; KW\_i^K,\; VW\_i^V)$$

Neutrino-Instruct uses grouped-query attention (GQA): the 32 query heads are partitioned into $g=8$ groups, each group sharing a single key/value projection ($KW^K_g, VW^V_g$) instead of every query head having its own — trading a small amount of expressivity for a much smaller KV cache at inference time.

Position encoding — rotary position embeddings (RoPE), applied to query/key vectors by rotating consecutive coordinate pairs as a function of token position $m$ and frequency $\theta_i$:

$$f(x_m,m)=[x_m^{(1)}\cos m{\theta }_i-x_m^{(2)}\sin m{\theta }_i,x_m^{(1)}\sin m{\theta }_i+x_m^{(2)}\cos m{\theta }_i]f(x\_m,\; m)\; =\; \backslash big[x\_m^\{(1)\}\backslash cos\; m\backslash theta\_i\; -\; x\_m^\{(2)\}\backslash sin\; m\backslash theta\_i,\backslash \; \backslash \; x\_m^\{(1)\}\backslash sin\; m\backslash theta\_i\; +\; x\_m^\{(2)\}\backslash cos\; m\backslash theta\_i\backslash big]$$

Feed-forward block (SwiGLU-style gating):

$$\text{FFN}(x)=(\text{Swish}(x{W}_1)\otimes x{W}_3)W_2\backslash text\{FFN\}(x)\; =\; \backslash big(\backslash text\{Swish\}(xW\_1)\backslash otimes\; xW\_3\backslash big)W\_2$$

Pre-normalization (RMSNorm) around each sub-block:

$$\text{RMSNorm}(x)=\frac{x}{\sqrt{\frac{1}{n}\sum _{i=1}^n{x}_i^2+ϵ}}\odot g\backslash text\{RMSNorm\}(x)\; =\; \backslash frac\{x\}\{\backslash sqrt\{\backslash frac\{1\}\{n\}\backslash sum\_\{i=1\}^n\; x\_i^2\; +\; \backslash epsilon\}\}\; \backslash odot\; g$$

Training objective — standard next-token cross-entropy over the vocabulary:

Hyperparameter	Value
Layers	32
Hidden size	4096
Intermediate (FFN) size	14336
Attention heads	32
KV heads (GQA)	8
Vocabulary size	32,768
Max context length	32,768
Activation	SiLU (SwiGLU gating)
Normalization	RMSNorm (pre-norm, $\epsilon=1\text{e-}5$)
Position encoding	RoPE ($\theta=1{,}000{,}000$)
Sliding window	None (full attention)
Precision	bfloat16
Tied embeddings	No

---

Quantization & System Requirements

Setup	VRAM / RAM	Recommended quantization
CPU-only	32–64 GB RAM	Q4_K_M / Q5_K_M
GPU (entry)	4 GB VRAM	Q4
GPU (mid)	8 GB VRAM	Q5 / Q8
GPU (high)	12 GB+ VRAM	FP16 (full precision)

---

Limitations & Risks

• Hallucination: Like all LLMs, Neutrino-Instruct can generate plausible-sounding but incorrect or fabricated information. Outputs should be verified before use in any consequential context.
• Bias: Training data drawn from web and code sources may encode social, cultural, or representational biases present in that data. The model has not undergone a formal bias audit.
• Safety tuning: No dedicated safety alignment (red-teaming, RLHF/DPO refusal training, etc.) has been performed beyond the base instruction tuning. The model may comply with harmful, unsafe, or policy-violating requests.
• Language coverage: English only; behavior on other languages is unevaluated and likely degraded.
• Context length: Supports up to 32,768 tokens; quality may degrade toward the upper end of that window, as is typical for RoPE-based models — this has not been separately verified for Neutrino-Instruct.
• No factual grounding: The model has no live access to external tools, retrieval, or the internet; all knowledge is frozen at training time.

This model has not been evaluated by a third party or independent safety team. Users deploying it in any user-facing product are responsible for their own safety evaluation, content filtering, and monitoring.

---

License

Released under the Apache License 2.0. See LICENSE for full terms.

---

Citation

@misc{fardeennb2025neutrino,
  title        = {Neutrino-Instruct: A 7B Instruction-Tuned Conversational Model},
  author       = {Fardeen NB},
  year         = {2025},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/neuralcrew/neutrino-instruct}
}

Acknowledgements

Built on top of the Neutrino base model. Thanks to the maintainers of the open datasets listed above, and to the llama.cpp, Ollama, and llama-cpp-python projects for inference tooling.