tetu-AI/Tetu-2

Têtu-2

A small French language model trained from scratch, with a custom architecture.

Têtu-2 is an independent research model built in October 2026. No weights from any other model went into it. Each word it writes gets a color for how sure it is. It also has a built-in calculator for the arithmetic it writes out.

It is not a reliable assistant: it is small and it is often wrong. Every number below was measured, and its limits are given with it.

Versions in this repository

Folder	Version	What it is
. (root, default)	Têtu-2 chat (v2.1)	Conversation model: everyday French chat plus step-by-step reasoning
base/	Têtu-2 base	Pre-trained model (6.23B tokens), for text completion or your own fine-tuning

Model summary

Developed by	tetu-AI (independent project)
Model type	Decoder-only language model, custom architecture
Parameters	702M in total, of which 115M do the computation for each word. The rest is a large memory that the model reads at almost no cost.
Context length	32,768 tokens
Languages	French first, English
Vocabulary	32,768 tokens (French and English, digits split one by one)
Precision	bfloat16 (model.safetensors), except the doubt head in float32
License	CC BY-NC 4.0 (non-commercial)

What it does:

• Doubt head. It estimates, for every word, the probability that the word is wrong. The chat script colors the words: green when sure, yellow when it hesitates, red when in doubt.
• Built-in calculator. When it writes « 48 + 27 + 66 = », the exact result is inserted.
• Fast. About 275 tokens/s on an RTX 5070 (our previous model, Têtu v1: 135).

Training

Stage	What
Pre-training	6.23B tokens, 18.5 h on one A100. Sources: FineWeb-Edu, FineWeb2-HQ (French), French Wikipedia, FineMath. At the end, some of our own generated data was mixed in; a program checked all of it.
Instruction tuning (v2.0)	Our own generators, checked by program: math, stories, multi-turn conversations that refer back to earlier turns, and concepts. Plus French conversation and code that was run to check it.
Chat tuning (v2.1)	Everyday French conversations written by Qwen2.5-7B-Instruct (Apache 2.0) acting as a teacher. Its reasoning answers were kept only when two samples agreed or the code ran.

Evaluation

All results come from measurements we had already made. Nothing was re-run for this page.

MGSM-fr (250 grade-school math problems, translated into French by humans)

Same 250 problems, greedy decoding, same answer checker for every model. Têtu-2 never saw these problems during training.

Model	Parameters	MGSM-fr
Qwen2.5-1.5B-Instruct	1,544M	48.4 %
Qwen2.5-0.5B-Instruct	494M	17.2 %
Têtu-2 chat (v2.1)	115M compute (702M total)	5.6 %
Têtu v1.1 (our previous model)	96M active (181M total)	4.8 %
Têtu-2 instruct (v2.0)	115M compute (702M total)	4.0 %
SmolLM2-135M-Instruct	135M	1.6 %
MobileLLM-R1.5-140M	140M	0.4 % *

* With our prompt, MobileLLM-R1.5-140M did not finish its answers within 768 tokens, so this score underestimates it.

• The calculator: the Têtu-2 scores use it. The other models answer without one.
• Têtu v1 reaches 11.2 % when it votes over 16 sampled answers. That vote was not run for Têtu-2.
• Reading the table: Têtu-2 beats the other models of its size, but it is far behind Qwen2.5-0.5B, which is about 4 times larger in compute and was trained on 3,000 times more tokens.

Chat quality (pairwise judge)

The judge was Qwen2.5-14B-Instruct. It saw the answers in both orders, on 250 held-out requests that no model learned from.

	Wins	Losses	Ties	p (sign test)
Têtu-2 chat v2.1 vs Têtu v1.1	189	26	35	10⁻³¹
Têtu-2 chat v2.1 vs Têtu-2 instruct v2.0	156	44	50	7·10⁻¹⁶

One caveat: the judge comes from the same family as the chat teacher, so it may prefer that teacher's style.

Our own generated tests

These tests use held-out phrasings. The calculator was on.

	Têtu v1.0	Têtu-2 instruct v2.0	Têtu-2 chat v2.1
Multi-turn follow-up (1,475 turns)	93.2 %	71.0 %	61.1 %
Concepts (350)	75.4 %	62.9 %	52.6 %
Template math (215)	79.1 %	64.7 %	64.7 %

Têtu v1, a much deeper model per word, stays better at following a conversation over several turns.

Base model: on the same held-out text windows, the validation loss is 2.680 for Têtu-2 base and 2.838 for our previous base model (Têtu v0.2, trained on about 2B tokens).

Limitations

• It makes things up. This happens especially on rare topics. Do not use it for medical, legal or financial information, or for any important decision.
• Its reasoning on new problems is weak. It can mix up units (60 km/h for 30 min → "1,800 km") or divide by the wrong number in an average.
• v2.1 sometimes confuses "I" and "you". Asked « what did I ask you then? », it may answer as if it had been the one asking.
• It was little trained on English conversation, and it is not a coding model.

Usage

The architecture is custom, so it does not load with transformers. Use the bundled inference code.

git clone https://huggingface.co/tetu-AI/Tetu-2 && cd Tetu-2
pip install -r requirements.txt
python chat.py                 # chat with v2.1; --cpu without a GPU; --no-calc to turn the calculator off

Text completion with the base model:

import torch
from tetu2.generate import Generator, load
model, tok = load("base", device="cuda")      # or device="cpu", dtype=torch.float32
gen = Generator(model, tok)
for piece, doubt in gen.stream("La capitale de la France est", max_new=40, temp=0):
    print(piece, end="")

License

CC BY-NC 4.0 (non-commercial). The fine-tuning data includes material under a non-commercial license and synthetic conversations. The pre-training sources (FineWeb-Edu, FineWeb2-HQ, FineMath) are ODC-By; Wikipedia is CC BY-SA. The chat teacher, Qwen2.5-7B-Instruct, is Apache 2.0.

Citation

@misc{tetu2_2026,
  title  = {Têtu-2: a small French language model trained from scratch},
  author = {tetu-AI},
  year   = {2026},
  url    = {https://huggingface.co/tetu-AI/Tetu-2}
}