ymoslem/ModernBERT-base-TeleMath-router-qe-classifier-binary-10ep-lr2e-05-26b-5runs

Quality estimator for the Stage 2 cascade of CRE-Router on TeleMath, for Gemma4-26B (without reasoning). It gates clusters 2 and 3 of the E2EL routing at a budget of 120 s, whose Stage 1 sends them to Gemma4-26B; rejected answers escalate to Qwen3-30B-Thinking.

Given an efficient model's answer, it predicts whether to accept it or escalate to a stronger model. An answer is escalated when the probability of acceptance is below 0.5.

Input format

question [SEP] full_output [SEP] num_tokens

full_output is truncated to its last 1,000 words before tokenisation, and the tokeniser runs at max_length=4096. Using 2048 truncates roughly a fifth of these inputs from the right, which removes the final answer and the token count, the two things the classifier most needs.

Training

ModernBERT-base, 10 epochs, lr 2e-5, max_length 4096, on the train_m26 split of ymoslem/TeleMath-router. Labels are recomputed from the saved generation text with the current grader, so they carry no dependence on the answer parser that was in use when the generations were produced.

Intended use

Deployed only where Stage 1 gates to the corresponding tier. It is trained on that tier's own outputs and does not transfer to another model's outputs.