arxiv:2609.01572

From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix

Published on Sep 1

· Submitted by

Alex Medvedev on Sep 2

T-Tech

Upvote

Authors:

Olga Tsymboi ,

Ramil Latypov ,

Danil Taranets ,

Nikita Gulyakov ,

Arthur Babkin ,

Aleksandr Medvedev ,

Anatolii Potapov

Abstract

A smaller self-hosted LLM trained with separate GRPO experts merged via SLERP outperforms a much larger baseline on instruction following, function-calling, and internal tasks while serving half of platform traffic at lower cost.

Generated by thinkingmachines/Inkling-Small

Data-residency constraints force enterprises to self-host LLMs, but continuous adoption of newer models without decommissioning their predecessors expands the serving fleet, fragmenting a finite GPU pool. We consolidate traffic from over 200 internal applications onto a single model by closing quality gaps identified through production error analysis along three axes: instruction following, function-calling, and internal task distribution. Quality is tracked by offline benchmarks stratified to production traffic and scored by deterministic verifiers or calibrated LLM judges. Rather than optimising all objectives jointly, which introduces cross-domain reward interference, we train a separate GRPO expert per axis and merge them via two-stage SLERP. Each expert's reward exposes a distinct failure mode, namely semantic collapse, over-calling, and verbosity hacking, each requiring a domain-specific fix. In non-reasoning mode the recipe surpasses a {sim}7times larger by total parameters baseline on the in-house Arena with 69.6 to 65.8, instruction following with 0.85 to 0.83, and function-calling with 0.79 to 0.77, while lifting general dialogue benchmarks. The model absorbs 50% of platform traffic, 116M requests per month, at a fraction of the serving cost.

View arXiv page View PDF Project page Add to collection

Community

kenkaneki

Paper author Paper submitter about 23 hours ago

We consolidated traffic from 200+ internal apps onto one self-hosted model, closing the gaps production error analysis showed: instruction following, function calling, and our internal task mix. Instead of one joint objective we train a GRPO expert per axis and merge with two-stage SLERP — each axis hacks its reward differently (semantic collapse, over-calling, verbosity hacking). Non-reasoning mode beats a ~7× larger baseline on our Arena (69.6 vs 65.8) and now serves 50% of platform traffic, 116M requests/month.

O96a

about 21 hours ago

Consolidating the whole corporate request mix onto one self-hosted model only pays off if the retraining loop closes the gap faster than the apps drift. In my experience the request mix shifts weekly — new tools, new prompts, new failure modes — and if your post-training cycle is a monthly batch job, you're always chasing last quarter's traffic. The number I'd want isn't coverage at snapshot time, it's the half-life of that coverage.

And I'd want to see the judge setup before trusting the quality numbers. "Calibrated LLM judges" is where these pipelines usually leak — if the judge was tuned on the same traffic you're optimizing for, you're measuring how well the model mimics the judge, not how well it serves the request. Show me the judge's disagreement rate with human raters on the hard tail, not the aggregate score.