@Banaxi-Tech stop hiding my comments. AND STOP STEALING PAPERS AND SPREADING MISINFORMATION. your BGA blog is a copy of NSA (deepseek, 2025) branded under your name. literally the same top16 selected blocks, 512 local window, router over block summaries, all you did was change block size from 64 to 128. you didnt cite NSA once but you put a “please cite BGA” bibtex at the bottom. i commented under your post and said that there is no way that you can support claims like: “The Accuracy Should BE WAy better than DSA but untested yet.” you didnt run a single experiment. and the 256x isnt from BGA, its just n/2k with k=2048 so the exact same k DSA uses. if opus wrote this for you, at least read it before posting. i commented again after you hid my comment despite it having constructive and correct feedback and you hid that too. and again. you can hide the truth and just try to get hf post likes..... but is it really the thing that needs to be done? do you really want to take papers and make them yours while barely even changing the params?
admitting your mistakes and doing something about them needs humbleness, intelligence, humanness. i encourage you to admit your mistakes and try to do better next time (at least read what blog your ai wrote or do proper experiments to back your stuff up).
Pretrained on 4x more tokens than the previous releases (20b vs 5b). Instruct tuned versions are coming soon. Very interesting models are coming soon too (hint: super long context).
Just hit #14 and #15 with out FIRST models on Open SLM Leaderboard. The models were trained on 5B tokens, while competing with similarly sized models trained on more than 6-20x the data.
A new base model Speck1.5-140M being trained right now on a higher quality corpus and will be released soon. SpeckChat3 is coming very soon with 1 million samples, specifically designed to post train small base models.
Also, just to clarify stuff, we will NOT release anything that is NOT MIT licensed EVER. Openness is needed in small language research.
Thanks to everyone supporting the project, and stay tuned for new releases!
SPECK UPDATES: 1 New instruct model tuned on top of Speck1-140M: specklabs/Speck1-140M-Instruct 2 Instruction tuning datasets 2 GGUFs
Much more coming soon: Speck1.1-140M-Instruct that is post trained on SpeckChat2 will be coming very soon New base model Speck1.5-140M is coming with a much higher quality corpus
Thanks to everyone who is already supporting the project, and stay tuned for new releases!
new models coming very soon (both instruct and much better models), with much much higher training scale as i am getting marenostrum5 access soon! we will be looking at 100b-2t token budgets :)
Existing methods — GPTQ, AWQ, llama.cpp's k-quants — minimize empirical loss heuristically. None of them prove they are optimal in any information-theoretic sense. ICRB-Q builds a quantization scheme that is provably optimal via the Cramér-Rao lower bound (CRB): no unbiased estimator of a weight can have lower variance than [F(θ)]⁻¹, where F is the Fisher information matrix.
Eduhelp with more empathy, based on model finetuned on psychotheraputic preferences just landed on
Beck-8B as a base model, 13000 steps on educational dataset. Time to go further and build more 🥰 s3nh/EduHelp_Beck_8B Thanks to @basilic_ai for computations <3
Just tried to create an educational assistant for younger people who can struggle with visualsation of 'what is this sorcery all about'. Its first step of my spare time projects, sft on Qwen3-8B,
EduHelper is a child-friendly tutoring assistant fine-tuned from the Qwen3-8B base model using parameter-efficient fine-tuning (PEFT) with LoRA on the ajibawa-2023/Education-Young-Children dataset.
Qwen 3 Coder is a personal attack to k2, and I love it. It achieves near SOTA on LCB while not having reasoning. Finally people are understanding that reasoning isnt necessary for high benches...
There seems to multiple paid apps shared here that are based on models on hf, but some ppl sell their wrappers as "products" and promote them here. For a long time, hf was the best and only platform to do oss model stuff but with the recent AI website builders anyone can create a product (really crappy ones btw) and try to sell it with no contribution to oss stuff. Please dont do this, or try finetuning the models you use... Sorry for filling yall feed with this bs but yk...
Small Language Models Enthusiasts and GPU Poor oss enjoyers lets connect. Just created an organization which main target is to have fun with smaller models tuneable on consumer range GPUs, feel free to join and lets have some fun, much love ;3
