Justy: A three B model putting up those numbers is rude, honestly.
Cody: Yeah. And the rude part is it's not even claiming magic general intelligence stuff. It's making a narrower bet that verifiable reasoning is compressible, which is a MUCH more defensible claim.
Justy: Right, and that matters because small models have been stuck in this weird bucket where they're either toy demos or cost-saving backups. This is more like, no, maybe a compact model can be the main thing for math and code if the task has a clear notion of right and wrong.
Cody: Mm-hm.
Cody: Their headline is VibeThinker-3B, built on Qwen two point five Coder three B, and the reported scores are kind of wild for that size. Ninety four point three on AIME twenty-six, eighty point two pass at one on LiveCodeBench v six, and then with their claim-level test-time scaling thing they push AIME to ninety seven point one.
Cody: I mean, I did laundry at midnight because I forgot I was out of socks, so we're operating a very serious research institution here. But the reason I buy this more than some splashy benchmark post is they actually frame the boundary pretty clearly. They're not saying three B beats giant models everywhere. They're saying math, coding, and other answer-checkable tasks may be a different capability class than broad world knowledge.
Justy: Yeah, that's the part I liked. The product read is not "replace the flagship assistant." It's more like, if I'm building a coding helper, a math tutor, maybe a solver in a workflow with strong verification, I suddenly have a much cheaper model tier that doesn't feel like a consolation prize.
Cody: Exactly.
Cody: Mechanically, the paper is really a post-training story. They take that base model and run a staged pipeline: curriculum-based supervised fine-tuning with synthesized and filtered data, then multi-domain reinforcement learning, then offline self-distillation, and then instruction-focused R L to tighten controllability.
Justy: And they call back to the one point five B version, right? That earlier one was basically proving small models could hold a reasoning thread at all. This feels like them asking where the ceiling actually is if you stop treating the small model like a side quest.
Cody: Right, right.
Cody: They also keep this Spectrum-to-Signal idea from the earlier work. Broad diverse supervision gives the model a spectrum of behaviors, then later stages try to isolate the signal that actually cashes out as reliable reasoning. The interesting extra piece here is Long two Short Math R L, where they explicitly try to reduce bloated chains of thought so the model stays accurate without rambling forever.
Justy: That part feels very shippable to me. If a small model can reason well and ALSO not burn tokens narrating its entire internal monologue, that's huge for latency and cost.
Cody: And the test-time trick is pretty specific too. Claim-Level Reliability Assessment seems to be a math-only style boost where the model assesses claims inside its own solution path, so on answer-verifiable benchmarks you can spend extra inference to catch local mistakes before finalizing.
Justy: So not a general-purpose magic reranker. More like a targeted "check your work" layer for domains where claims can be audited.
Cody: Yeah.
Cody: Which is why I don't think the big number jump with C L R should be read as free capability. It's extra test-time compute in a domain that cooperates with verification. Still useful, absolutely, but it's part of the system, not some pure base-model property.
Justy: That's fair. Though honestly, from a product angle, I don't care that much if the gain comes from the base weights or a smart inference wrapper. If the whole package gets me a small deployable solver that hits that band of performance, that's real.
Cody: And somehow the benchmark chart is on a vinyl sleeve. But back to the actual paper, I do appreciate that they admit the gap on knowledge-heavy stuff like G P Q A Diamond. That makes the central hypothesis more believable. Reasoning depth may compress well, while knowledge breadth still looks like a coverage problem that wants lots of parameters.
Justy: Which means the builder persona is pretty clear. Teams doing code generation, contest-style problem solving, maybe constrained STEM workflows, maybe on-device or low-cost backends. I would not read this as "great, now your tiny customer support brain knows everything." Different job.
Cody: My only real caution is methodology scope, not because it looks sloppy, just because frontier-level on verifiable tasks is a narrower statement than people will want it to be. The LeetCode out-of-distribution result is strong, the instruction score staying high is reassuring, but I'd still want more production-style evals with messy specs, partial credit, and tool use.
Justy: No, that's good skepticism, not your usual rain cloud thing. And I think the paper kind of earns that reading by being pretty disciplined about what it's claiming. It's basically saying compact reasoning cores are a real path, not the whole map.
Cody: Mm-hm.
Cody: Also, I could be wrong, but this connects nicely to that smaller-model-explorer conversation we had. Same general vibe: if the reward is verifiable, you can push a lot more capability out of small systems than people expect. The supervision signal is doing a ton of work.
Justy: And that's maybe the most fun part of episode five hundred six of our extremely normal hobby. You keep telling me scale isn't the only lever, and every few weeks some paper shows up and annoyingly proves you right.
Justy: Okay, go buy socks before you build a frontier reasoning core out of pure chaos. I'll go find more coffee.