Jessica: Okay, the part I liked was that the winner was basically… no winner.
Cathy: Yeah. And that's actually the honest version of this whole thing.
Jessica: Because the argument isn't "three models vote and truth pops out." It's that you already know each of these local models fails differently, so making one of them read the others before it answers can be better than pretending your default model is enough.
Cathy: Right.
Cathy: Jessica, that's also why the article works for me more than I expected. The author isn't claiming the council is smarter in some mystical way. He's saying he was manually doing cross-model adjudication already, and the software just formalized that workflow on his own box.
Jessica: Which is SUCH an Exploring Next thing to get excited about. Not a new model. Just "what if the annoying tab-switching part died."
Jessica: My week is apparently going great because I read that and thought, yes, automate the petty courtroom drama between small models.
Cathy: I mean… same, unfortunately. This is how we've spent almost eight months now.
Cathy: Technically, the interesting bit is the constraint. Karpathy's setup assumed cloud inference and parallel calls. The author points the app at Ollama's OpenAI-compatible endpoint, keeps the same three-stage flow, then immediately smacks into reality because three eight to nine billion-ish models on a single RTX forty seventy Ti with twelve gigabytes of V RAM was never going to run cleanly in parallel.
Jessica: Mm-hm.
Cathy: And the failure mode was useful. One model just dropped out with no response. That's exactly the kind of local-systems problem that tells you the abstraction looked portable, but the resource model absolutely was not. So he serializes the stages instead. Slower, but reliable.
Jessica: That part I buy completely. It's the same boring answer you keep liking. If the thing finishes every time, users will accept the wait for harder prompts. If one council member vanishes into the floorboards, nobody trusts the workflow.
Cathy: Exactly.
Jessica: The evidence in the piece is also kind of modest in a way I appreciate. He doesn't have some giant benchmark victory lap. He gives that Cloudflare Tunnel versus Pangolin example where the leaderboard ends in a three-way tie because each model ranked itself first, which is honestly very on-brand for language models.
Jessica: But then the useful part is downstream. DeepSeek had practical deployment detail, Qwen surfaced trade-offs in a clearer way, and Gemma stitched the whole thing together structurally. That's the actual claim.
Cathy: Yeah, no, that's the strongest part. The ranking mechanism itself sounds half-broken as a truth signal. If every model self-promotes, the "street cred" score is theater. But if the chairman mostly ignores the numeric ranking and reads the critiques, then you're really building a synthesis pipeline, not a vote aggregator.
Jessica: Right, right.
Cathy: And I do think the article overgeneralizes a tiny bit by wrapping that in council language. The council metaphor implies deliberation with some robust consensus logic. What he actually demonstrated is one synthesizer model benefiting from diverse drafts and reviews. Still useful. Just narrower than the branding.
Jessica: Come on, Cathy, let him have the tiny parliament. The models wore little ties in my head immediately.
Jessica: But you're right. Who should care is pretty specific. If you're just chatting locally, this is too slow. If you're already doing what he was doing, opening Gemma by default and then checking a harder prompt against DeepSeek or Qwen, then this changes something practical because it removes the human compare-and-merge step.
Cathy: No, that's exactly it. This is not for "replace your daily assistant." It's for prompts where being a little more right is worth extra latency. And it matches that thing we said with coding model selection, where you're buying reliability and recovery friction, not a model in isolation.
Jessica: Also, tiny callback, this feels adjacent to your MRAgent conversion. Not because it's the same mechanism. Just the pattern of, okay, the win isn't raw generation, it's orchestration around weak spots.
Cathy: Yeah. Different layer, same instinct. And I'd be careful not to oversell the hardware angle. Privacy and no API cost are real. But on one twelve-gig card, every extra stage is a tax. Ask me in six months whether local councils become a normal category. Right now I'd put maybe sixty-forty on "niche but sticky" for power users, not broad default behavior.
Jessica: I think that's fair. I might go a little higher for the workflow itself, not this exact repo. The repo's real, obviously, but the pattern has legs because people already don't trust one model to have the last word on high-stakes prompts.
Cathy: Oh, and for Build Next, there actually is something concrete here. The repo is Karpathy's L L M Council on GitHub, and the local trick is routing it through Ollama's OpenAI-compatible endpoint. Open WebUI was just the familiar front end before the author rebuilt the interface.
Jessica: Which, honestly, is the most relatable sentence in the whole article. "I rebuilt the UI so I could stand looking at this." Deep respect for that species of productivity spiral.
Jessica: Okay, episode six seventeen: you can keep your tiny model parliament, Cathy. I just want the chair to stop filibustering and ship the answer.