Justy: My coffee’s still too hot, and somehow this is already the weirdest token bill conversation I’ve had this week. But yeah, Cody, the whole JSON thing in LLM prompts feels like a very real annoyance right now.
Cody: Yeah, and I think the annoyance is legit. What’s overhyped is the idea that JSON is somehow the problem everywhere. JSON is fine in the backend. The issue is when you shove a hundred records into a prompt and keep paying for braces, quotes, commas, and the same field names over and over.
Justy: Right, but that’s exactly the user story I keep seeing. Somebody’s feeding support tickets, CRM rows, product catalogs, whatever, into a model and suddenly the prompt is bloated for no good reason. If TOON trims that down, it’s not glamorous, but it’s a pretty direct cost win.
Cody: Sure. TOON’s basic move is clever: declare the fields once, then stream the rows in a compact table-ish shape. So instead of repeating id, name, role three times, you get something like users{id,name,role}: and then the values. Same JSON data model, fewer tokens.
Justy: Mm-hm. And the article’s point that matters to me is that you don’t have to replace JSON in your app. You keep JSON where JSON belongs, and only convert at the edge when you’re about to send structured data into the model. That feels a lot more adoptable than asking teams to rewrite their whole pipeline.
Cody: Exactly. The adoption barrier is probably not the syntax itself. It’s the extra step and the question of whether your data is uniform enough to make it worth it. If your records are all over the place, or the payload is tiny, TOON doesn’t buy much. In some cases it could even be more annoying than the savings are worth.
Justy: Yeah, and that’s where product reality kicks in. Teams don’t want another format unless it clearly saves money or improves output quality. But for repeated stuff like tickets or analytics rows, the market is basically anyone doing serious LLM ops with a lot of structured context. That’s a real audience.
Cody: I see. And I do like that the article keeps saying lossless. That matters. If I can round-trip JSON to TOON and back without losing info, then this is a transport choice, not a data-model religion. That’s the sane framing.
Justy: Data-model religion is exactly how these things go sideways. Somebody finds a neat format and suddenly wants to baptize the whole stack in it. But if TOON is just the thing between retrieval and the model call, that’s a much cleaner story.
Cody: No way I’m signing up for a format cult. But the structure itself is pretty thoughtful. The model gets a clearer cue that, okay, these are rows with shared columns, not random prose and not a wall of JSON noise. That can help both token count and maybe parsing reliability.
Justy: Wait—so the question isn’t really, ‘Is TOON better than JSON?’ It’s more, ‘Is TOON better than JSON for this one narrow path where the model is reading a bunch of similar records?’ That feels like the honest version.
Cody: Yeah, that’s the honest version. And I’d add one more wrinkle: if your downstream prompt logic depends on exact field names and nested shape, you have to be careful about how you generate and validate the conversion. The format is compact, but your pipeline still has to prove it’s not mangling anything.
Justy: That’s the part I’d test first. I’d pick one ugly real payload, run JSON and TOON side by side, and compare token counts plus model behavior. If TOON saves a chunk and the output stays stable, that’s a nice weekend win. If not, you’ve learned something without wrecking the stack.
Cody: Yeah, and for a solo builder, that’s honestly enough. Grab the CLI, convert a sample dataset, and wire it into one prompt path. The article mentions the CLI, and that’s the right kind of low-friction start. Don’t rebuild your whole app. Just measure the actual delta.
Justy: I think that’s where we land, then. TOON sounds genuinely useful, but only in the places where repeated structure is eating your token budget. Everywhere else, JSON is still the boring winner, which is kind of comforting.
Cody: Mm-hm. Boring is good. I’d call TOON a sharp optimization, not a replacement. Useful when the shape fits, questionable when people try to stretch it past that.
Justy: Alright, I’m putting that on the list: one real dataset, one conversion, one token count comparison. And if it turns into a tiny format argument in Slack, well, that’s just another Tuesday.