Cooper: Okay, the annoying part is this is actually a good benchmark headline.
Miles: Yeah. Because it's not just vibes or a cherry-picked demo. Snowflake ran one hundred and three tasks, three times each, and the code had to work in both DuckDB and Snowflake, which is a pretty specific failure trap.
Cooper: Right, and that's why I perked up. This isn't some abstract coding score. If your thing has to survive two SQL dialects at once, that's much closer to the mess people actually pay for.
Miles: Mm-hm.
Cooper: Also, my week is apparently just me reading pricing tables and getting emotionally invested in token economics, which feels bleak even for episode five sixty-five of this silly show.
Miles: It's a strong use of a Friday, honestly. But the central claim here is narrower than the article's broader drama. Snowflake's CEO is basically saying GLM-5.2 is competitive with Opus 4.7 on their hands-on code benchmark, while costing a lot less.
Cooper: And the evidence for that is the near tie once each model gets three shots, right? Sixty-six percent versus sixty-seven percent solved. That's close enough that a buyer at least has to look.
Miles: Exactly.
Miles: But the part that keeps me from doing the full cheap-model victory lap is first-attempt accuracy. Opus got fifty-three point seven percent. GLM got forty-seven point six. So if your workflow really values one clean pass, these are not the same model.
Cooper: Yeah, no, that's real. The article's own numbers kind of say Opus is better, just not better enough to justify the premium everywhere. Which, Miles, is very much my favorite category of product decision.
Miles: GLM also used more compute to get there. Average runs per task were ninety-nine versus eighty. Total tokens were eight hundred sixty million versus four hundred thirty-nine million. So the sticker price gap is huge, but the effective gap narrows once you account for how noisy the loop is.
Cooper: Right.
Cooper: Still, even after that, the math can work. If GLM is one dollar forty in and four dollars forty out, and Opus is five in and twenty-five out, you can afford a lot of mess before the cheap model stops being cheap. That's the practical argument, not some grand civilization thing.
Miles: And Snowflake's anecdote is actually useful there. They said GLM was the only one that could solve certain tasks because it was good at validating code across both platforms. That's interesting because it suggests the benchmark is partly measuring search behavior under tool use, not just code taste.
Cooper: Yeah.
Miles: Then you get the ugly counterexample. One task, GLM made four hundred eleven tool calls over twenty-four minutes, checking row counts, distributions, nulls, column types, and it still failed all three tries. Opus solved that same task in forty-nine calls and nine minutes. That's not elegance. That's a loop getting lost.
Cooper: Which is very episode five sixty-three of us. The model isn't enough. Your harness has to survive the model's mistakes. If the cheap model wanders, the real question is whether your system can fence that in before it burns the budget.
Miles: Exactly. This is why I don't read this as 'GLM is basically Opus.' I read it as 'for some coding workloads, the frontier is compressing and the control loop matters more.' Different claim. Better claim, honestly.
Cooper: I think that's also who should care. Not everybody. If you're doing occasional ad hoc prompting, whatever. If you're running lots of agentic coding or data transformation work at scale, especially with retries already baked in, this changes the shopping list.
Miles: Okay okay.
Cooper: Because then you're not buying a model in isolation. You're buying cost times reliability times latency times how annoying recovery is. And if you're already set up for retries, a cheaper model that's a little twitchier can be a GREAT deal.
Miles: I do want to push back on the article's leap from that to 'Western AI bubble under pressure.' Maybe! But one internal benchmark from Snowflake is not enough to cash that out. It's evidence of pricing pressure in a flagship category. That's different from proving revenue collapse.
Cooper: Yeah, that's fair. The market-stress-test stuff is where it goes a little wide. I buy 'pricing pressure is real.' I don't buy 'one benchmark means the whole stack is suddenly upside down.'
Miles: Also, I love when articles jump from 'this model needed twice the tokens' to 'therefore valuations tremble.' Like, man… maybe finish the cost accounting first.
Cooper: That's such your lane. But you're right. And I should say, this also rhymes with what I said when GLM-5.2 showed up on all those coding tiers. Availability plus giant context plus low price is a serious product move, even if the reasoning quality is uneven.
Miles: Cooper, your optimism is showing again.
Cooper: It is, but not blindly. I think the honest read is: if you have a tolerance-first setup, GLM gets more interesting fast. If you need high confidence on the first pass, you still pay for Opus and move on.
Miles: Yeah, I think that's basically where I land too. And I could be wrong, but I'd put maybe seventy-thirty on more enterprise teams running formal bake-offs like this over the next quarter, because the price delta is now too big to ignore.
Cooper: That's a good call. I won't add another one or we'll turn into those weird prediction people. Also, tiny detour, 'competitive at a fraction of the cost' is the most guaranteed way to get every vendor spreadsheet in the world opened at once.
Miles: A thousand tabs bloom instantly. Somewhere, someone is building a bar chart with very aggressive colors.
Cooper: All right, that's enough spreadsheet cinema for one day. Send me the benchmark when somebody does this with real latency budgets, and I'll happily go all the way down again.