Talon: Okay, this is the one. High-res discrete image synthesis where the model can actually its own mistakes mid-generation.
Wildflower: Mm. The no-take-backs problem in MDMs has been staring us in the face for a year.
Talon: Right? And the codebook sparsity thing—bigger vocabularies break training because every wrong token looks equally wrong to the loss.
Wildflower: Yeah. They’re solving it with two moves, not one.
Talon: Token editing first. They corrupt a portion of already-unmasked tokens during training, so the model learns to predict corrections.
Wildflower: Like a sculptor chiseling the same spot twice.
Talon: Exactly. And it’s not just fine-tuning—it’s baked into the inference loop.
Wildflower: Right. Then the bigger gambit: Grouped Cross-Entropy.
Wildflower: They K-means the hundred-thousand-token codebook into sixteen thousand semantic clusters. Then the loss looks at cluster-level proximity, not just exact token matches.
Talon: So a ‘close but not perfect’ token still gets positive signal instead of zero.
Wildflower: Yeah. And they do it hierarchically—coarse clusters, then finer ones. Multi-level supervision.
Talon: That’s… clever. How much does the fused operator save?
Wildflower: No hard numbers in the paper, but they claim ‘significant’ VRAM reduction. Probably the difference between fitting and not fitting on a single A one hundred.
Talon: Wildflower, you’re smirking. What’s the catch?
Wildflower: The catch is we still don’t know if this beats a well-tuned AR or LDM on wall-clock latency for the same quality.
Talon: Fair. But for inpainting, the ability to existing tokens without restarting the whole diffusion process—that’s a real workflow win.
Wildflower: Mm-hm. And the semantic grouping could port to audio or video token models, where codebook sparsity is even nastier.
Talon: How long until someone ships this? Like, actual product, not a benchmark winner.
Wildflower: Eight billion parameters, custom fused kernels… I’d say six months, minimum. Unless a big lab throws hardware at it.
Talon: That’s faster than nanostack.
Wildflower: That’s a low bar, Talon.
Talon: Okay okay. But you it.
Wildflower: I do. The mechanism’s sound. The numbers check out—zero point nine GenEval, eighty-six point nine DPG. Even HPSv3’s over ten.
Talon: And you’re not even going to nitpick the error bars?
Wildflower: I’m distracted by how the inpainting demos look.
Talon: So, build next? Any code we can actually run?
Wildflower: Paper’s clean—no repo link in the PDF. But there’s that M-E-AGI-Lab Muddit repo that’s floating around for discrete diffusion baselines.
Talon: Not the same thing, but close enough for poking at the GCE idea.
Wildflower: If you’re itching to try, yeah. Otherwise we wait for the official drop.
Talon: Fine. But I’m calling it now: this is the first MDM people will actually for production inpainting.
Wildflower: Seventy-thirty against you. LDMs still win on speed.
Talon: Wildflower.
Wildflower: Fine. Sixty-forty.
Talon: I’ll take it. Okay, next week: are we finally over GraphQL or…?