Justy: Okay, so Cameron Wolfe just dropped this piece on Agentic RL and the central claim is wild — we've been training models to answer questions, but now we have to train them to live in a loop.
Cody: Right.
Justy: Like, the agent isn't just the LLM backbone anymore. It's the whole harness — the tools, the environment, the instructions all running in this multi-turn trajectory where the model has to recover from its own mistakes.
Cody: Yeah, that's the key shift. He's saying single-turn completions are basically dead for real work. If your agent can't handle state changes across ten tool calls, it's not an agent, it's just a chatty API wrapper.
Justy: Exactly. And the way they're doing it now is encoding tool calls directly in the token stream. Qwen3 uses these XML-style delimiters — <tool_call> tags to trigger a stop, parse the call, execute it, then feed the result back as <tool_response>.
Cody: Mm-hm.
Justy: So the environment isn't some external black box anymore. The tool definitions ARE the environment dynamics. You're literally teaching the model the rules of the world via token delimiters.
Cody: Wait — are we actually still not over the harness thing from ep five fourteen? Because this feels like Akshay's thesis but now with RL math strapped to it.
Justy: Oh my god, you're right. 'Product is the harness' but now the harness is part of the loss function. That is such an Exploring Next take.
Cody: I cannot believe we're making this connection on a Monday morning. But yeah, technically it holds up — the problem is long-horizon tasks produce these super-long execution traces that break standard optimizers.
Justy: How so?
Cody: So most post-training uses GRPO, which does group-wise optimization. But when you compact a long trajectory into sub-traces, different rollouts under the same prompt give you wildly different numbers of trainable steps. You can't compare groups anymore.
Justy: Right.
Cody: That's why GLM five point two switched to PPO. They needed a critic to estimate token-level advantages instead of relying on group-relative comparisons. It's more stable when the agent has to reason over hours of tool calls.
Justy: Okay, but who actually cares about this? Like, is this for every team slapping an agent on their SaaS product?
Cody: No way. This is for the folks building agents that run autonomously for days — think coding agents that debug across multiple files, or robots planning cross-day tasks like that WorldLines benchmark we covered.
Justy: Right, right. If your agent only does one-and-done tasks, you don't need agentic RL. But if you're building something that has to maintain state, recover from errors, and coordinate across sessions? This is the infrastructure.
Cody: Exactly. And the memory piece is crucial here — he mentions memory becomes part of the environment, accessible via tool calls. So persistence isn't just a UX feature anymore, it's an RL constraint.
Justy: That's the product reframe. You're not just selling 'smarter models,' you're selling a harness that can survive its own mistakes over long horizons. The model is just the brain stem; the product is the loop.
Cody: I hate when you're right about the product angle. But yeah, the technical bottleneck is real — if your rollout infrastructure can't handle variable-length traces with stateful feedback, your agent will collapse.
Justy: So no generic 'add agentic RL' button then. This is deep infra work for teams ready to build the whole loop.
Cody: NOT a feature toggle. You either build the harness or you don't have an agent.
Justy: Okay, fair. Alright, I think we've tortured this paper enough for one morning. You want to grab lunch or are you gonna go re-read the GLM five point two appendix?
Cody: Lunch. Definitely lunch. I need to recover from all this optimism.
Justy: See you in ten.