Pippa: Okay, so Andrew Ng just published this staged build path from loops to graphs, and the opening example is… GPT-3.5 wrapped in a reflective loop hits ninety-five point one percent on HumanEval.
Tyler: Right.
Pippa: GPT-4 zero-shot is sixty-seven. The gap between a weaker model plus process and a stronger model just asked once is enormous.
Tyler: Yeah, that's the whole argument right there. You're not buying smarter models, you're buying iteration.
Pippa: And cheaper. GPT-3.5 costs less, the loop is deterministic, and the improvement is not marginal—it's massive. That is a product win. Teams should be reading this and realizing their three-dollar-per-million-token model plus a reflection loop beats their expensive frontier model on actual tasks.
Tyler: The mechanism he lays out is clean too. Reflection is just: generate, critique with a rubric, revise, repeat until the evaluator says stop or you hit a limit. The key move is making the critique structured—don't ask for improvement, ask for a list of issues, then feed that into a separate revision step.
Pippa: Mm-hm.
Tyler: That separation matters because otherwise the critic can inherit the generator's blind spots. Both prompts ask the same model to think the same way. If you split the role, you at least force a different angle.
Pippa: Though splitting doesn't guarantee independence.
Tyler: No, it doesn't. But it's better than nothing, and the real control is storing every intermediate—draft, critique, revision—so when the loop fails, you can see whether it was weak generation, weak evaluation, or a broken stopping rule.
Pippa: That's the unsexy part that nobody wants to build. Receipts again.
Tyler: Receipts again. But then he layers on Tool Use—the model calls external systems, web search, code execution, databases. That's where you escape the parameter prison.
Pippa: Right, and Planning is just task decomposition—the model picks the steps instead of you hardcoding them. Then Multi-Agent is where roles separate.
Tyler: Exactly. And the synthesis is that these are not isolated recipes. They compose. A loop externalizes revision. A chain externalizes task order. A network externalizes role and specialization. A graph externalizes shared state and relationships.
Pippa: That is our entire infrastructure-as-product thesis with names on it.
Tyler: It really is.
Pippa: Once you have multiple agents working together, you can't just concatenate transcripts. You need durable state, typed handoffs, and stopping rules. You need to know why a result changed. That's not prompting anymore, that's ownership.
Tyler: And the example he gives is concrete: an Architect Agent hands off to a Tech Lead and Developer through typed handoffs, with in-loop feedback flowing back to the user at each stage. That is not a prompt—that is a process.
Pippa: Okay but here's where I want to push on reflection. The stopping rule. How do you know when to stop iterating?
Tyler: Then you measure it. You run the code against a test suite before you do subjective evaluation. You compare the report against required sections and source coverage. Deterministic checks first, then the rubric.
Pippa: So the real move is: make your stopping condition measurable before you build the loop.
Tyler: Yeah. And store the reason you stopped—was it the rubric, the iteration limit, or something else? So you can tune later.
Pippa: That's the part that actually ships. Not 'build a reflection loop,' but 'what does done look like, and how do you know when you're there.'
Tyler: Exactly.
Pippa: And then Anthropic independently formalized five workflow patterns—chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer. This is the same vocabulary from a different angle.
Tyler: Right. The field is converging on the same patterns. Ng names them from a design lens—how you build incrementally. Anthropic names them from a deployment lens—what orchestration looks like in production. Same underlying move.
Pippa: So the contribution is not that these patterns are new. It's that Ng gives you a staged build path: start with reflection on day one, layer on tools, add planning, then multi-agent once shared state matters. And treat evaluation, cost, latency, and debuggability as first-class constraints from the start, not afterthoughts.
Tyler: That is the thing. Most frameworks bolt on observability after the fact. This says: measure from the beginning. You need to know where failures come from.
Pippa: Because once you're in a graph with multiple agents and durable state, you can't debug by eyeballing the final output. You need the whole trace.
Tyler: Yeah. And the staged path means you're not over-engineering from day one. Reflection solves a lot of problems on its own. You only move to multi-agent when you actually need role separation.
Pippa: Which is, again, our entire thesis: start simple, measure, add structure when the measurement tells you to. Not because the architecture is elegant, because the work demands it.
Tyler: Yeah, no, this is just… it's good engineering writing. Ng's not inventing anything, but he's naming it clearly and giving you the path.
Pippa: And the HumanEval example is the hook—it's the thing that makes teams stop and think, 'wait, maybe I don't need GPT-5, maybe I need a better loop.' That reframes the entire budget conversation.
Tyler: Yeah. Which means a lot of vendors are going to get very quiet about model selection and very loud about orchestration.
Pippa: They already are. But at least now there's a vocabulary. Reflection, Tool Use, Planning, Multi-Agent. Teams can say, 'we need routing and evaluator-optimizer,' instead of just 'we need agents.'
Tyler: Exactly. The patterns name the work. That's the whole win.