Justy: I burned maybe ten tokens on that pagination subagent and still shipped the wrong pagination. It didn’t even leave a footprint in the diff—just a green check that made me wonder what the hell happened.
Cody: Yeah, because the diff is the last ten seconds of the show. The trace? It’s the whole episode.
Justy: Exactly. We had one little cursor-based spec, the agent spawned off an offset-based helper from six months ago—no mention in the final diff, just a quiet incorrect branch.
Cody: MCP doesn’t always surface the tool call, so the trace tree swallows the typo. That’s the leak you’re chasing—obvious mistake hiding behind a pretty JSON schema.
Justy: Right—and I don’t want to care which agent I’m using today. Yesterday it was Cursor, today it’s Codex, tomorrow it’s Claude Code, and every time I have to remember a new mental model to debug. That’s not infra, that’s a zoo.
Cody: So LangSmith sells a single layer: same trace fields, same query syntax, same dashboard, no matter which coding agent wrote the code. They map Claude Code, Codex, Cursor, Copilot Chat, Pi, OpenCode, and DeepAgents Code into one canonical schema.
Justy: So even if the underlying agent emits OpenTelemetry shaped like a snowflake, I still see user turns, assistant turns, model calls, tool calls, shell commands, MCP activity, subagent invocations, errors, retries, timing, tokens, cost—all in one place.
Cody: Exactly. And when you zoom in, you find the exact moment the test failed, the agent read the assertion message, and edited the file instead of re-running—mutation, not iteration. The diff just shows the green check at the end.
Justy: That’s the hidden shortcut you miss if you swear by diffs only. Traces show the scaffolding that wasn’t torn down.
Cody: So the loop is reconstruct, debug, improve. Reconstruct the sequence, filter by thread_id, find the failure path, then turn the mistake into a rule—Skills you can run across every agent.
Justy: Which means your eval suite grows the moment you save the bad run. You can prove the fix holds and catch regression later. It’s not prompt tuning, it’s session tuning.
Cody: All good until the MCP layer decides to ghost the tool call from the trace tree. Their schema captures the start and end of the MCP invocation, but inputs, outputs, and token counts can still vanish.
Justy: So they haven’t solved the black box at MCP level. But at least the coding agent’s decisions live in one place instead of six different dashboard tabs. It’s progress.
Cody: Progress that still leaves you staring at a zero-dollar line item when the MCP tool gobbled a hundred tokens.
Justy: Okay, fine, but imagine you’ve got Cursor running one flow, Copilot Chat another, Codex a third—same repo, same regression pattern popping up—now you can ask one query: ‘show me the failing sessions by thread_id this week.’
Cody: Right, and you can compare behavior across agents on the exact same workflow. That’s the friction they’re removing—no more re-instrumenting every provider.
Justy: The part I’m sold on is the Skills export. Once you’ve fixed a pagination bug in one agent, you can ship that same Skill to every other agent that touches paginated endpoints. It’s not model enlightenment, it’s constraint sharing.
Cody: So you’re basically turning your worst runs into guardrails. That’s an infra win.
Justy: Wait—this is going to sound insane on a podcast, but if every coding agent in your stack is emitting traces, and LangSmith slurps them all into one schema—you finally have a single pane to ask the same question across providers instead of six.
Cody: Yeah, yeah. Just temper expectations: MCP still manages to veil itself.
Justy: Fair. So for teams already running multiple agents, this is table-stakes. For teams that don’t, it’s a good reason to start.
Cody: And for the rest of us? LangSmith’s post is basically the README for ‘how to stop debugging agents by diffs.’
Justy: Alright, practical next step—go flip the switch for Claude Code, Cursor, and Copilot Chat. Don’t rewrite your agents; just let LangSmith ingest the traces. Then filter for subagent fanouts and hunt the silent helpers.