Vince: Ava, my week is mostly tabs I meant to close and one very suspicious spreadsheet. Perfect mood for admitting we say "in-context learning" constantly and may have never actually defined it like normal beings.
Ava: That is embarrassingly true. We toss it into every agent conversation like it is self-explanatory, then sprint away into context windows and tool calls. Very Exploring Next of us. Deeply unserious, technically correct, spiritually annoying.
Vince: And people keep nudging us on it, which is fair. If you hear "the model learns in context," that sounds like the model is updating itself. Like it got smarter forever because I pasted three examples.
Ava: Right, and that is the trap. The clean intuition is: imagine you hand an apprentice a little desk packet before a task. The packet has a few solved examples, maybe a format, maybe labels. The apprentice uses that packet while doing the next item, but when the packet is gone, you have not changed the apprentice’s long-term training.
Vince: Right.
Ava: That temporary packet is the prompt. In-context learning is the model using that temporary packet to infer the task pattern and apply it right now. No new training run. No permanent weight update. Just adaptation inside the current working text.
Vince: I like the desk packet. Also I’m picturing the apprentice with six highlighters and a tiny panic face, which is basically every enterprise workflow diagram.
Ava: The tiny panic face is load-bearing.
Ava: So the formal name is in-context learning, often shortened to I C L. A large language model, or L L M, gets examples inside the prompt and performs a new task by recognizing the pattern those examples imply. If I show three reviews labeled positive or negative, then give a fourth review with no label, the model can continue the pattern and produce the likely label.
Vince: Okay, so zero-shot versus few-shot lives inside this. Zero-shot is me saying, "classify this review," with no examples. Few-shot is me giving the desk packet with a handful of mini demos.
Ava: Exactly. Zero-shot can work if the instruction is clear and the model already has a strong pattern for that task. Few-shot often works better because the examples reduce ambiguity. They show what counts as positive, what counts as negative, what the output should look like, and sometimes the weird edge cases the instruction forgot to say.
Vince: And the crucial part is still: it did NOT retrain. The model didn’t go off and do homework.
Ava: Yes. When people say a model has learned from the examples, they mean behaviorally, within the prompt. The weights are the long-term parameters learned during training. Fine-tuning changes those weights with additional training data. In-context learning leaves them alone and uses the prompt as temporary evidence.
Vince: Mm-hm.
Ava: This rests on transformers, so quick gloss. We did the full Overview on transformer architecture in episode six thirty-eight, but for this episode: a transformer is the model design that processes a chunk of text by letting the pieces of that text influence one another as the model computes what comes next.
Vince: Which is why the examples in the packet are not just dead text sitting above the question. They are part of the thing the model is computing over.
Ava: Right. And the key sub-piece is attention. We did that in episode six oh seven. Attention is the mechanism that lets the model weigh different earlier tokens, meaning little text pieces, when deciding what matters for the current token it is about to produce.
Vince: Okay okay.
Ava: Then there is the context window, episode six hundred if someone wants the deep version. The short version is: the context window is the amount of text the model can consider at once. In our desk-packet picture, it is the size of the desk.
Vince: So if the packet is too big, pages fall off the desk. If it is badly organized, the apprentice can still see it, but might grab the wrong page.
Ava: Exactly. Last prerequisite: autoregressive generation, which we covered in episode six thirty. That just means the model generates one token after another, each time using the prompt plus whatever it has already written. It is not producing the whole answer in one magical blob.
Vince: I still want "magical blob" as an architecture diagram.
Ava: Please do not submit that to a standards body.
Ava: Now the step-by-step version. You put examples into the prompt: input, output, input, output. The transformer reads that whole sequence. As it predicts the answer for the new input, attention can connect the new case to the earlier cases and the output format, so the model can continue the pattern that fits the packet.
Vince: Sure.
Ava: And this is why format matters so much. If your examples use "Review colon" and "Sentiment colon," the model often keeps that structure. If your labels are terse, it may answer tersely. If one example is inconsistent, the model may treat that inconsistency as part of the pattern instead of an error.
Vince: This is where my product brain lights up, because the painful old version was: collect task data, fine-tune a model, deploy it, monitor it, maybe repeat. In-context learning says, for a lot of tasks, change the packet and you changed the behavior.
Ava: Yes, with a giant asterisk, but yes. It is faster and more flexible than training a custom model for every little classification rule. The asterisk is that temporary steering is brittle compared with a well-trained system when the task is complex, high stakes, or full of contradictions.
Vince: Give me the concrete sentiment one, because it’s boring in the useful way.
Ava: Sure. Prompt says: "Review: the setup was painless. Label: positive." Then: "Review: support took three days and solved nothing. Label: negative." Then maybe: "Review: the price is high, but the tool saved us hours. Label: positive." Now a new review says: "The interface is confusing, but the export worked." The model has to infer both the labeling task and the style of the answer.
Vince: And if I only wrote "classify this," it might still know what to do, but the examples remove a bunch of guessing.
Ava: Yes. That connects directly to the prompt engineering episode, six sixty-one. We said then that prompting is mostly reducing ambiguity and shaping the pattern the model completes. In-context learning is one of the reasons that is true. The examples are not decoration. They are the task definition in executable form, basically.
Vince: That "executable form" phrasing is going to make some engineer mad and some P M delighted.
Ava: A rare two-audience upset.
Vince: Come on, Ava, the apprentice needs the packet, the P M needs the packet, the engineer needs to complain about the packet. This is a complete organizational system.
Ava: And somehow still cheaper than the steering committee.
Ava: The order of examples can matter too. The model is sensitive to the local pattern it sees. If the last example is weird, or if labels are imbalanced, it can pull the answer toward that. Researchers argue about exactly how much is lookup, how much is pattern induction, how much is knowledge already stored in the weights, but for a newcomer the practical point is enough: the packet changes the model’s next behavior.
Vince: That shows up in stuff we’ve already looked at. The unified memory build was basically giving an agent a better packet about what mattered from prior interactions. Not permanent learning in the model weights, but better context at the moment of action.
Ava: Yeah.
Vince: Same with that agentic testing stack. The agent was more useful when the prompt had examples of how the test environment worked, what failure looked like, what the output should contain. Without that, it is just a very confident intern wandering through end-to-end tests with a clipboard.
Ava: And the moral-values classification example is another angle. If your examples overrepresent one cultural framing, the model can infer that framing as the desired rule. In-context learning does not magically make the examples neutral. It amplifies whatever task pattern the packet seems to imply.
Vince: Which is the uncomfortable product lesson. You can ship faster because you do not need retraining for every workflow, but your prompt becomes part of your product surface. The examples are product decisions.
Ava: That is where current production systems are interesting. You see teams stuffing domain rules, workflow examples, tool schemas, and output conventions into prompts for agents. Atlassian’s People Insights team has talked about treating AI more like a coworker inside an operating model. Rippling’s agent work leans on context around a huge company-data ontology. Those are not just "ask the chatbot nicely" situations. They depend on giving the model the right working packet.
Vince: And named model-wise, this is why people cared when models like GPT-four and IBM Granite showed stronger in-context behavior. If the same base model can handle support classification, summarization, translation, code suggestions, and internal workflow steps by changing the prompt, that is a very different adoption curve.
Ava: The scaling story matters there. In-context learning was much weaker in smaller language models and became more visible as models got larger. I’d be careful with the word "emergent," because it can turn into sparkle dust, but the empirical surprise was real: capability improved with scale in ways people did not hand-code task by task.
Vince: So it is not spooky little AGI goblin behavior.
Ava: No. Please retire the goblin. The less spooky read is that training on huge text corpora teaches the model many patterns of tasks, explanations, demonstrations, quizzes, code examples, labels, and completions. Then, at inference time, the prompt can activate a task-like pattern. That is not consciousness. It is a very powerful pattern-completion system using context.
Vince: The goblin has been reassigned to compliance.
Ava: That lands. I’d only add that "as if" is doing real work there. The behavior can be extremely useful, but it is still bounded by the model’s prior training, the size of the context window, and the quality of the packet you hand it.
Vince: And now we’re officially allowed to say I C L again without feeling like frauds. Ava, tiny apprentice goblin survives another Wednesday.