Pippa: Tyler, I think we have said active parameters versus total parameters maybe forty times and then just… sprinted away.
Tyler: Yeah. Guilty. We keep treating it like the label explains the mechanism, which is a very Exploring Next disease.
Pippa: It is. And it matters because people see a model with a trillion parameters and either panic or swoon. Sometimes both. But the number they saw might be the whole warehouse, not the amount of machinery running for each word.
Tyler: Good. Start there. Picture a huge library. Total parameters are every book on every shelf. Active parameters are the few books the librarian actually pulls onto the desk to answer this one question.
Pippa: I love that the model gets a librarian. Of course it does. Tiny cardigan, massive compute bill.
Tyler: Exactly. The library can be enormous, and that still does not mean the librarian reads the whole building every time. The distinction is stored capacity versus used capacity.
Pippa: Right.
Tyler: And before we go further, Pippa, parameters are the learned numbers inside a neural network. We did the full Overview on that in episode six ninety-three, but the one-line version is: training adjusts a giant pile of numbers until the model behaves usefully.
Pippa: So total parameters is just the full pile of learned numbers. All the books on the shelves.
Tyler: Yes. Active parameters are the subset involved in producing a particular output during inference. Inference just means the model is running after training, taking a prompt and generating tokens. Tokens are the little chunks of text the model predicts one after another.
Pippa: Okay, wait. If the parameters are the learned numbers, why would some of them be asleep? Isn't the model… the model?
Tyler: In a plain dense model, mostly yes. Dense means the usual path runs through essentially the whole network for each token. In the library picture, dense is like requiring the librarian to check nearly every shelf every time, which is expensive but simple.
Pippa: Mm-hm.
Tyler: Sparse models change that. Sparse activation was our episode six seventy-three topic, and the quick gloss is: for a given input, only some parts of the network light up. Same building, fewer rooms used for this specific job.
Pippa: And conditional computation is the decision to spend compute only where the input needs it, right? We did that Overview in seven twenty-three. The quick version is: the model has branches, and the input helps choose the branch.
Tyler: Yes. Active versus total parameters is one of the cleanest ways to see that idea in numbers. Total asks how big is the library. Active asks how many books hit the desk per generated token.
Pippa: This is the part I want to slow down on, Tyler. Per token. Not per whole chat.
Tyler: Correct. For large language models, the model predicts the next token, then the next, then the next. The active count is usually described for each of those prediction steps, because the routing decision can happen again at each step.
Pippa: So if I ask one question and the answer is long, the librarian keeps going back to the shelves for each little chunk.
Tyler: That is the right picture. And in a mixture-of-experts model, which means a model with many specialist subnetworks inside it, a small router chooses which experts handle each token. The router is the librarian deciding which shelves are relevant.
Pippa: Okay okay.
Tyler: Usually it does not pick all the experts. It might pick the top few. Top-k routing just means choose the best k options, where k is a small number like two or eight, depending on the design.
Pippa: So the headline model size can be huge because all the experts exist. But the runtime bill is tied more closely to the experts selected for the token.
Tyler: Look at you refusing to over-pitch it. I'm proud.
Pippa: Do not sound so shocked. I can be restrained for almost eleven seconds.
Tyler: That is actually the line. Small caveat: deployment also depends on hardware layout, memory bandwidth, batching, and routing overhead. The active number is useful, but it is not a magic price tag.
Pippa: Sure.
Tyler: Concrete examples help. Tencent Hy3 is reported as a two hundred ninety-five billion parameter sparse mixture-of-experts model with twenty-one billion active parameters. So the library is two hundred ninety-five billion learned numbers, but the desk work per token is much closer to twenty-one billion.
Pippa: And that is why the open-weight story got noisy around it. If you only say two hundred ninety-five billion, it sounds like an enormous thing to serve. If you include twenty-one billion active, the economics look different.
Tyler: DeepSeek V4 is an even louder version: one point six trillion total parameters, forty-nine billion active per token. That gap is the whole point. It is a very large library with a much smaller reading pile for each step.
Pippa: And Qwen3 two hundred thirty-five B A twenty-two B basically puts the distinction in the name. Two hundred thirty-five billion total, twenty-two billion active. It is almost like the label is trying to save us from ourselves.
Tyler: Right.
Pippa: Tiny off-topic complaint: model names now look like someone dropped a parts catalog into a blender. B, A, point two, Pro, Scout, Maverick. Somewhere there is a spreadsheet sobbing.
Tyler: The spreadsheet deserves it. But yes, the ugly names sometimes carry the useful number. The A twenty-two B part is telling you active parameters, and that is the number a cost-sensitive builder should notice.
Pippa: Back to the library before the spreadsheet unionizes.
Tyler: The router has a hard job. If it sends every token to the same expert, that expert gets overloaded and the others waste capacity. If it spreads tokens badly, quality drops. So training has to make the experts useful and make the routing balanced enough to run.
Pippa: So the librarian cannot just have one favorite shelf.
Tyler: Exactly. And there is another catch: moving data around can eat the savings. Sparse routing looks cheap on paper, but if the hardware spends too much time shuttling token representations to different experts, the real speedup shrinks.
Pippa: That feels like our old infrastructure disease. The architecture is clever, then the deployment stack says, cute idea, now pay the logistics bill.
Tyler: Yes. That is why active parameters are not enough by themselves. You want instrumentation. Total parameters are easy to count from the model. Active parameters vary by input and architecture, so you profile the model and measure what actually fires.
Pippa: And this is where I get allergic to marketing slides. If a model says trillion-scale, I want to know dense or mixture-of-experts, how many active per token, and whether the serving cost actually reflects that.
Tyler: Same. Also, not every workload benefits equally. Some tasks may need broader capacity, and some routing patterns are harder to optimize. Sparse activation gives you a lever. It does not repeal physics.
Pippa: Where does this stand now, though? Because I do not want someone walking away thinking active versus total is some weird footnote from old model cards.
Tyler: No, it is very live. In mid twenty twenty-six, mixture-of-experts is a normal way people scale large language models because it decouples stored capacity from per-token computation. The current work is about how sparse to make the model under memory and inference constraints, not whether the distinction matters.
Pippa: So not replaced. More like absorbed into the daily accounting of serious model design.
Tyler: Yes. And the newer work around budget-aware expert activation is basically asking: given a compute budget, how many experts should fire, and when? Same library, stricter desk space.
Pippa: That lands for me. The one thing I would remember is: total parameters tell you what the model owns. Active parameters tell you what it spends on this step.
Tyler: You were restrained for eleven seconds and then produced the sticker version.
Pippa: I earned one sticker. Also, please keep the cardigan librarian. That is the only model mascot we can afford.