Glossary Entry

Active Parameters

The parameters a sparse model actually computes with for one token, as opposed to the total it stores; the compute-cost side of the "X total, Y active" description of mixture-of-experts models.

Architecture Models

Also called: active parameters per token, activated parameters

Seed source: Mixture of Experts Explained (Hugging Face)

Active parameters count the weights that actually participate in one token’s forward pass. In a dense model this equals the total parameter count, but in a mixture-of-experts model only the selected experts run, so the active count can be a small fraction of the total: DeepSeek-V3 stores 671B parameters and activates 37B per token, about 5.5%.

The number matters because per-token training and serving FLOPs scale with active parameters while memory scales with total parameters. Two sparse models with the same total can have very different serving costs, which is why modern model names and cards quote both figures.