Glossary Entry

Hyperparameter

A configuration setting of a learning algorithm, such as tree depth or learning rate, chosen before training rather than learned from the data.

Models Training Optimization

Also called: hyperparameters, hyperparameter tuning

Seed source: Google ML Glossary

Hyperparameters are the levers you set before training begins: how deep trees can grow, how fast weights update, how strongly the model is regularized. They are distinct from parameters, which the model learns from data during training itself.

Because their optimal values depend on the dataset, hyperparameters are usually found by search: trying candidate values, scoring each configuration with a held-out set or cross-validation, and keeping the best. Techniques range from simple grid and random search to Bayesian optimization, which uses past trials to decide where to look next.