Implicit regularization

Axel Böhm, 05. Jul 2023

In overparametrized models, where the training objective contains multiple global minima, the use of a specific algorithm, such as gradient descent, implicitly guides the solutions towards specific global minima. This implicit bias plays a vital role in the learning process by introducing effective capacity control that is not explicitly defined in the objective.