Researched
Gradient Descent
Cauchy's method of steepest descent (1847) finds a minimum by repeatedly stepping downhill.
Open in the interactive tree →Cauchy proposed it for solving systems of equations in astronomy, and it became the workhorse of numerical optimisation. Combined with backpropagation, it trains almost all neural networks today.
Prerequisites
Unlocks
- Backpropagation1986Backpropagation computes the gradient that gradient descent follows