Researched
Neural Scaling Laws
Kaplan et al. (2020) show that model quality improves smoothly and predictably with more data, parameters and compute.
Open in the interactive tree →The finding justified ever larger training runs, and DeepMind's 2022 Chinchilla study refined the balance of data and model size. It set the investment logic behind today's AI data centres.
Prerequisites
Unlocks
- Large Language Models2022Scaling laws guided the size of models and training data
- Open-Weight Language Models2023-2026
- AI Accelerators & Data Centers2026Scaling laws justify building ever larger training clusters